Skip to content

Build queries only in the Ecto adapter - #2338

Draft
Flo0807 wants to merge 1 commit into
developfrom
feature/no-ecto-outside-adapter
Draft

Flo0807 wants to merge 1 commit into
developfrom
feature/no-ecto-outside-adapter

Conversation

@Flo0807

@Flo0807 Flo0807 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Stack (merge in this order)

  1. Build queries only in the Ecto adapter #2338 Build queries only in the Ecto adapter, based on develop ← this PR
  2. Let the adapter turn filter conditions and metric aggregates into queries #2340 Let the adapter turn filter conditions and metric aggregates into queries
  3. Configure filters in the LiveResource like fields #2431 Configure filters in the LiveResource like fields
  4. Describe the resource in the adapter and cast expressions with it #2448 Describe the resource in the adapter and cast expressions with it
  5. Read through a Backpex.Query #2449 Read through a Backpex.Query
  6. Apply the parts of a query an adapter doesn't run in memory #2450 Apply the parts of a query an adapter doesn't run in memory
  7. Write through neutral changes and let adapters declare writes #2451 Write through neutral changes and let adapters declare writes
  8. Replace item_query, :select and :custom_alias with scope, load and computed values #2454 Replace item_query, :select and :custom_alias with scope, load and computed values
  9. Compute aggregates of related items #2455 Compute aggregates of related items

Backpex is Ecto-first, but Ecto leaked into the core: the index view built the metric queries with Backpex.Adapters.Ecto.list_query/4 and Ecto.Query, the form view called __struct__ and the changeset functions from the adapter config, default_attrs/3 and the association fields inspected schemas, BelongsTo, HasMany and HasManyThrough built and ran their own option queries, and the Delete item action matched Postgrex.Error and Ecto.ConstraintError. Now only the adapter builds queries, calls the repo and inspects schemas. Everything else goes through Backpex.Adapter callbacks, mostly via Backpex.Resource. Changesets stay the currency of forms.

Based on #2337, which this PR moves into the adapter.

New adapter callbacks

  • metric/5: the data of a metric. The Ecto adapter still calls c:Backpex.Metric.query/3.
  • new_item/2: the item of the :new form.
  • association/2: cardinality, owner key and through associations of an association, for default_attrs/3, BelongsTo and HasManyThrough.
  • list_options/4, count_options/4 and the optional list_options_by_key/4: the items an association field can select, with :ids, :search, :offset and :limit. list_options_by_key/4 keeps the index forms of BelongsTo at one query per distinct options query. The Ecto adapter applies :ids after the :options_query, so the index form of BelongsTo only saves an option of its select, even if the options query ignores or widens (or_where) the query it receives. It matches :ids against the primary key of the associated schema, not a hardcoded :id.
  • put_assoc/4: used by the form and index views and HasMany.

Other changes

  • The criteria no longer carry Ecto schemas: searchable fields lose :queryable and the order loses :schema. The Ecto adapter resolves them from the fields.
  • Field.schema/2 and Field.search_condition/3 are optional callbacks that only the Ecto adapter calls. use Backpex.Field no longer injects them or import Ecto.Query.
  • Search casts every column that is not a string in the schema to text, and every column the schema doesn't know, e.g. of a join in the item_query under a :custom_alias. Number and Currency did this with their own search_condition/3, which is gone.
  • delete_all/2 returns {:error, :foreign_key_violation}. Backpex.Resource.delete_all/4 passes it on without broadcasting, and the Delete action shows the existing "used elsewhere" message.
  • LiveResource.search_options/2 (was /3) and Backpex.Adapters.Ecto.apply_criteria/4 (was /3).
  • test/backpex/adapter_boundary_test.exs walks the AST of lib/ and fails on Ecto queries, repo calls, __schema__/__struct__, adapter_config/1, Postgrex or Backpex.Adapters.Ecto outside the adapters. Docs are strings, so mentions there don't count. The query callbacks of filters and metrics are allowed to use Ecto.Query and the repo, as they belong to the Ecto adapter. Run against develop, it reports every leak listed above.
  • The v0.25 upgrade guide covers custom adapters, custom fields, the BelongsTo :queryable assign, the HasMany options query and the HasManyThrough :association assign, before_changeset on form mount, search and delete_all/4.

Merged with develop (v0.23.0)

develop gained features that built queries or inspected schemas outside the adapter. They keep working through the adapter:

  • New optional adapter callback options_key/3: a term that changes when the options of an association field may change. HasMany and HasManyThrough load their options again when it changes, e.g. when the :options_query depends on a value of the form (Load HasMany options again when their options query changes #2329). The Ecto adapter returns the options query; the fields only compare the term and never look into it. Without the callback, the fields load their options once.
  • New optional adapter callback field_type/2: the primitive type of a field. Backpex.Filters.Range uses it via Backpex.Resource.field_type/2 to refuse decimal boundaries of a :number range on an integer field, instead of reading the schema.
  • Backpex.Resource.metric/5 calls Backpex.Metric.data/2 (Let metrics get their data from the assigns of the index view #2347) if the metric implements it, and the adapter otherwise.
  • The index view takes the validated search from the query options (Ignore search params that are not a string #2381) and passes it to search_options/2.

Merged with develop (v0.24.0)

The relation fields of v0.24.0 check submitted values against their options query and filter the form params with param_keys/2. Both now go through the adapter:

  • association/2 also returns the :related_key the owner key refers to and the :primary_key of the related items.
  • list_options/4 takes a :key criterion for :ids and keeps a limit or offset of the options query in a subquery, so the check also holds for a limited options query.
  • BelongsTo, HasMany and HasManyThrough compare submitted ids as strings instead of casting them with the schema.
  • InlineCRUD reads the primary keys of the rows from the hidden inputs of the nested forms, which also covers the changeset of a resource action.
  • The upgrade notes of this stack moved to guides/upgrading/v0.25.md.

Not in this PR

  • Callbacks don't get a context struct yet; they keep taking assigns.
  • The :select, :options_query and item_query options and the query callbacks of filters and metrics still take Ecto queries. They're documented as part of the Ecto adapter.
  • No MySQL or SQLite dialect. The search cast and NULLS FIRST/LAST now live in one place in the adapter.

Testing

  • Merged with the rebased Test every untested option of LiveResources and fields #2436 (2026-10-28), so the stack builds on the test infrastructure of Move the feature tests into the library and measure coverage #2435: mix test incl. integration_test/ 1369 passed (167 doctests), mix compile --warnings-as-errors, mix format --check-formatted, mix credo and mix deps.unlock --check-unused pass. The recording adapter of Test every untested option of LiveResources and fields #2436 delegates the new adapter callbacks, and its BelongsTo form test gets the options from an adapter instead of a repo.
  • Library, merged with current develop: 811 tests and 167 doctests (incl. test/doc_test.exs and the adapter boundary test), mix credo, mix docs without warnings, mix compile --warnings-as-errors --force, Sobelow, mix deps.unlock --check-unused and mix gettext.extract --check-up-to-date pass.
  • Demo, on the merged head: 392 tests and 3 doctests, the 52 Playwright tests and bun run lint pass, including the query count tests for the BelongsTo index forms and for the HasMany options that depend on the form.
  • Manually in the demo (before the merge with develop):
    • Metrics.
    • New and edit forms with BelongsTo, HasMany (search, Select all) and HasManyThrough (attach an address).
    • Search on an integer column, on a custom alias and ordering by association columns.
    • BelongsTo index edit (clear and set).
    • Deleting a referenced category shows "The item is used elsewhere."

@Flo0807 Flo0807 added the breaking-change A breaking change label Oct 10, 2026
@Flo0807
Flo0807 added this pull request to stack #2339 October 10, 2026 16:14
@Flo0807 Flo0807 self-assigned this Oct 10, 2026
@Flo0807
Flo0807 marked this pull request as draft October 10, 2026 16:30
Base automatically changed from fix/has-many-select-all-options-query to develop October 10, 2026 16:32
@Flo0807
Flo0807 force-pushed the feature/no-ecto-outside-adapter branch from d8621ca to 31a8395 Compare October 10, 2026 16:32
@Flo0807
Flo0807 force-pushed the feature/no-ecto-outside-adapter branch 2 times, most recently from 89e6a36 to 09e171b Compare October 16, 2026 07:19
Flo0807 added a commit that referenced this pull request Oct 26, 2026
…eutral-filters-and-metrics

Bring in develop through #2338:

- Backpex.Metric.data/2 of v0.23 becomes data/3, which also receives the
  criteria. A metric that still defines data/2 doesn't compile, and
  Backpex.Resource.metric/5 raises for one that only declares the behaviour.
- Boolean filters ignore selected keys without an option, as on develop.
- Range filters parse number boundaries on integer fields as integers in
  condition/4, with the field type from the adapter.
@Flo0807
Flo0807 removed this pull request from stack #2339 October 28, 2026 15:41
@Flo0807
Flo0807 changed the base branch from develop to feature/option-tests October 28, 2026 15:41
@Flo0807
Flo0807 added this pull request to stack #2437 October 28, 2026 15:41
Base automatically changed from feature/option-tests to develop October 29, 2026 08:12
@Flo0807
Flo0807 removed this pull request from stack #2437 October 29, 2026 11:40
@Flo0807
Flo0807 added this pull request to stack #2456 October 29, 2026 11:40
@Flo0807
Flo0807 removed this pull request from stack #2456 October 29, 2026 13:28
@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

- Build queries only in the Ecto adapter
- Document the adapter boundary in the v0.23 upgrade guide
- Note that only the Ecto adapter calls the query callbacks of filters and metrics
- Limit the options to the ids after the options query
- Cast searched columns of unknown type to text
- Document the form mount, the options of HasMany fields and HasManyThrough in the upgrade guide
- Select the options by the primary key of the associated schema
- Move the upgrade notes to the v0.24 upgrade guide
- Pass the LiveResource to HasMany.before_changeset/6 in the options test
- Test every option without a test
- Test the options the trimmed fixtures no longer configure
- Enable the placeholder function and Number help text tests
- Adapt the recording adapter and the BelongsTo form test to the adapter callbacks
- Match the struct of a change without __struct__
- Move the upgrade notes to the v0.25 guide
- Refer to v0.24 as the previous version in the upgrade guide
@Flo0807
Flo0807 force-pushed the feature/no-ecto-outside-adapter branch from 6c5d154 to 6ca3f2e Compare November 13, 2026 09:46

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change A breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant