Skip to content

Translate nested for and joins into one flat LINQ chain - #20902

Draft
xperiandri wants to merge 6 commits into
dotnet:mainfrom
xperiandri:fix/query-flat-chains
Draft

xperiandri wants to merge 6 commits into
dotnet:mainfrom
xperiandri:fix/query-flat-chains

Conversation

@xperiandri

Copy link
Copy Markdown
Contributor

Draft, do not merge. SQLProvider 1.5.27 depends on the nested layout this PR removes: a third for over an independent table fails there with "no such column", and two of its own tests fail. EF Core for SQL Server also stops translating two shapes that rely on a lateral join: an in-memory collection as the middle for, and a group read again after a for over it and a join. Kept for discussion of a narrower design for #8459.

Stacked on #20894

Over an IQueryable, a third for, a yield! or a join after two fors now continues one chain of SelectMany, Join and GroupJoin, as C# builds it, so the two left joins spelled groupJoin … into g and for x in g.DefaultIfEmpty() in #8459 become two SQL LEFT JOINs.

Details
  • The rest of the query used to be nested in the collection selector of the first SelectMany. EF Core translates that only as a lateral join (APPLY), which SQLite lacks, and not at all when a join is nested there.
  • A join whose inner source or inner key uses the variable of an earlier for stays nested, since it differs per outer element.
  • What follows a leftOuterJoin is translated as before.

🤖 Generated with Claude Code

xperiandri and others added 5 commits November 1, 2026 22:03
Every query test so far runs over AsQueryable() and compares rows, so a
tree that only LINQ to Objects can run passes. These scenarios also check
the tree: the query must stay on its provider, and must not contain what a
C# expression lambda never does - a block, a parameter nothing binds, a
call into FSharp.Core, a join nested in a collection selector, a member
read from a tuple built without naming its members.

A scenario names what is wrong with its tree today. The name has to be
removed by the change that makes it untrue, because a scenario fails when
its tree is cleaner than it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Selecting a struct tuple failed with "unreachable": the tuple was replaced
by an anonymous object, but only reference tuples had an anonymous object
type to be replaced by, and the way back always built a reference tuple.
Both kinds now share the anonymous object of their arity, and the way back
builds the kind the query asked for.

Taking a struct tuple apart in a lambda failed with a tuple index out of
range, because a struct tuple has fields where a reference tuple has
properties.

Fixes dotnet#15522

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
join, groupJoin and leftOuterJoin repeated the same translation with
two differences: the LINQ operator they call and the conversion the
inner result goes through. They are now recognized by one active
pattern and translated by one function that takes the outer source
already translated, and the translation of a for body takes its
source, conversion and variable as arguments. This is a refactoring
without a change in behavior; the next commit reuses both pieces to
translate a for or a join that follows a for into a flat chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A query over an IQueryable that went on after two fors, with a third
for, a yield!, or a join, groupJoin or leftOuterJoin, nested the rest
of the query inside the collection selector of the first SelectMany.
EF Core translates that only as a lateral join (APPLY), which SQLite
and several other databases lack, and cannot translate a join nested
there at all, so two left joins in a row failed or ran on the client.

The translation now pairs the elements of the first two sources, as
C# does with its transparent identifiers, and continues the query from
the pairs: a further for becomes the next SelectMany and a join the
next Join or GroupJoin of the main chain. A tuple's elements go into
the pair one by one, as they do after two fors. The rewrite applies
only when the query is over an IQueryable and a join's inner source
and key do not use the variable of the outer for; everything else is
translated as before.

What follows a leftOuterJoin is also translated as before. Its
elements hold the group already put through DefaultIfEmpty, a sequence
EF Core cannot carry along the main chain, while it does translate the
nested form with a lateral join on SQL Server and PostgreSQL.

Fixes dotnet#8459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Release notes checked


✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/FSharp.Core` docs/release-notes/.FSharp.Core/11.0.200.md

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri

Copy link
Copy Markdown
Contributor Author

Covered cases

EF Core SQLite translates 99 of 138 scenarios, 92 without this change. Now translated: for over two levels of navigation collections, three and four fors, if … then between three fors, join and groupJoin after two fors, and two left joins spelled groupJoin and DefaultIfEmpty.

Not changed: a join whose inner source or inner key uses a variable of an earlier for stays nested, and EF Core needs APPLY for it, as for the equivalent C#. What follows a leftOuterJoin stays as it was.

Why this is a draft

  • SQLProvider 1.5.27 on SQLite: a third for over an independent table, or over a navigation for followed by an independent table, fails with "no such column: pair.Id", and its tests simple select cross-join test 3 tables and simple select join and cross-join test fail. Its cross-join branch takes the name of the projection parameter as a table alias; the nested shape never reached it.
  • EF Core for SQL Server: for c in db.Customers do for n in ns do for o in c.Orders … with ns an in-memory collection, and a group read again after a for over it and a join, translate on main and no longer do.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

1 participant