Test query translation against real LINQ providers

@T-Gro

@T-Gro, I'd like to agree on test projects for F# query translation before sending the fixes.

Every query test in the repo runs over AsQueryable(), and LINQ to Objects executes any expression tree, so shapes that break real providers still pass. #11131 and #15648 are closed, but an anonymous record with fields out of alphabetical order still fails in EF Core: the Lambda.Invoke became an Expression.Block with a local variable, which no provider translates either. The same blind spot hides struct tuples (#15522), nested for and double left joins (#8459, #6552) and nested lambdas (#2758).

What I propose:

Tree-shape tests in FSharp.Core.UnitTests, with no new dependencies: an IQueryProvider that keeps queries on itself and runs them in memory, plus a validator that fails on nodes a C# expression lambda never contains (blocks, unbound parameters, calls into FSharp.Core, joins nested in a SelectMany collection selector, member reads from a tuple or record built without Members). One line per query shape, about 130 so far. This part is ready.
A provider test project, e.g. tests/FSharp.Core.LinqProviders.Tests, that runs the same scenarios on Microsoft.EntityFrameworkCore.Sqlite (in-memory database: real SQL from ToQueryString(), rows compared) and Microsoft.Azure.Cosmos (offline ToQueryDefinition(), no emulator, no network). Only this proves that a shape actually translates.
Port the ~50 expression-text baselines in tests/fsharp/core/queriesOverIQueryable to (1); today they run only in the net472 FSharpSuite leg.

Questions:

Can (2) live in this repo? Restore goes through the dnceng feeds only, so EF Core, SQLitePCLRaw (native e_sqlite3) and the Cosmos SDK would need to be mirrored into dotnet-public and added to eng/Packages.props. If that is a no-go, I would keep (2) in a separate repo that tests nightly FSharp.Core builds, and land only (1) and (3) here.
A separate project, or part of FSharp.Core.UnitTests?
Which CI legs should run it? It takes under a minute.

We can have tests that use nuggets, also via .fsx (search for them, we did have one using Dapper for example).
Definitety do not edit the repo's own engineering for it, this would be test-only deps and should be treated as such.

It should not go to fsharp.core.unittests though, ComponentTests are more appropriate here.

Porting desktop only tests to be Multiplatform is definitely good either way, so this (ideally in a separate PR) is a good idea for sure.

Or, another idea if there is an OSS repo heavily utilizing LINQ and any of those providers - add such project to the regression testing pipeline.
Well isolated, can use its own engineering setup, and still be exercised with compiler and fslib changes.

Following the second idea, I put the provider tests into a separate repository: https://github.com/fsprojects/FSharp.Linq.ProviderTests

It runs the 227 query scenarios of the query PR stack (#20894–#20901) against EF Core (SQLite with the rows compared, SQL Server and PostgreSQL by their SQL), Azure Cosmos DB (the SQL, and the rows on the emulator in its own CI), linq2db (SQLite) and SQLProvider (59 queries written in its style, run against a SQLite file). Most providers cannot translate most scenarios, so every outcome is compared with a recorded baseline. With the build of FSharp.Core that recorded the baseline, any difference fails. With any other build, such as the one the pipeline makes, only regressions fail: ok that is not ok any more, a failure that now returns wrong rows, a scenario that is no longer run. Improvements are printed.

It keeps to what the matrix needs: implicit FSharp.Core only, xUnit v3 on VSTest, ContinuousIntegrationBuild=false, NuGetAudit=false. I ran it the way regression-test-jobs.yml does, with BuildWithLocalFSharp.fsx --verify against a Release -pack build of main.

Entries for azure-pipelines-PR.yml:

- repo: fsprojects/FSharp.Linq.ProviderTests
  commit: 0ded899e10858c5788fcb6f4fd6a6bbd7c8ee5f5
  buildScript: dotnet test tests/ProviderTests/FSharp.Linq.ProviderTests.fsproj -c Release
  displayName: FSharpLinqProviderTests
  expectLocalCore: true
- repo: fsprojects/FSharp.Linq.ProviderTests
  commit: 0ded899e10858c5788fcb6f4fd6a6bbd7c8ee5f5
  buildScript: dotnet test tests/SQLProviderTests/FSharp.Linq.ProviderTests.SQLProvider.fsproj -c Release
  displayName: FSharpLinqProviderTests_SQLProvider
  expectLocalCore: true

Two entries, so that a failure of one does not hide the results of the other.

Results today:

main: no difference from the baseline.
the query PR stack: more scenarios translate (EF Core SQLite 102 → 209 ok, Cosmos 19 → 100, linq2db 148 → 215, SQLProvider 59 of 59), and the run reports one regression, in linq2db: groupValBy a tuple of values, the group as a sequence is still the group.
the draft Apply DefaultIfEmpty where a leftOuterJoin group is read #20903 (left outer join): three SQLProvider queries return wrong rows and one fails, because LEFT OUTER JOIN becomes INNER JOIN.

If this direction is fine, I will open a PR with the two entries.

Navigation Menu

Repository navigation

Test query translation against real LINQ providers #20649

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Test query translation against real LINQ providers #20649

Description

Activity

T-Gro commented on Oct 12, 2026

T-Gro commented on Oct 12, 2026

xperiandri commented on Nov 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions