[alamb]

Which issue does this PR close?

This follows the same rationale as earlier PRs like

• Minor: Consolidate some integration tests into core_integration #10588
• Minor: consolidate path_partition test into core_integration #11831
• Consolidate core_integration/datasource and rename parquet_source --> parquet_integration #18226
• minor: combine set_comparison test into core_integration binary #26419

Rationale for this change

Each .rs file directly in datafusion/ffi/tests becomes its own test binary. In this case, the 10 standalone ffi_* binaries require about 2.48 GB (2,664,138,160 bytes) in total on my machine (macOS, aarch64)

cargo test -p datafusion-ffi --features integration-tests --tests --no-run

The binaries were added in

• ffi_integration: feat: Add ScalarUDF support in FFI crate #14579
• ffi_udaf: feat: Add Aggregate UDF to FFI crate #14775
• ffi_udf: Allow type coersion of zero input arrays to nullary #15487
• ffi_udtf: Add Table Functions to FFI Crate #15581
• ffi_udwf: feat: Add Window UDFs to FFI Crate #16261
• ffi_catalog: Implement CatalogProviderList in FFI #18657
• ffi_config: Add support for FFI config extensions #19469
• ffi_execution_plan: feat: support repartitioning of FFI execution plans #20449
• ffi_physical_optimizer: feat: implement PhysicalOptimizerRule in FFI crate #20451
• ffi_query_planner: feat: Implement FFI_QueryPlanner #24028

What changes are included in this PR?

Combine the 10 datafusion/ffi/tests/ffi_*.rs files into the single ffi_integration test binary:

Size of the binaries (debug build):

binary	bytes
ffi_catalog	382,295,824
ffi_config	6,295,496
ffi_execution_plan	71,841,312
ffi_integration (before)	382,373,904
ffi_physical_optimizer	71,361,096
ffi_query_planner	382,445,168
ffi_udaf	320,780,232
ffi_udf	319,111,528
ffi_udtf	407,588,120
ffi_udwf	320,045,480
total before (10 binaries)	2,664,138,160
ffi_integration after (1 binary)	408,453,600

So this goes from about 2.48 GB to about 390 MB (408,453,600 bytes) (~85% smaller) and from 10 link steps to 1.

What is the testing strategy for this PR?

CI. The same 32 tests run before and after:

cargo test -p datafusion-ffi --features integration-tests --test ffi_integration

Are there any user-facing changes?

No, developer productivity / CI efficiency

[alamb]

we can remove this extra level of indent as a follow on PR, but I wanted to keep the diff down for this one

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.88%. Comparing base (1da3cf0) to head (9d40536).

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #26423      +/-   ##
==========================================
- Coverage   82.89%   82.88%   -0.01%     
==========================================
  Files        1153     1153              
  Lines      457487   457487              
  Branches   457487   457487              
==========================================
- Hits       379213   379202      -11     
- Misses      55264    55272       +8     
- Partials    23010    23013       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nuno-faria]

LGTM, almost 1GB saved on my end! Thanks @alamb.

# before
143M    datafusion_ffi
132M    ffi_catalog
4.3M    ffi_config
28M     ffi_execution_plan
132M    ffi_integration
28M     ffi_physical_optimizer
132M    ffi_query_planner
108M    ffi_udaf
108M    ffi_udf
142M    ffi_udtf
108M    ffi_udwf
1.1G    total

# now
143M    datafusion_ffi
142M    ffi_integration
284M    total

[alamb]

Thank you for the review @nuno-faria -- I am hoping every little bit helps