Repository navigation
[YAML] Fix SQL-backed Join generating invalid syntax when joining 3+ inputs - #41068
vishalmore90 wants to merge 1 commit into
Conversation
5a76ee5 to
6b1bda8
Compare
There was a problem hiding this comment.
📋 Review Summary
This PR resolves #41052 by constructing an adjacency graph from the parsed join equalities and traversing connected tables from tables[0] rather than iterating blindly in input dictionary order. This guarantees every JOIN clause has a valid ON condition and only references tables already introduced in the FROM/JOIN sequence. It also fixes multi-table outer join propagation across visited tables and adds '*' wildcard support in _parse_fields.
🔍 General Feedback
- The graph traversal and composite
ONclause construction cleanly handle arbitrary input orderings, multi-hop chains, and cyclic/triangle join predicates. - Adding
SqlJoinQueryTestto validate the generated SQL strings directly without requiring an external runner/expansion service provides good precommit test coverage.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
|
Assigning reviewers: R: @derrickaw for label yaml. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Fixes #41052.
Description
In Beam YAML,
_SqlJoinTransformgenerated invalid SQL queries when joining three or more inputs in topologies or dictionary key orderings that did not match a star topology centered attables[0]. This caused two failure modes:ONclause (INNER JOIN tableB INNER JOIN tableC ON ...), leading to CalciteSqlParseException.ONclauses before being joined into the query scope._parse_fields, specifying'*'as explicitly documented in_SqlJoinTransformraised aValueError.This PR:
tables[0]. Each newly joined table is connected to already visited tables with all connecting predicates placed into itsONclause, guaranteeing valid SQL scoping and syntax.cols == '*'in_parse_fields.yaml_join_test.pycovering multi-table arbitrary order joins, cycle/triangle joins, and wildcard field parsing.CHANGES.md.