Repository navigation
Stop reading the right stream when the hash table is empty - #23
Merged
Merged
Conversation
An inner hash join over an empty hash table produces nothing, so the right stream never has to be read. It was read in full regardless, which is cheap for an in-memory iterator but not for one backed by a remote source. The iterator is ended from `read` rather than as soon as the left stream ends, because `_end` notifies the listeners attached at that moment and then drops them, and a consumer that is still being set up would never see the event. In Comunica over a Triple Pattern Fragments endpoint this made three WatDiv queries download every page of two fragments to produce no results: S3 went from 7 to 67 HTTP requests, S5 from 7 to 28. Both are back to 7. The CI workflow is updated alongside, since `actions/cache@v2` is now refused by GitHub and the run fails before installing anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019fVTUvQ2uCR1p4KQxAYrgs
rubensworks
force-pushed
the
claude/hash-join-empty-build-shortcircuit
branch
from
September 7, 2026 13:23
0dc4046 to
38f0bb5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An inner hash join over an empty hash table produces nothing, so the right stream never has to be read.
HashJoinread it in full regardless: once the left stream ends,allowJoiningunconditionally starts pulling from the right and hashing every item against an empty map.That is cheap for an in-memory iterator, but not for one backed by a remote source.
Impact
Found while investigating a performance regression in Comunica's WatDiv Triple Pattern Fragments benchmark. Three of the queries return no results, and the plan puts an empty intermediate on the build side of a hash join. Comunica then downloaded every page of the remaining fragments to produce nothing.
Measured against the WatDiv-10 dataset used in Comunica's CI, with a local LDF server, 5 query instances per template, HTTP requests summed:
All 20 WatDiv templates return identical results, and the other 18 issue an identical number of requests.
Why the iterator is ended from
readThe obvious implementation, ending as soon as the left stream ends, is wrong.
AsyncIterator._endemitsendsynchronously and then callsremoveAllListeners('end'), so a consumer that attaches its listeners after that point never sees the event. Ending as soon as the left stream ends is exactly such a point: nothing has necessarily read the join yet.This is not hypothetical. It hangs Comunica's
ActorRdfJoinHashtest for empty input, which awaitsmetadata()before touching the stream. So the empty case sets a flag, andreadends the iterator, which mirrors how the non-empty case already ends from the right stream'sendhandler.The added test
ends for a listener that is attached after the left stream endedcovers this and fails against the naive version.Correctness
The short-circuit only fires when the hash table is empty after the left stream has ended, which for an inner join means the output is empty.
_endalready destroys both streams, so the right stream is cleaned up as before.merges an empty stream with a stream of length 3,can handle streams that are already doneandpropagates errors in the right stream.endstill reaches a late listener, and a non-empty left stream whose join function filters everything out still reads the right stream and terminates.CI
actions/cache@v2is now refused outright by GitHub, so every run fails before installing anything, onmasteras well as here. The workflow is updated toactions/checkout@v4andactions/setup-node@v4with its built-in yarn cache, and the Node matrix moves from 12/14/16 to 20/22/24. Install uses--frozen-lockfileagainst the existingyarn.lock; nopackage-lock.jsonis introduced.🤖 Generated with Claude Code
https://claude.ai/code/session_019fVTUvQ2uCR1p4KQxAYrgs