Repository navigation
Conversation
Add flushCharacters() to ParserStream and Tokenizer instances. Character-token batching can leave trailing text out of the tree until more input arrives. Consumers implementing document.write(), such as jsdom, need to make that text synchronously observable without ending the input stream. Flushing delivers buffered character tokens but leaves incomplete tags and character references pending. It does not resume suspended scripts, drain queued stream writes, or bypass the tree builder's pending table-character processing. Calls from parser callbacks are rejected to prevent reentrant token delivery. Derive source boundaries from tokenizer state at flush time so ordinary parsing needs no per-character boundary tracking. Keep attribute locations independent of character locations, since flushing can occur while an attribute is incomplete. Correct source positions for lookahead emissions and surrogate pairs so offsets and columns agree. Flushing splits character-token runs, so text-node source ranges can differ from parsing without flushing: subsequently ignored input, such as CDATA delimiters or the leading newline in a <pre> element, does not extend an already flushed run. Co-Authored-By: codex <codex@openai.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 21, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Report exceptions from synchronous customized built-in element construction without abandoning parsing. Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 21, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Report exceptions from synchronous customized built-in element construction without abandoning parsing. Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 21, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Report exceptions from synchronous customized built-in element construction without abandoning parsing. Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 21, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Report exceptions from synchronous customized built-in element construction without abandoning parsing. Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com>
Collaborator
|
given its still a large change, can you explain what its for? what exactly are you trying to do in JSDOM that needs it? some context would be good |
Contributor
Author
|
It's for jsdom's proper parser-blocking scripts / document.open();
document.write("<p>hello");
console.log(document.querySelector("p").textContent);We need to flush parse5's buffered text before |
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 22, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Replacing the parser cancels its blocking and deferred scripts without aborting independent requests. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Report exceptions from synchronous customized built-in element construction without abandoning parsing. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. Report unexpected frame parsing failures through resource error handling and release parent-document load delays, including when parsing resumes after a blocking script. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs, feeding ordinary markup in large chunks while retaining script execution checkpoints. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Sep 22, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Replacing the parser cancels its blocking and deferred scripts without aborting independent requests. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Report exceptions from synchronous customized built-in element construction without abandoning parsing. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. Report unexpected frame parsing failures through resource error handling and release parent-document load delays, including when parsing resumes after a blocking script. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs, feeding ordinary markup in large chunks while retaining script execution checkpoints. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
domenic
added a commit
to jsdom/jsdom
that referenced
this pull request
Oct 17, 2026
Suspend HTML and XML parsing at parsing-blocking scripts until their source and blocking stylesheets are ready. Resume from the same input position, preserving parser order without delaying unrelated resources behind scripts. Include blocking stylesheets in ancestor documents and stop consuming input when the document is stopped, replaced, or destroyed. Replacing the parser cancels its blocking and deferred scripts without aborting independent requests. Feed document.write() into the HTML parser's input stream, retaining insertion points across nested scripts and processing written markup synchronously. Keep script-created input streams open until document.close(). Mark scripts from inert fragment parsing as already started, while preserving executable HTML fragments from range.createContextualFragment(). Invoke custom element insertion reactions before processing subsequent tokens, including upgrades in shadow trees. Report exceptions from synchronous customized built-in element construction without abandoning parsing. Preserve written scripts and pending input when lifecycle callbacks reenter document.write() or document.close(). Construct initial about:blank frame documents directly and fire their frame load events synchronously, independent of registered event listeners. Keep subsequent navigations on the document-loading path. Report unexpected frame parsing failures through resource error handling and release parent-document load delays, including when parsing resumes after a blocking script. The JSDOM constructor and fromURL()/fromFile() can now return a partially parsed document when a blocking resource is pending. Report XML syntax errors encountered after asynchronous resumption through the virtual console; synchronous constructor errors still throw. Documents without pending blockers remain synchronous. Use supported parse5-parser-stream and saxes APIs, feeding ordinary markup in large chunks while retaining script execution checkpoints. Trailing character data from document.write() can still remain buffered until another token arrives; exposing it synchronously requires inikulin/parse5#1854. Writes from parse5 tree-builder callbacks remain buffered until the callback returns because parse5 cannot tokenize that input reentrantly. Full VM microtask checkpoints remain unimplemented, affecting custom element construction, slotchange delivery, and mutation observer ordering around parser-inserted scripts. Refs #2413. Co-authored-by: codex <codex@openai.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Gentle ping! jsdom is prepping for a new major release, and it would be great to have this bundled in. Without it, jsdom's |
Collaborator
|
sorry this has taken me so long to get to. i'll read through it today 👍 |
43081j
reviewed
Oct 19, 2026
Collaborator
|
Looked into merging this yesterday. My own agent review here flagged that this doesn't deal well with \r\n being split. Otherwise would be nice if tests could use it.each instead of the manual loops |
Exclude the ignored LF from the next character token's source range when its preceding CR was flushed. Text following the discarded leading newline in pre, textarea, and listing elements then starts at the actual character offset. Co-Authored-By: codex <codex@openai.com>
Compute flushed character-token locations with indexOf() scans instead of allocating an array of line substrings. Keep the scans bounded to emitted text so large unfinished tags or attributes do not increase flush cost. Co-Authored-By: codex <codex@openai.com>
Contributor
Author
|
Thank you both for the helpful reviews! I think I've addressed everything. CI failures look to be due to some out of sync CodeQL dependencies. |
43081j
approved these changes
Oct 20, 2026
Collaborator
|
have reviewed and all seems good 👍 there's actually 2 bug fixes in here too which flushing presumably uncovered (to you or your agent). locations were off in some places. codeql stuff is sorted in main which i've now merged into your branch |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I asked GPT-6 Astra Extra High for the minimal parse5 change that would allow us to implement
document.write()correctly in jsdom. It seems like it's mostly additive. It spent most of its time on tests, which is a good sign, and in particular was particularly concerned with getting the character offsets right.I asked it a few times
and by the third round it seemed like it was happy with the results.
Thank you for the great library :)