[PR #1545] [MERGED] fix(javascript): yield during the largest AST traversals #2961

New issue

Closed

opened 2026-10-17 21:12:18 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:12:18 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1545
Author: @kaoru0822-kitauji
Created: 10/10/2026
Status: ✅ Merged
Merged: 10/10/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/javascript-traversal-steps-2

📝 Commits (1)

fe5650b fix(javascript): yield during the largest AST traversals

📊 Changes

8 files changed (+166 additions, -97 deletions)

View changed files

📝 src/application/javascript/JavaScriptArtifactAnalysis.ts (+2 -2)
📝 src/domain/javascript/javascriptSemanticAnalysis.ts (+2 -3)
📝 src/domain/javascript/javascriptSemanticAnalysisSteps.test.ts (+27 -0)
📝 src/domain/javascript/javascriptSemanticDerivedAnalysis.ts (+15 -14)
📝 src/domain/javascript/javascriptSemanticFingerprints.ts (+53 -52)
📝 src/domain/javascript/javascriptSemanticMemberMutations.ts (+11 -9)
📝 src/domain/javascript/javascriptSemanticTraversal.ts (+35 -9)
📝 src/domain/javascript/javascriptStaticAnalysis.ts (+21 -8)

📄 Description

Refs #1462. This is the next step after #1525 and #1542, splitting the largest remaining single-traversal blocks inside the analyzers.

Change

traverseJavaScriptAstSteps (javascriptSemanticTraversal.ts) is the existing explicit-stack traversal with a yield every 1,024 visited nodes. Visit order, readAncestors and exit are unchanged. traverseJavaScriptAst drains it, so the traversal loop exists once and all existing callers behave as before. completeSemanticSteps is the shared drain helper.
Member mutations (collectSemanticMemberMutationsSteps): both traversals are stepwise, and the pending-reference and iterable fixpoint loops yield every 64 references. Visit and fixpoint order are unchanged.
Static analysis (analyzeParsedJavaScriptStaticSourceSteps): the source traversal is stepwise. analyzeParsedJavaScriptStaticSource remains as the synchronous form.
Fingerprints (collectJavaScriptSemanticFingerprintsSteps): a yield between callables, plus a stepwise subtree traversal per callable. A bundle wrapper's subtree can be the whole file. Callable order and digests are unchanged.
Wiring: semantic analysis, derived collection and per-file analysis yield* into these. The synchronous member-mutation and fingerprint wrappers had no remaining callers and are removed.

Results are identical

reconstructJavaScriptArtifact on current main (ca7ef69c, after #1542) and on this branch gives the same application graph ID, semantic graph ID and SHA-256 of the complete serialized result for:

the #1462 1,200-function fixture;
Tabby app.asar;
Obsidian obsidian.asar (23 MB).

Measured effect

Obsidian obsidian.asar, full in-process analysis with a 1 ms event-loop gap monitor, two runs each, M-series Mac:

	longest block	2nd	blocks > 250 ms	total
main	936 / 911 ms	866 / 862 ms	28 / 26	40.0 / 39.6 s
this branch	758 / 736 ms	599 / 591 ms	19 / 18	40.7 / 40.3 s

Remaining

The remaining blocks of about 0.5–0.75 s include:

smaller traversals still run synchronously: definitions, open-receiver classification, references, and the data-effect helpers;
Babel's parse, which can't be split;
some apparently GC-bound pauses during sealing, on a heap of about 1.6 GB.

These can be converted the same way. Each step is smaller, so I kept this PR to the three largest.

Verification

javascriptSemanticAnalysisSteps.test.ts has a new test: the step traversal yields on a 400-statement file and records exactly the synchronous enter/exit sequence. The existing semantic-steps equality test now also exercises the stepwise member-mutation and fingerprint passes.
npx vitest run src/domain src/application passes (171 files, 2,139 tests).
npm run test:focused on the JavaScript composition, acceptance and analysis-view boundary suites passes (20 files, 112 tests).
npm run check:fast and npm run docs:check pass. Lint warnings in the touched files match main (pre-existing complexity, max-lines and max-params).

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1545 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/10/2026 **Status:** ✅ Merged **Merged:** 10/10/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/javascript-traversal-steps-2` --- ### 📝 Commits (1) - [`fe5650b`](https://github.com/morluto/rea/commit/fe5650bd57ccbaf75f8311047e811207695ead42) fix(javascript): yield during the largest AST traversals ### 📊 Changes **8 files changed** (+166 additions, -97 deletions) <details> <summary>View changed files</summary> 📝 `src/application/javascript/JavaScriptArtifactAnalysis.ts` (+2 -2) 📝 `src/domain/javascript/javascriptSemanticAnalysis.ts` (+2 -3) 📝 `src/domain/javascript/javascriptSemanticAnalysisSteps.test.ts` (+27 -0) 📝 `src/domain/javascript/javascriptSemanticDerivedAnalysis.ts` (+15 -14) 📝 `src/domain/javascript/javascriptSemanticFingerprints.ts` (+53 -52) 📝 `src/domain/javascript/javascriptSemanticMemberMutations.ts` (+11 -9) 📝 `src/domain/javascript/javascriptSemanticTraversal.ts` (+35 -9) 📝 `src/domain/javascript/javascriptStaticAnalysis.ts` (+21 -8) </details> ### 📄 Description Refs #1462. This is the next step after #1525 and #1542, splitting the largest remaining single-traversal blocks inside the analyzers. ## Change - **`traverseJavaScriptAstSteps`** (`javascriptSemanticTraversal.ts`) is the existing explicit-stack traversal with a yield every 1,024 visited nodes. Visit order, `readAncestors` and `exit` are unchanged. `traverseJavaScriptAst` drains it, so the traversal loop exists once and all existing callers behave as before. `completeSemanticSteps` is the shared drain helper. - **Member mutations** (`collectSemanticMemberMutationsSteps`): both traversals are stepwise, and the pending-reference and iterable fixpoint loops yield every 64 references. Visit and fixpoint order are unchanged. - **Static analysis** (`analyzeParsedJavaScriptStaticSourceSteps`): the source traversal is stepwise. `analyzeParsedJavaScriptStaticSource` remains as the synchronous form. - **Fingerprints** (`collectJavaScriptSemanticFingerprintsSteps`): a yield between callables, plus a stepwise subtree traversal per callable. A bundle wrapper's subtree can be the whole file. Callable order and digests are unchanged. - **Wiring:** semantic analysis, derived collection and per-file analysis `yield*` into these. The synchronous member-mutation and fingerprint wrappers had no remaining callers and are removed. ## Results are identical `reconstructJavaScriptArtifact` on current `main` (`ca7ef69c`, after #1542) and on this branch gives the same application graph ID, semantic graph ID and SHA-256 of the complete serialized result for: - the #1462 1,200-function fixture; - Tabby `app.asar`; - Obsidian `obsidian.asar` (23 MB). ## Measured effect Obsidian `obsidian.asar`, full in-process analysis with a 1 ms event-loop gap monitor, two runs each, M-series Mac: | | longest block | 2nd | blocks > 250 ms | total | |---|---|---|---|---| | main | 936 / 911 ms | 866 / 862 ms | 28 / 26 | 40.0 / 39.6 s | | this branch | 758 / 736 ms | 599 / 591 ms | 19 / 18 | 40.7 / 40.3 s | ## Remaining The remaining blocks of about 0.5–0.75 s include: - smaller traversals still run synchronously: definitions, open-receiver classification, references, and the data-effect helpers; - Babel's parse, which can't be split; - some apparently GC-bound pauses during sealing, on a heap of about 1.6 GB. These can be converted the same way. Each step is smaller, so I kept this PR to the three largest. ## Verification - `javascriptSemanticAnalysisSteps.test.ts` has a new test: the step traversal yields on a 400-statement file and records exactly the synchronous enter/exit sequence. The existing semantic-steps equality test now also exercises the stepwise member-mutation and fingerprint passes. - `npx vitest run src/domain src/application` passes (171 files, 2,139 tests). - `npm run test:focused` on the JavaScript composition, acceptance and analysis-view boundary suites passes (20 files, 112 tests). - `npm run check:fast` and `npm run docs:check` pass. Lint warnings in the touched files match `main` (pre-existing complexity, `max-lines` and `max-params`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:12:18 +00:00