morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #3782] [MERGED] fix(javascript): parse JavaScript sources with JavaScript grammar #5895

New issue

Closed

opened 2026-10-31 01:10:14 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:10:14 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3782
Author: @kaoru0822-kitauji
Created: 10/15/2026
Status: ✅ Merged
Merged: 10/15/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: fix/javascript-parse-js-grammar-fallback

📝 Commits (1)

672f340 fix(javascript): parse JavaScript sources with JavaScript grammar

📊 Changes

3 files changed (+53 additions, -4 deletions)

View changed files

📝 docs/javascript-artifact-reconstruction.md (+3 -0)
➕ src/domain/javascript/javascriptSourceParser.test.ts (+42 -0)
📝 src/domain/javascript/javascriptSourceParser.ts (+8 -4)

📄 Description

Fixes #3780
Refs #3670

Root cause

parserPluginsForPath is the single owner of parser dialect selection, and it enabled Babel's typescript plugin for .js, .jsx, .mjs and .cjs sources. TypeScript reads some valid JavaScript differently:

c?H=>({…}):H=>({…}) throws, because ):H=> is read as an arrow return-type annotation. parseJavaScriptSource returns null, so the whole file is lost. VS Code's workbench.desktop.main.js (20.3 MB) and sessions.desktop.main.js (21.3 MB) both hit this at line 550. That is #3670's VSCodium parse_status: "failed", which covers about 68% of the text read.
fetch<limit>("/api") is read as a generic call, which produces an endpoint that JavaScript semantics do not have.
An earlier misparse can also create false recovered errors further down the file ('return' outside of function in Claude.app's heavy-work worker).

Change

Explicit JavaScript extensions now use JavaScript grammar (decorators-legacy + jsx). TypeScript dialects (.ts/.tsx/.mts/.cts/.d.*) and unnamed sources keep their current plugins. The producing boundary is fixed directly, with no retry-on-throw fallback, so a JavaScript file has exactly one meaning, and the reference-source import parser gets the same dialect because it shares parserPluginsForPath. The parser contract in docs/javascript-artifact-reconstruction.md is updated.

Validation

Grammar equivalence on real application code. I compared the current plugins with JavaScript grammar across 3,732 .js/.jsx/.mjs/.cjs files from Discord, Notion, Signal, Claude, Tabby, VS Code out/ and VS Code node_modules:

	TypeScript plugin (main)	JavaScript grammar (this PR)
files that throw	2 (both VS Code bundles)	0
recovered errors on valid JS	1 file	0
static facts, files both parse	baseline	identical on 3,729 / 3,730 (the one difference is the false error removed)
parse time, the 3,730 files	5,606 ms	4,644 ms (−17%)

End-to-end CLI, VS Code resources/app/out (analyze-javascript-application --json, macOS arm64, Node 26.5):

	main	this PR
reporter's flags `--max-heap-mb 6144 --analysis-timeout-ms 900000`	49.5 s, 1.50 GB RSS; `parse_failures: 2`; 3,476,040 AST nodes; 2,256 findings	86.5 s, 3.54 GB RSS; `parse_failures: 0`; 13,950,257 AST nodes; 2,937 findings; both bundles `parse_status: "partial"` with complete static facts, and the parent result budget limits reported explicitly
default limits	51.8 s; both bundles reported as parse failures (false: they are valid JS)	58.6 s; `parse_failures: 0`; both bundles report the real cause, the 1 GiB worker heap exhausted during analysis
output size	239,975,877 B (flags) / 239,975,825 B (default)	240,125,378 B (+149 KB) / 240,020,056 B (+44 KB)

The extra wall time with the reporter's flags is the 41.6 MB of bundles that main never analyzed. Output grows by under 0.07%, because the new facts sit on the file nodes and the expanded relationships stay bounded by the existing result budget.

Tests.

New javascriptSourceParser.test.ts covers three things: the VS Code conditional-arrow shape across all four JavaScript extensions (no errors; the IPC handler after it is recovered), JavaScript comparison meaning for a<b>(c) vs the .ts generic call, and TypeScript syntax still accepted for .ts and unnamed sources. The first two fail on main; I checked by disabling the change.
158 JavaScript/domain test files pass (2,965 tests).
npm run check:fast and npm run docs:check pass.

Remaining

With default limits, 20 MB bundles now get past parsing but exhaust the 1 GiB worker heap. #3779 lowers the static-phase peak heap (1,520 → 1,052 MB on a 5 MB bundle). Bundles this large still need either a larger heap or a streaming/static-only mode, which I'm treating as separate work under #3670.
.js files containing TypeScript-only syntax (not valid JavaScript) now produce recovered parse errors instead of being accepted. None of the 3,732 measured files had any.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3782 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/15/2026 **Status:** ✅ Merged **Merged:** 10/15/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `fix/javascript-parse-js-grammar-fallback` --- ### 📝 Commits (1) - [`672f340`](https://github.com/morluto/rea/commit/672f340cfefac62a7d92faf8b83445e0c2ac6de8) fix(javascript): parse JavaScript sources with JavaScript grammar ### 📊 Changes **3 files changed** (+53 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `docs/javascript-artifact-reconstruction.md` (+3 -0) ➕ `src/domain/javascript/javascriptSourceParser.test.ts` (+42 -0) 📝 `src/domain/javascript/javascriptSourceParser.ts` (+8 -4) </details> ### 📄 Description Fixes #3780 Refs #3670 ## Root cause `parserPluginsForPath` is the single owner of parser dialect selection, and it enabled Babel's `typescript` plugin for `.js`, `.jsx`, `.mjs` and `.cjs` sources. TypeScript reads some valid JavaScript differently: - `c?H=>({…}):H=>({…})` throws, because `):H=>` is read as an arrow return-type annotation. `parseJavaScriptSource` returns `null`, so the whole file is lost. VS Code's `workbench.desktop.main.js` (20.3 MB) and `sessions.desktop.main.js` (21.3 MB) both hit this at line 550. That is #3670's VSCodium `parse_status: "failed"`, which covers about 68% of the text read. - `fetch<limit>("/api")` is read as a generic call, which produces an endpoint that JavaScript semantics do not have. - An earlier misparse can also create false recovered errors further down the file (`'return' outside of function` in Claude.app's heavy-work worker). ## Change Explicit JavaScript extensions now use JavaScript grammar (`decorators-legacy` + `jsx`). TypeScript dialects (`.ts/.tsx/.mts/.cts/.d.*`) and unnamed sources keep their current plugins. The producing boundary is fixed directly, with no retry-on-throw fallback, so a JavaScript file has exactly one meaning, and the reference-source import parser gets the same dialect because it shares `parserPluginsForPath`. The parser contract in `docs/javascript-artifact-reconstruction.md` is updated. ## Validation **Grammar equivalence on real application code.** I compared the current plugins with JavaScript grammar across 3,732 `.js/.jsx/.mjs/.cjs` files from Discord, Notion, Signal, Claude, Tabby, VS Code `out/` and VS Code `node_modules`: | | TypeScript plugin (main) | JavaScript grammar (this PR) | |---|---:|---:| | files that throw | 2 (both VS Code bundles) | **0** | | recovered errors on valid JS | 1 file | 0 | | static facts, files both parse | baseline | identical on 3,729 / 3,730 (the one difference is the false error removed) | | parse time, the 3,730 files | 5,606 ms | **4,644 ms** (−17%) | **End-to-end CLI**, VS Code `resources/app/out` (`analyze-javascript-application --json`, macOS arm64, Node 26.5): | | main | this PR | |---|---|---| | reporter's flags `--max-heap-mb 6144 --analysis-timeout-ms 900000` | 49.5 s, 1.50 GB RSS; `parse_failures: 2`; 3,476,040 AST nodes; 2,256 findings | 86.5 s, 3.54 GB RSS; `parse_failures: 0`; 13,950,257 AST nodes; **2,937 findings**; both bundles `parse_status: "partial"` with complete static facts, and the parent result budget limits reported explicitly | | default limits | 51.8 s; both bundles reported as **parse failures** (false: they are valid JS) | 58.6 s; `parse_failures: 0`; both bundles report the real cause, the 1 GiB worker heap exhausted during analysis | | output size | 239,975,877 B (flags) / 239,975,825 B (default) | 240,125,378 B (+149 KB) / 240,020,056 B (+44 KB) | The extra wall time with the reporter's flags is the 41.6 MB of bundles that main never analyzed. Output grows by under 0.07%, because the new facts sit on the file nodes and the expanded relationships stay bounded by the existing result budget. **Tests.** - New `javascriptSourceParser.test.ts` covers three things: the VS Code conditional-arrow shape across all four JavaScript extensions (no errors; the IPC handler after it is recovered), JavaScript comparison meaning for `a<b>(c)` vs the `.ts` generic call, and TypeScript syntax still accepted for `.ts` and unnamed sources. The first two fail on `main`; I checked by disabling the change. - 158 JavaScript/domain test files pass (2,965 tests). - `npm run check:fast` and `npm run docs:check` pass. ## Remaining - With default limits, 20 MB bundles now get past parsing but exhaust the 1 GiB worker heap. #3779 lowers the static-phase peak heap (1,520 → 1,052 MB on a 5 MB bundle). Bundles this large still need either a larger heap or a streaming/static-only mode, which I'm treating as separate work under #3670. - `.js` files containing TypeScript-only syntax (not valid JavaScript) now produce recovered parse errors instead of being accepted. None of the 3,732 measured files had any. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:10:14 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:45:26 +00:00

[PR #5918] [MERGED] fix(model): admit safetensors header lengths that resemble pickle markers #7106

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#5895

No description provided.

Rows
Columns