morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6127] `analyze-javascript-application` exhausts a 6 GiB worker heap on a 233 KB file #3759

New issue

Open

opened 2026-10-31 00:43:44 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-31 00:43:44 +00:00

Owner

Originally created by @carlogiovannineri-rgb on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6127

Version: rea-agents 6.7.0 (npm), Linux x64

What happens
On a set of 29 exported web scripts, one 233 KB file (a Next.js/Turbopack bootstrap chunk) exhausts the
worker heap even with --max-heap-mb 6144 (run inside a 7 GiB memory scope). The other 28 files finish.
In the same run the semantic layer is cut off at 100,000 nodes (truncated: true, 51,540 unknowns).
A second bundle shows the same pattern: a 5.6 MB vendor-only chunk exhausts the heap, 3 of 4 files finish.

Related: compare-application-versions has no --max-heap-mb; with two ~230 MB evidence files it fails with
resource_constraint at the default 4 GiB CLI heap and only passes after raising NODE_OPTIONS=--max-old-space-size.

Expected
The run itself completes with a partial result, which is fine. The problem is the size ratio: a 233 KB file
should not need more than 6 GiB. Memory use roughly proportional to input size would let the whole set finish.
A --max-heap-mb option on compare-application-versions would also help.

Repro
Hard to share the file itself (third-party bundle). Any large minified Turbopack runtime chunk should show it;
happy to run a debug build or share heap statistics.

Related: #2498, #4601 (other memory/time outliers on single JavaScript files).

Originally created by @carlogiovannineri-rgb on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6127 **Version:** rea-agents 6.7.0 (npm), Linux x64 **What happens** On a set of 29 exported web scripts, one 233 KB file (a Next.js/Turbopack bootstrap chunk) exhausts the worker heap even with `--max-heap-mb 6144` (run inside a 7 GiB memory scope). The other 28 files finish. In the same run the semantic layer is cut off at 100,000 nodes (`truncated: true`, 51,540 unknowns). A second bundle shows the same pattern: a 5.6 MB vendor-only chunk exhausts the heap, 3 of 4 files finish. Related: `compare-application-versions` has no `--max-heap-mb`; with two ~230 MB evidence files it fails with `resource_constraint` at the default 4 GiB CLI heap and only passes after raising `NODE_OPTIONS=--max-old-space-size`. **Expected** The run itself completes with a partial result, which is fine. The problem is the size ratio: a 233 KB file should not need more than 6 GiB. Memory use roughly proportional to input size would let the whole set finish. A `--max-heap-mb` option on `compare-application-versions` would also help. **Repro** Hard to share the file itself (third-party bundle). Any large minified Turbopack runtime chunk should show it; happy to run a debug build or share heap statistics. Related: #2498, #4601 (other memory/time outliers on single JavaScript files).

bsakaguchi commented

2026-10-31 00:43:45 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 22, 2026):

确认，233 KB 输入吃掉 6 GiB 说明内存增长不是随输入规模线性走的，而是某条遍历路径（大概率就是成员变更追踪）组合爆炸——#6146 加 bound 的方向是对的，建议验收时带上一条"堆使用随输入近似线性"的断言，防止换个输入形态又爆。另外截断信息目前只有 truncated: true，用户根本不知道语义层在哪里、丢了多少节点：建议把截断点和被裁掉的区域（比如哪些 module/成员相关的 unknown 是被截断贡献的）写进报告，至少让"部分结果"可解释。compare-application-versions 缺 --max-heap-mb 也确实是个坑，跟 #5653 一起补齐比较合理。

@XinwangTao commented on GitHub (Oct 22, 2026): 确认，233 KB 输入吃掉 6 GiB 说明内存增长不是随输入规模线性走的，而是某条遍历路径（大概率就是成员变更追踪）组合爆炸——#6146 加 bound 的方向是对的，建议验收时带上一条"堆使用随输入近似线性"的断言，防止换个输入形态又爆。另外截断信息目前只有 truncated: true，用户根本不知道语义层在哪里、丢了多少节点：建议把截断点和被裁掉的区域（比如哪些 module/成员相关的 unknown 是被截断贡献的）写进报告，至少让"部分结果"可解释。compare-application-versions 缺 --max-heap-mb 也确实是个坑，跟 #5653 一起补齐比较合理。

bsakaguchi commented

2026-10-31 00:43:45 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 26, 2026):

Follow-up from the bug-fix batch, tested at c6893d1ac6a9ccb8a11d64b4e9ba1b57e79cbfd8 on Linux x86-64, Node 24.22.1, two CPUs. The reported private-input 6 GiB OOM is still NOT_ENOUGH_INFO: the original 233 KB artifact is unavailable. These observations do not reproduce that OOM.

I retained 29 public, same-origin immutable script assets from a nextjs.org HTML snapshot (1,538,705 bytes total), recording URLs, HTTP status, byte counts and SHA-256. No target JavaScript was executed. Actual compiled CLI command:

node scripts/rea.mjs analyze-javascript-application "$fixture_dir" \
  --source-path 3l4u49ac-_ccg.js --max-heap-mb 512 \
  --analysis-timeout-ms 20000 --detail summary \
  --evidence_output "$output_bundle" --json

Selected public chunk: 229,234 bytes, SHA-256 317586c23d5337ab6e61a873ff074cb257ef5af0f7b74daaa4ee55959d209daf. Observed provider_timeout at the 20-second source deadline, during static analysis, before semantic analysis. Whole harness: 22.550 s elapsed, 26.765 s CPU, sampled process-family RSS peak 551.74 MiB; no OOM or external resource stop. Partial failure evidence was retained.
Control Turbopack runtime: 12,863 bytes, SHA-256 96860acf83d42ba7a295c65d2e1cdeaf582c98156ce0d520a1ff29739580ba38. Same CLI/options completed in 7.504 s, 9.142 s CPU, 569.04 MiB family peak. One selected source parsed, 28 explicitly unselected; semantic coverage remains partial with 1,481 unknowns. Summary reply 11,516 bytes; complete retained Evidence bundle 6,709,960 bytes. Model-token cost was not measured.

A bounded CPU profile points to ambient reference-store receiver resolution in indexJavaScriptAmbientWritesSteps, rather than establishing the member-mutation collector as the cause. Direct static analysis also failed to complete within a separate 60-second guard. A per-query binding-origin deduplication experiment passed 56 related source tests and reduced a synthetic repeated-alias case, but still failed that public 60-second case and showed no completed small-real-input speedup. It was withdrawn; no fix or speedup is claimed.

RSS is a 100 ms /proc descendant sum: shared pages may be counted twice and short peaks may be missed. Caches were uncontrolled. This is a reproducible public timeout investigation, not proof that the private OOM is fixed.

@gensokyo-dev commented on GitHub (Oct 26, 2026): Follow-up from the bug-fix batch, tested at `c6893d1ac6a9ccb8a11d64b4e9ba1b57e79cbfd8` on Linux x86-64, Node 24.22.1, two CPUs. **The reported private-input 6 GiB OOM is still NOT_ENOUGH_INFO:** the original 233 KB artifact is unavailable. These observations do not reproduce that OOM. I retained 29 public, same-origin immutable script assets from a nextjs.org HTML snapshot (1,538,705 bytes total), recording URLs, HTTP status, byte counts and SHA-256. No target JavaScript was executed. Actual compiled CLI command: ```sh node scripts/rea.mjs analyze-javascript-application "$fixture_dir" \ --source-path 3l4u49ac-_ccg.js --max-heap-mb 512 \ --analysis-timeout-ms 20000 --detail summary \ --evidence_output "$output_bundle" --json ``` - Selected [public chunk](https://nextjs.org/_next/static/immutable/chunks/3l4u49ac-_ccg.js): 229,234 bytes, SHA-256 `317586c23d5337ab6e61a873ff074cb257ef5af0f7b74daaa4ee55959d209daf`. Observed `provider_timeout` at the 20-second source deadline, during **static** analysis, before semantic analysis. Whole harness: 22.550 s elapsed, 26.765 s CPU, sampled process-family RSS peak 551.74 MiB; no OOM or external resource stop. Partial failure evidence was retained. - Control [Turbopack runtime](https://nextjs.org/_next/static/immutable/chunks/turbopack-22zuhxnmsa7m2.js): 12,863 bytes, SHA-256 `96860acf83d42ba7a295c65d2e1cdeaf582c98156ce0d520a1ff29739580ba38`. Same CLI/options completed in 7.504 s, 9.142 s CPU, 569.04 MiB family peak. One selected source parsed, 28 explicitly unselected; semantic coverage remains partial with 1,481 unknowns. Summary reply 11,516 bytes; complete retained Evidence bundle 6,709,960 bytes. Model-token cost was not measured. A bounded CPU profile points to ambient reference-store receiver resolution in `indexJavaScriptAmbientWritesSteps`, rather than establishing the member-mutation collector as the cause. Direct static analysis also failed to complete within a separate 60-second guard. A per-query binding-origin deduplication experiment passed 56 related source tests and reduced a synthetic repeated-alias case, but still failed that public 60-second case and showed no completed small-real-input speedup. It was withdrawn; no fix or speedup is claimed. RSS is a 100 ms `/proc` descendant sum: shared pages may be counted twice and short peaks may be missed. Caches were uncontrolled. This is a reproducible public timeout investigation, not proof that the private OOM is fixed.

bsakaguchi commented

2026-10-31 00:43:45 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 27, 2026):

认同 gensokyo-dev 的评估：私有工件不可用时 OOM 无法直接复现。建议报告者提供一个脱敏后的最小可复现样例（只需保留触发组合爆炸的 bootstrap chunk 结构），否则修复只能靠推断验证。另外建议把这类 Turbopack bootstrap chunk 纳入回归测试集——用公开同源资产固定一个内存基线断言（如 --max-heap-mb 2048 内完成），防止后续 member-mutation traversal bound 的调整重新引入爆炸路径。

@XinwangTao commented on GitHub (Oct 27, 2026): 认同 gensokyo-dev 的评估：私有工件不可用时 OOM 无法直接复现。建议报告者提供一个脱敏后的最小可复现样例（只需保留触发组合爆炸的 bootstrap chunk 结构），否则修复只能靠推断验证。另外建议把这类 Turbopack bootstrap chunk 纳入回归测试集——用公开同源资产固定一个内存基线断言（如 --max-heap-mb 2048 内完成），防止后续 member-mutation traversal bound 的调整重新引入爆炸路径。

bsakaguchi commented

2026-10-31 00:43:45 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 27, 2026):

Agree with the fix direction (bounding member-mutation traversal + a max-heap-mb cap). One additional thought from gensokyo-dev's experiment: the public 233 KB bootstrap chunk still hits the 20 s/60 s guards even under a heap cap, so with only a memory bound users will keep seeing a generic timeout with no hint whether the input is too complex or the budget too small. Suggest distinguishing OOM vs. "analysis did not converge within budget" in failure output, and printing partial stats (resolved/unknown references, visited member paths) on timeout. That would also make the proposed --max-heap-mb 2048 regression baseline much easier to act on.

@XinwangTao commented on GitHub (Oct 27, 2026): Agree with the fix direction (bounding member-mutation traversal + a max-heap-mb cap). One additional thought from gensokyo-dev's experiment: the public 233 KB bootstrap chunk still hits the 20 s/60 s guards even under a heap cap, so with only a memory bound users will keep seeing a generic timeout with no hint whether the input is too complex or the budget too small. Suggest distinguishing OOM vs. "analysis did not converge within budget" in failure output, and printing partial stats (resolved/unknown references, visited member paths) on timeout. That would also make the proposed --max-heap-mb 2048 regression baseline much easier to act on.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3759

No description provided.

Rows
Columns

[GH-ISSUE #6127] analyze-javascript-application exhausts a 6 GiB worker heap on a 233 KB file #3759

[GH-ISSUE #6127] `analyze-javascript-application` exhausts a 6 GiB worker heap on a 233 KB file #3759