morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7808] [Bug] analyze-javascript-application exhausts a 2–16 GiB worker heap on one 75 KB webpack module (minified XPath library); rest of the bundle analyzes after module-level splitting #4646

New issue

Closed

opened 2026-10-31 00:51:48 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:51:48 +00:00

Owner

Originally created by @ChanwheKim on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7808

Before submitting

I searched the open and closed issues for a related report (related: #6127, #4601, #6311, #2498 — all large-file heap/stall reports; this one narrows the trigger to a single 75 KB module).
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction (static JavaScript analysis)

Observed behavior

rea analyze-javascript-application <extracted Chrome extension dir> --json skips service-worker.js (774 KB) and main-world.js (240 KB) with:

<file>: Resource constraint during analyze_javascript_application: Owned JavaScript worker exhausted its N MiB V8 old-space heap (max_heap_mb) while analyzing <file> during static

for N = 3072, 12288 and 16384 (--max-heap-mb maximum). Time to failure 4–6 min per file. The other bundles of the same extension (e.g. content-script.js, 227 KB, 109 webpack modules) analyze fine in a few minutes.

After splitting both bundles at webpack module boundaries (see Reproduction) and re-running with --max-heap-mb 8192:

service-worker: 10/11 chunks analyzed, 176/177 modules recovered. A single 298 KB module (puppeteer-core, exports ExtensionTransport, connect) passed with examination: limited.
main-world: 4/5 chunks analyzed, 142/143 modules recovered.
The only failing chunk in both bundles is the same webpack module 8555: a ~75 KB minified IIFE XPath 1.0 implementation (tokenizer/parser tables; strings XPath, XPATH_INVALID_PARSE, …). Put alone in a directory it still fails: with --max-heap-mb 2048 the worker exhausts the heap after ~27 s, parsed_javascript_files: 0.

So the blow-up is a code pattern in that one module, not bundle size. Deterministic (3/3 runs on the full bundle at different heap sizes, 2/2 on the split chunk, 1/1 standalone).

Reproduction

Target is public: Chrome Web Store extension "Datadog test recorder" (id kkbncfpddhdmkfmalecgnphegacgejoa), version 4.5.1, unpacked from Chrome's Extensions/<id>/4.5.1_0/ directory.

npx -y rea-agents@latest analyze-javascript-application /abs/ext --json → service-worker.js and main-world.js skipped with the heap limitation above (also with --max-heap-mb 16384 --analysis-timeout-ms 1200000).
Split at module boundaries. Both bundles are webpack 5 (()=>{var e={3314(e,t){…},…};…r(entry)})(). I used a small acorn script: parse, take the big ObjectExpression module map, group Property ranges into ≤60 KB chunks, and write each chunk as head + "{" + modules + "}" + tail where head/tail are the original text before/after the object (so each chunk keeps the real webpack bootstrap and runtime). Script: https://github.com/ChanwheKim (can paste it in a comment on request; ~40 lines).
rea analyze-javascript-application /abs/split/main-world --max-heap-mb 8192 --json → every chunk succeeds except the one holding module 8555.
Minimal: a directory containing only that chunk ((()=>{var e={8555(e,t){…}};<original runtime tail>})(), 81,694 bytes) → --max-heap-mb 2048 --analysis-timeout-ms 900000 fails after ~27 s with the same limitation.

I can attach the 81 KB minimal file (it is third-party minified code shipped in the public extension) — say the word and I will add it as a gist link.

Expected behavior

Either the module analyzes (it parses fine with acorn, and REA parsed 318 other modules of the same bundles including a 298 KB one), or REA bails out of the offending phase for that file quickly with a typed partial result/limitation instead of consuming the whole worker heap for minutes. The docs' max_heap_mb remediation does not help here because the growth is unbounded relative to input size (75 KB → >16 GiB).

Impact and scope

Any application that bundles this XPath library (it is common in recorder/scraper tooling) loses the whole containing file from the application graph, and the user pays several minutes of CPU per attempt. With the default 1024 MiB heap the failure is also silent until the end of the run. Evidence integrity is otherwise fine: the skipped file is reported as parse_status: skipped with the limitation string.

REA version

rea-agents 6.8.0 (via npx -y rea-agents@latest)

Runtime and platform

Node v24.21.0; macOS 15.6 arm64 (Darwin 25.6.0); 36 GB RAM; no native provider involved.

Target and artifact context

Extracted Chrome MV3 extension directory; webpack 5 bundles (module-map style var e={<id>(e,t,n){…}}); --artifact-format directory.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction (the limitation says during static)

Relevant output or evidence

Full-bundle run (--max-heap-mb 16384, main-world.js only):

{"relevant_files": 8, "parsed_javascript_files": 0, "skipped_javascript_files": 1, "visited_ast_nodes": 0, "findings": 0, "modules": 0, "parse_failures": 0}
"main-world.js: Resource constraint during analyze_javascript_application: Owned JavaScript worker exhausted its 16384 MiB V8 old-space heap (max_heap_mb) while analyzing main-world.js during static"

Split run (--max-heap-mb 8192, 5 chunks):

{"relevant_files": 6, "parsed_javascript_files": 4, "skipped_javascript_files": 1, "visited_ast_nodes": 50144, "findings": 245, "modules": 142, "parse_failures": 0}
"chunk_02.js: Resource constraint … exhausted its 8192 MiB V8 old-space heap (max_heap_mb) while analyzing chunk_02.js during static"

Standalone module 8555 (--max-heap-mb 2048), 27 s:

{"relevant_files": 1, "text_bytes_read": 81694, "parsed_javascript_files": 0, "skipped_javascript_files": 1}
"xpath-module-8555.js: Resource constraint … exhausted its 2048 MiB V8 old-space heap (max_heap_mb) while analyzing xpath-module-8555.js during static"

First 160 bytes of the module (for identification):

{8555(e,t){!function(e){"use strict";var t=function(e,t){for(var r=0;r<t.length;r+=1)e(t[r],r,t)},r=function(e,r,n){var i=r;return t((function(t,r){i=e(i,t,r)})

Typed outcome or recovery expected

A per-file resource_constraint limitation returned within seconds (not minutes) and the rest of the file set analyzed — which the split run already achieves once the module is isolated — plus, ideally, the other modules of the same file retained instead of the whole file being dropped.

Analysis or suspected cause

Hypothesis only (no REA source read): the module is one large IIFE whose body builds the XPath grammar as nested closures and tables (hundreds of small functions assigned into shared objects). Something in the static phase (alias/selection expansion or member-mutation traversal — cf. #2498) appears to grow super-linearly on that shape, since the 298 KB puppeteer-core module and 318 other modules from the same bundles are fine.

Additional context

Related reports: #6127 (6 GiB heap on 233 KB file), #4601 (277 KB webpack factory stall), #6311 (hang on 664 KB UMD), #2498 (member-mutation traversal 1.2 GB on openjpeg). This report adds a deterministic 81 KB minimal input and shows that module-level splitting is a practical workaround for everything except that module.

Originally created by @ChanwheKim on GitHub (Oct 28, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7808 ### Before submitting - [x] I searched the open and closed issues for a related report (related: #6127, #4601, #6311, #2498 — all large-file heap/stall reports; this one narrows the trigger to a single 75 KB module). - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction (static JavaScript analysis) ### Observed behavior `rea analyze-javascript-application <extracted Chrome extension dir> --json` skips `service-worker.js` (774 KB) and `main-world.js` (240 KB) with: ``` <file>: Resource constraint during analyze_javascript_application: Owned JavaScript worker exhausted its N MiB V8 old-space heap (max_heap_mb) while analyzing <file> during static ``` for N = 3072, 12288 and 16384 (`--max-heap-mb` maximum). Time to failure 4–6 min per file. The other bundles of the same extension (e.g. `content-script.js`, 227 KB, 109 webpack modules) analyze fine in a few minutes. After splitting both bundles at webpack module boundaries (see Reproduction) and re-running with `--max-heap-mb 8192`: - `service-worker`: 10/11 chunks analyzed, 176/177 modules recovered. A single 298 KB module (puppeteer-core, exports `ExtensionTransport`, `connect`) passed with `examination: limited`. - `main-world`: 4/5 chunks analyzed, 142/143 modules recovered. - The only failing chunk in **both** bundles is the same webpack module `8555`: a ~75 KB minified IIFE XPath 1.0 implementation (tokenizer/parser tables; strings `XPath`, `XPATH_INVALID_PARSE`, …). Put alone in a directory it still fails: with `--max-heap-mb 2048` the worker exhausts the heap after ~27 s, `parsed_javascript_files: 0`. So the blow-up is a code pattern in that one module, not bundle size. Deterministic (3/3 runs on the full bundle at different heap sizes, 2/2 on the split chunk, 1/1 standalone). ### Reproduction Target is public: Chrome Web Store extension "Datadog test recorder" (id `kkbncfpddhdmkfmalecgnphegacgejoa`), version 4.5.1, unpacked from Chrome's `Extensions/<id>/4.5.1_0/` directory. 1. `npx -y rea-agents@latest analyze-javascript-application /abs/ext --json` → `service-worker.js` and `main-world.js` skipped with the heap limitation above (also with `--max-heap-mb 16384 --analysis-timeout-ms 1200000`). 2. Split at module boundaries. Both bundles are webpack 5 `(()=>{var e={3314(e,t){…},…};…r(entry)})()`. I used a small acorn script: parse, take the big `ObjectExpression` module map, group `Property` ranges into ≤60 KB chunks, and write each chunk as `head + "{" + modules + "}" + tail` where `head`/`tail` are the original text before/after the object (so each chunk keeps the real webpack bootstrap and runtime). Script: https://github.com/ChanwheKim (can paste it in a comment on request; ~40 lines). 3. `rea analyze-javascript-application /abs/split/main-world --max-heap-mb 8192 --json` → every chunk succeeds except the one holding module `8555`. 4. Minimal: a directory containing only that chunk (`(()=>{var e={8555(e,t){…}};<original runtime tail>})()`, 81,694 bytes) → `--max-heap-mb 2048 --analysis-timeout-ms 900000` fails after ~27 s with the same limitation. I can attach the 81 KB minimal file (it is third-party minified code shipped in the public extension) — say the word and I will add it as a gist link. ### Expected behavior Either the module analyzes (it parses fine with acorn, and REA parsed 318 other modules of the same bundles including a 298 KB one), or REA bails out of the offending phase for that file quickly with a typed partial result/limitation instead of consuming the whole worker heap for minutes. The docs' `max_heap_mb` remediation does not help here because the growth is unbounded relative to input size (75 KB → >16 GiB). ### Impact and scope Any application that bundles this XPath library (it is common in recorder/scraper tooling) loses the whole containing file from the application graph, and the user pays several minutes of CPU per attempt. With the default 1024 MiB heap the failure is also silent until the end of the run. Evidence integrity is otherwise fine: the skipped file is reported as `parse_status: skipped` with the limitation string. ### REA version rea-agents 6.8.0 (via `npx -y rea-agents@latest`) ### Runtime and platform Node v24.21.0; macOS 15.6 arm64 (Darwin 25.6.0); 36 GB RAM; no native provider involved. ### Target and artifact context Extracted Chrome MV3 extension directory; webpack 5 bundles (module-map style `var e={<id>(e,t,n){…}}`); `--artifact-format directory`. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction (the limitation says `during static`) ### Relevant output or evidence Full-bundle run (`--max-heap-mb 16384`, main-world.js only): ```json {"relevant_files": 8, "parsed_javascript_files": 0, "skipped_javascript_files": 1, "visited_ast_nodes": 0, "findings": 0, "modules": 0, "parse_failures": 0} "main-world.js: Resource constraint during analyze_javascript_application: Owned JavaScript worker exhausted its 16384 MiB V8 old-space heap (max_heap_mb) while analyzing main-world.js during static" ``` Split run (`--max-heap-mb 8192`, 5 chunks): ```json {"relevant_files": 6, "parsed_javascript_files": 4, "skipped_javascript_files": 1, "visited_ast_nodes": 50144, "findings": 245, "modules": 142, "parse_failures": 0} "chunk_02.js: Resource constraint … exhausted its 8192 MiB V8 old-space heap (max_heap_mb) while analyzing chunk_02.js during static" ``` Standalone module 8555 (`--max-heap-mb 2048`), 27 s: ```json {"relevant_files": 1, "text_bytes_read": 81694, "parsed_javascript_files": 0, "skipped_javascript_files": 1} "xpath-module-8555.js: Resource constraint … exhausted its 2048 MiB V8 old-space heap (max_heap_mb) while analyzing xpath-module-8555.js during static" ``` First 160 bytes of the module (for identification): ``` {8555(e,t){!function(e){"use strict";var t=function(e,t){for(var r=0;r<t.length;r+=1)e(t[r],r,t)},r=function(e,r,n){var i=r;return t((function(t,r){i=e(i,t,r)}) ``` ### Typed outcome or recovery expected A per-file `resource_constraint` limitation returned within seconds (not minutes) and the rest of the file set analyzed — which the split run already achieves once the module is isolated — plus, ideally, the other modules of the same file retained instead of the whole file being dropped. ### Analysis or suspected cause Hypothesis only (no REA source read): the module is one large IIFE whose body builds the XPath grammar as nested closures and tables (hundreds of small functions assigned into shared objects). Something in the static phase (alias/selection expansion or member-mutation traversal — cf. #2498) appears to grow super-linearly on that shape, since the 298 KB puppeteer-core module and 318 other modules from the same bundles are fine. ### Additional context Related reports: #6127 (6 GiB heap on 233 KB file), #4601 (277 KB webpack factory stall), #6311 (hang on 664 KB UMD), #2498 (member-mutation traversal 1.2 GB on openjpeg). This report adds a deterministic 81 KB minimal input and shows that module-level splitting is a practical workaround for everything except that module.

bsakaguchi closed this issue

2026-10-31 00:51:49 +00:00

bsakaguchi referenced this issue

2026-10-31 01:13:41 +00:00

[PR #4646] [MERGED] fix(javascript): keep directory-form HTML URL references distinct from files #6339

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425