morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #2498] [Perf] Semantic member-mutation traversal takes 84 s and 1.2 GB on pdf.js's 462 KB openjpeg_nowasm_fallback.js #828

New issue

Open

opened 2026-10-15 18:28:48 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-15 18:28:48 +00:00

Owner

Originally created by @kaoru0822-kitauji on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2498

Summary

analyze_javascript_application spends most of its time on one generated file that ships in pdf.js: wasm/openjpeg_nowasm_fallback.js, an Emscripten wasm2js build of 462,316 bytes. Analyzed alone, it takes 84 s and peaks at 1.24 GB RSS, yet yields 31 semantic nodes and 372 unknowns. Almost all of that time is in markValue inside collectSemanticMemberMutationsSteps (src/domain/javascript/javascriptSemanticMemberMutations.ts).

This isn't a correctness failure. But the file is common: any app bundling pdf.js 5.x carries it, and on a slower or loaded machine it approaches the default 300 s per-source deadline.

Reproduction (public input)

The file is byte-identical in pdfjs-dist@5.3.31 and pdfjs-dist@5.3.93 (SHA-256 53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9). Obsidian 1.12.7 also bundles it as lib/pdfjs/wasm/openjpeg_nowasm_fallback.js.

npm pack pdfjs-dist@5.3.93 && tar -xzf pdfjs-dist-5.3.93.tgz package/wasm/openjpeg_nowasm_fallback.js
mkdir app && cp package/wasm/openjpeg_nowasm_fallback.js app/
rea analyze-javascript-application ./app --format json

Measurements

Main at bb540826d, macOS 26 (Apple silicon), Node 26.5.0, local build.

Run	Wall	Max RSS
Whole `obsidian.asar` (24 MB, 36 sources)	135.4 s	1.34 GB
Its slowest source, this file	70.0 s of the above	—
`--source-path lib/pdfjs/wasm/openjpeg_nowasm_fallback.js`	84.0 s	1.24 GB
In-process phases for this file (no worker heap cap)	parse 0.13 s, open receivers 0.12 s, static 0.47 s, semantic 143–146 s	3.4–3.6 GB

A CPU profile of the in-process semantic phase (5 ms sampling) attributes 118 of 145 s (total time) to markValue. That time is spread across key building and lookups, not one hot call: JSON.stringify([...capture, current.path]) (15%), the pathState key (13%), JSON.stringify(capture) (12%), and coveringPaths.get (11%). resolveSemanticBindingState adds 17 s self time and GC 13 s.

I counted with temporary instrumentation (not committed) over the whole semantic phase:

Counter	Value
`markValue` calls	28,814
work-list iterations (`pending` items processed)	264,997,167 (≈ 9,200 per call)
largest `pending` length in one call	274,683
iterations that reach a bound identifier	77,631,161
mean `current.path` length	0.22
mean `current.bindings` size	6.8
items identical to an earlier item in the same call (same node, path, `bindings` Set, origin)	0

So the cost isn't repeated identical work that a per-call seen-set could drop. The 265 M items are distinct, mostly in their alias-chain bindings sets and origins, and each is cheap (about 0.5 µs). That points to the size of the explored alias/origin space for this code shape: wasm2js's huge single functions, heavy reuse of a few locals, and many HEAP32[...] = ... member writes. Faster key encoding alone would give a constant factor at best.

#1401: factorial member-reference propagation through reassignment loops, fixed earlier. This file still hits a large (if not factorial) case.
#1387 / #2097: large-bundle resource limits in the parent.

Not done

I haven't proposed an algorithm change: the antichain, selection union and repeated-prefix rules here are deliberate, and changing what they explore affects which mutations are reported. A maintainer who knows the intended lattice is better placed to choose a bound or a summary for this case.
I haven't measured other wasm2js outputs or Linux/Windows.

AI agent note

An AI agent (Claude Code) collected these measurements and wrote this issue during a scheduled maintenance loop. The instrumentation lived only in a local build output and was removed. No code changes were made.

Originally created by @kaoru0822-kitauji on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2498 ### Summary `analyze_javascript_application` spends most of its time on one generated file that ships in pdf.js: `wasm/openjpeg_nowasm_fallback.js`, an Emscripten wasm2js build of 462,316 bytes. Analyzed alone, it takes 84 s and peaks at 1.24 GB RSS, yet yields 31 semantic nodes and 372 unknowns. Almost all of that time is in `markValue` inside `collectSemanticMemberMutationsSteps` (`src/domain/javascript/javascriptSemanticMemberMutations.ts`). This isn't a correctness failure. But the file is common: any app bundling pdf.js 5.x carries it, and on a slower or loaded machine it approaches the default 300 s per-source deadline. ### Reproduction (public input) The file is byte-identical in `pdfjs-dist@5.3.31` and `pdfjs-dist@5.3.93` (SHA-256 `53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9`). Obsidian 1.12.7 also bundles it as `lib/pdfjs/wasm/openjpeg_nowasm_fallback.js`. ```sh npm pack pdfjs-dist@5.3.93 && tar -xzf pdfjs-dist-5.3.93.tgz package/wasm/openjpeg_nowasm_fallback.js mkdir app && cp package/wasm/openjpeg_nowasm_fallback.js app/ rea analyze-javascript-application ./app --format json ``` ### Measurements Main at bb540826d, macOS 26 (Apple silicon), Node 26.5.0, local build. | Run | Wall | Max RSS | | --- | ---: | ---: | | Whole `obsidian.asar` (24 MB, 36 sources) | 135.4 s | 1.34 GB | | Its slowest source, this file | 70.0 s of the above | — | | `--source-path lib/pdfjs/wasm/openjpeg_nowasm_fallback.js` | 84.0 s | 1.24 GB | | In-process phases for this file (no worker heap cap) | parse 0.13 s, open receivers 0.12 s, static 0.47 s, **semantic 143–146 s** | 3.4–3.6 GB | A CPU profile of the in-process semantic phase (5 ms sampling) attributes 118 of 145 s (total time) to `markValue`. That time is spread across key building and lookups, not one hot call: `JSON.stringify([...capture, current.path])` (15%), the `pathState` key (13%), `JSON.stringify(capture)` (12%), and `coveringPaths.get` (11%). `resolveSemanticBindingState` adds 17 s self time and GC 13 s. I counted with temporary instrumentation (not committed) over the whole semantic phase: | Counter | Value | | --- | ---: | | `markValue` calls | 28,814 | | work-list iterations (`pending` items processed) | 264,997,167 (≈ 9,200 per call) | | largest `pending` length in one call | 274,683 | | iterations that reach a bound identifier | 77,631,161 | | mean `current.path` length | 0.22 | | mean `current.bindings` size | 6.8 | | items identical to an earlier item in the same call (same node, path, `bindings` Set, origin) | **0** | So the cost isn't repeated identical work that a per-call seen-set could drop. The 265 M items are distinct, mostly in their alias-chain `bindings` sets and origins, and each is cheap (about 0.5 µs). That points to the size of the explored alias/origin space for this code shape: wasm2js's huge single functions, heavy reuse of a few locals, and many `HEAP32[...] = ...` member writes. Faster key encoding alone would give a constant factor at best. ### Related - #1401: factorial member-reference propagation through reassignment loops, fixed earlier. This file still hits a large (if not factorial) case. - #1387 / #2097: large-bundle resource limits in the parent. ### Not done - I haven't proposed an algorithm change: the antichain, selection union and repeated-prefix rules here are deliberate, and changing what they explore affects which mutations are reported. A maintainer who knows the intended lattice is better placed to choose a bound or a summary for this case. - I haven't measured other wasm2js outputs or Linux/Windows. ### AI agent note An AI agent (Claude Code) collected these measurements and wrote this issue during a scheduled maintenance loop. The instrumentation lived only in a local build output and was removed. No code changes were made.

bsakaguchi added the

area: native

performance

labels

2026-10-15 18:28:48 +00:00

bsakaguchi commented

2026-10-15 18:28:49 +00:00

Author

Owner

@gongfpp commented on GitHub (Oct 13, 2026):

Reproduced this and opened a draft PR for the semantic-phase part: #2941.

Host: macOS 26 (Apple silicon, 24 GiB), Node 24.19.0, local build at 3e7c6302, input = the report's public file (pdfjs-dist@5.3.93 → package/wasm/openjpeg_nowasm_fallback.js, 462,316 bytes, SHA-256 53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9) analyzed alone in a directory. The machine carried unrelated load (load average 5.0–6.5, swap in use), so treat these as single-host estimates.

Two things to add to the report

At the default worker heap (--max-heap-mb 1024) this input does not complete on my host. In both the base revision and with my patch, analyze-javascript-application returns resource_constraint: Owned JavaScript worker exhausted its heap while analyzing openjpeg_nowasm_fallback.js during semantic, coverage.status: "partial". Default-setting wall time here measures time to heap exhaustion at the same limit (median of 3: 144.4 s base, 123.7 s patched, ~1.14 GiB peak both), not completed work.
With the limit raised, the base revision needs more than the default 300 s per-source deadline: at --max-heap-mb 3072 it returns provider_timeout at 302.6 s wall, while the patched revision completes the same analysis in 240.6 s. Raising the deadline to 900 s to let the base finish gives 396.7 s base vs 240.6 s patched, with identical output (20,000 semantic nodes / 28,411 unknowns at the per-file node budget; the two completed CLI envelopes compare equal across 979,087 flattened paths once volatile identifiers and captured worker logs are excluded).

An in-process --cpu-prof of the semantic phase on base attributes self time to markValue 38.6%, resolveSemanticBindingState 15.7%, the initializer position helpers (directInitializerPosition / directMutationPosition / referenceInitializerPosition) ~15% and GC 8%.

What the patch does

resolveSemanticBindingState re-walked the scope chain for every visit of the same identifier node. Definition collection completes before this phase, and addBinding is reachable only from it (it returns early for incomplete scopes), so the result is memoizable per node for the phase.
Those three position helpers are pure functions of (node, parent map) but were recomputed for every mutation and initializer; they are now memoized per analysis parent map.
markValue re-encoded the same capture tuple with JSON.stringify and built a fresh long key for every Set/Map lookup, forcing a full string hash each time. The capture key is now cached per (binding, origin, owner) and the lifetime key is derived from it.

The semantic IR is byte-identical before and after, and the semantic phase drops from 272.9 s to 177.8 s in-process.

Still open: the memory ceiling. Peak RSS did not move measurably (1,169 MiB at default heap; ~3.4 GiB in-process) and the default-heap completion limit is unchanged, so the retained worklist / alias-chain state that sets it is a separate change from this one. If you would rather see that first, say so and I will look there instead.

@gongfpp commented on GitHub (Oct 13, 2026): Reproduced this and opened a draft PR for the semantic-phase part: #2941. Host: macOS 26 (Apple silicon, 24 GiB), Node 24.19.0, local build at `3e7c6302`, input = the report's public file (`pdfjs-dist@5.3.93` → `package/wasm/openjpeg_nowasm_fallback.js`, 462,316 bytes, SHA-256 `53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9`) analyzed alone in a directory. The machine carried unrelated load (load average 5.0–6.5, swap in use), so treat these as single-host estimates. ### Two things to add to the report 1. **At the default worker heap (`--max-heap-mb 1024`) this input does not complete on my host.** In both the base revision and with my patch, `analyze-javascript-application` returns `resource_constraint`: `Owned JavaScript worker exhausted its heap while analyzing openjpeg_nowasm_fallback.js during semantic`, `coverage.status: "partial"`. Default-setting wall time here measures *time to heap exhaustion at the same limit* (median of 3: **144.4 s** base, **123.7 s** patched, ~1.14 GiB peak both), not completed work. 2. With the limit raised, the base revision needs more than the default 300 s per-source deadline: at `--max-heap-mb 3072` it returns `provider_timeout` at 302.6 s wall, while the patched revision completes the same analysis in **240.6 s**. Raising the deadline to 900 s to let the base finish gives **396.7 s** base vs **240.6 s** patched, with identical output (20,000 semantic nodes / 28,411 unknowns at the per-file node budget; the two completed CLI envelopes compare equal across 979,087 flattened paths once volatile identifiers and captured worker logs are excluded). An in-process `--cpu-prof` of the semantic phase on base attributes self time to `markValue` 38.6%, `resolveSemanticBindingState` 15.7%, the initializer position helpers (`directInitializerPosition` / `directMutationPosition` / `referenceInitializerPosition`) ~15% and GC 8%. ### What the patch does 1. `resolveSemanticBindingState` re-walked the scope chain for every visit of the same identifier node. Definition collection completes before this phase, and `addBinding` is reachable only from it (it returns early for incomplete scopes), so the result is memoizable per node for the phase. 2. Those three position helpers are pure functions of (node, parent map) but were recomputed for every mutation and initializer; they are now memoized per analysis parent map. 3. `markValue` re-encoded the same capture tuple with `JSON.stringify` and built a fresh long key for every `Set`/`Map` lookup, forcing a full string hash each time. The capture key is now cached per (binding, origin, owner) and the lifetime key is derived from it. The semantic IR is byte-identical before and after, and the semantic phase drops from 272.9 s to 177.8 s in-process. **Still open:** the memory ceiling. Peak RSS did not move measurably (1,169 MiB at default heap; ~3.4 GiB in-process) and the default-heap completion limit is unchanged, so the retained worklist / alias-chain state that sets it is a separate change from this one. If you would rather see that first, say so and I will look there instead.

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi commented

2026-10-16 21:02:36 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Oct 16, 2026):

Update with measurements on main at 1bd5c5c87 (macOS 26 arm64, Node 26.5.0, same public input, SHA-256 53c22842…a0bd9). AI-assisted (Claude Code); temporary instrumentation lived only in a scratch copy of dist/.

Two PRs take part of the time cost:

#4321: since 018ab2c5d, the static phase also ran the whole member-mutation pass for any pending fetch(...) target. In this file the only target is the parameter readAsync = async url => fetch(url), which can never resolve. Static went from 0.47 s to 242 s and exhausted the default worker heap, so the file's static facts were lost. With the PR: 0.6 s, facts retained.
#4331: markValue selected capture origins (semanticMutationInitializers) before the coverage check that rejects ~97% of visits. Semantic phase: 249.9 s → 148.6 s with an identical IR digest. CLI with --max-heap-mb 6144: 590 s → 327 s, byte-identical JSON.

Where the remaining time and memory go (observed):

Stage of `collectSemanticMemberMutationsSteps`	Time	Heap after (forced GC)	Recorded effects
AST traversal	~14 s	399 MB	~300 k (291,907 escapes + 4,887 writes on bindings)
Fixpoint round 1 (43 pending references + 4,868 iterable references)	~109 s	2,033 MB	1,715,471
Fixpoint round 2	0.4 s	2,033 MB	0

Almost all of round 1 (115.6 of 124.3 s) is iterable references whose source is an Identifier under an AssignmentExpression. They are wasm2js typed-array stores such as H[e+16>>2]=b, with H = new Int32Array(...). Each store escapes the stored local (markValue(b, [], …, "escape", store)). That walks every assignment of b, many of the form b = H[...], back to H[null], and records up to ~1,900 escape effects per store. Each effect carries its store site as node, so nothing is deduplicated across the 4,868 stores, and escapedPaths grows to ~2 M entries. Those entries are what remains in the 2 GB. Every later binding evaluation also iterates them (javascriptSemanticValues.ts, for (const escape of binding.escapedPaths)).

Not done: collapsing these effects. Consumers use each effect's node for timing (semanticEscapeFollowsCapture, effectsForReferenceAt timings, originalBuiltinReceiver), so merging per-site effects is a lattice decision. Two directions a maintainer could choose between: (a) recognize that a value read from a typed-array element via a numeric key is primitive, so storing or escaping it has no referent; (b) summarize same-binding, same-path escapes from one function activation into one effect whose timing is the earliest site.

@kaoru0822-kitauji commented on GitHub (Oct 16, 2026): Update with measurements on main at `1bd5c5c87` (macOS 26 arm64, Node 26.5.0, same public input, SHA-256 `53c22842…a0bd9`). AI-assisted (Claude Code); temporary instrumentation lived only in a scratch copy of `dist/`. **Two PRs take part of the time cost:** - #4321: since `018ab2c5d`, the **static** phase also ran the whole member-mutation pass for any pending `fetch(...)` target. In this file the only target is the parameter `readAsync = async url => fetch(url)`, which can never resolve. Static went from 0.47 s to 242 s and exhausted the default worker heap, so the file's static facts were lost. With the PR: 0.6 s, facts retained. - #4331: `markValue` selected capture origins (`semanticMutationInitializers`) before the coverage check that rejects ~97% of visits. Semantic phase: 249.9 s → 148.6 s with an identical IR digest. CLI with `--max-heap-mb 6144`: 590 s → 327 s, byte-identical JSON. **Where the remaining time and memory go (observed):** | Stage of `collectSemanticMemberMutationsSteps` | Time | Heap after (forced GC) | Recorded effects | | --- | ---: | ---: | ---: | | AST traversal | ~14 s | 399 MB | ~300 k (291,907 escapes + 4,887 writes on bindings) | | Fixpoint round 1 (43 pending references + 4,868 iterable references) | ~109 s | 2,033 MB | **1,715,471** | | Fixpoint round 2 | 0.4 s | 2,033 MB | 0 | Almost all of round 1 (115.6 of 124.3 s) is iterable references whose source is an `Identifier` under an `AssignmentExpression`. They are wasm2js typed-array stores such as `H[e+16>>2]=b`, with `H = new Int32Array(...)`. Each store escapes the stored local (`markValue(b, [], …, "escape", store)`). That walks every assignment of `b`, many of the form `b = H[...]`, back to `H[null]`, and records up to ~1,900 escape effects per store. Each effect carries its store site as `node`, so nothing is deduplicated across the 4,868 stores, and `escapedPaths` grows to ~2 M entries. Those entries are what remains in the 2 GB. Every later binding evaluation also iterates them (`javascriptSemanticValues.ts`, `for (const escape of binding.escapedPaths)`). **Not done:** collapsing these effects. Consumers use each effect's node for timing (`semanticEscapeFollowsCapture`, `effectsForReferenceAt` timings, `originalBuiltinReceiver`), so merging per-site effects is a lattice decision. Two directions a maintainer could choose between: (a) recognize that a value read from a typed-array element via a numeric key is primitive, so storing or escaping it has no referent; (b) summarize same-binding, same-path escapes from one function activation into one effect whose timing is the earliest site.

bsakaguchi referenced this issue

2026-10-16 21:16:31 +00:00

[PR #828] [MERGED] refactor(domain): extract source-map envelope walk and disambiguate stringValue #2269

bsakaguchi commented

2026-10-31 00:29:44 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 20, 2026):

Current-main audit at c23e16162914e6bffc9549959c3faef0a66f6027: this input still does not complete within a caller-selected 600 s source deadline. No fix is proposed from this audit; this issue remains open.

Public input: pdfjs-dist@5.3.93/package/wasm/openjpeg_nowasm_fallback.js, 462,316 bytes, SHA-256 53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9; npm archive integrity verified. Actual public stdio MCP workflow: discover all 210 tools → analyze_javascript_application with the one selected source, detail: summary, max_heap_mb: 3072, analysis_timeout_ms: 600000 → retained summary and Evidence export when available. The target was parsed as inert text, never executed. Node 24.18.0/npm 11.16.0, Linux x86-64; fresh sessions, unflushed OS caches, one shared CPU, 6 GiB family hard/5 GiB early memory stop, no swap.

Attempt	Outcome	Whole wall s	Family CPU s	Kernel family peak MiB
Clean baseline	provider timeout	605.587	568.014	2331.3
Immutable-lookup candidate	provider timeout	607.655	579.742	2441.5
Whole-binding negative-result candidate	provider timeout	607.729	570.007	2569.3
Member-path negative-result candidate	provider timeout	610.980	602.505	2708.2

All payload exits are 1, observed cleanup is verified, and observed OOM/resource-stop events are zero. Peak accounting includes descendants and file cache. Final whole-scope counters are unavailable. Later controllers also retrieve the partial summary and export it; the first controller retained the timeout/reference but did not export its body, so these failed-task totals are not a matched completed-task comparison. Candidate partial exports contain the same ev_389b6645940cd9ac28dccc1ae98357e7ee04ee2848a54baf1b1dc7fe31750356, zero completed parsed files/visited AST nodes, one semantic root and 12 unknowns. None is a successful analysis.

The earliest measured bottleneck differs from the original report's markValue phase. A separate 30 s diagnostic profile, sampled at 5 ms, is dominated by createAmbientPathResolver from indexJavaScriptAmbientWritesSteps during static analysis. Diagnostic stage markers reach the final escape loop after under 0.6 s of traversal/store closure/writes. A subsequent candidate-only counter run finds 11,611 escape queries and at least 5,069 negative-result hits before its 30 s deadline. These are diagnostic observations, not a completed-task speedup or CPU-phase timing. They do not justify integrating a cache when the public task still times out.

The last candidate passed two test files / 201 tests and a runtime build. A completed 76-byte source-built fixture independently verifies function source position, parameter ownership and CommonJS export; its entire normalized result agrees before/after (8324b78c85bf55455c61dd3a2c143ed285063283b80e4f89a30c41356a56d921). Its one-run whole wall/CPU/peak is 7.170 s / 6.836 s / 753.3 MiB before and 16.509 s / 14.343 s / 740.1 MiB after: no established small-task gain either. Input-catalog + arguments + text-reply token proxies are unchanged at 198,894 (tiktoken 0.12.0, cl100k_base, all discovery and four calls counted). Actual model token usage is unmeasured. Large failed-attempt proxies differ because recovery calls were added, not because context became smaller.

#2941 already addresses earlier lookup work. Open #5139 addresses typed-array member propagation at another owner; I did not modify or merge it. Default 1 GiB behavior, full large-file IR equality, other platforms and extended-deadline completion remain unverified. The three candidate patches, package manifests, raw transcripts, diagnostic profile, failed attempts and resource receipts are preserved locally; none has been committed or integrated. The final experimental patch SHA-256 is 1c3da903329c39a8b9aef307dffbd8db344365eecb500591d6de599a2a7aac86.

@gensokyo-dev commented on GitHub (Oct 20, 2026): Current-main audit at `c23e16162914e6bffc9549959c3faef0a66f6027`: this input still does not complete within a caller-selected 600 s source deadline. No fix is proposed from this audit; this issue remains open. Public input: `pdfjs-dist@5.3.93/package/wasm/openjpeg_nowasm_fallback.js`, 462,316 bytes, SHA-256 `53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9`; npm archive integrity verified. Actual public stdio MCP workflow: discover all 210 tools → `analyze_javascript_application` with the one selected source, `detail: summary`, `max_heap_mb: 3072`, `analysis_timeout_ms: 600000` → retained summary and Evidence export when available. The target was parsed as inert text, never executed. Node 24.18.0/npm 11.16.0, Linux x86-64; fresh sessions, unflushed OS caches, one shared CPU, 6 GiB family hard/5 GiB early memory stop, no swap. | Attempt | Outcome | Whole wall s | Family CPU s | Kernel family peak MiB | | --- | --- | ---: | ---: | ---: | | Clean baseline | provider timeout | 605.587 | 568.014 | 2331.3 | | Immutable-lookup candidate | provider timeout | 607.655 | 579.742 | 2441.5 | | Whole-binding negative-result candidate | provider timeout | 607.729 | 570.007 | 2569.3 | | Member-path negative-result candidate | provider timeout | 610.980 | 602.505 | 2708.2 | All payload exits are 1, observed cleanup is verified, and observed OOM/resource-stop events are zero. Peak accounting includes descendants and file cache. Final whole-scope counters are unavailable. Later controllers also retrieve the partial summary and export it; the first controller retained the timeout/reference but did not export its body, so these failed-task totals are not a matched completed-task comparison. Candidate partial exports contain the same `ev_389b6645940cd9ac28dccc1ae98357e7ee04ee2848a54baf1b1dc7fe31750356`, zero completed parsed files/visited AST nodes, one semantic root and 12 unknowns. None is a successful analysis. The earliest measured bottleneck differs from the original report's `markValue` phase. A separate 30 s diagnostic profile, sampled at 5 ms, is dominated by `createAmbientPathResolver` from `indexJavaScriptAmbientWritesSteps` during static analysis. Diagnostic stage markers reach the final escape loop after under 0.6 s of traversal/store closure/writes. A subsequent candidate-only counter run finds 11,611 escape queries and at least 5,069 negative-result hits before its 30 s deadline. These are diagnostic observations, **not** a completed-task speedup or CPU-phase timing. They do not justify integrating a cache when the public task still times out. The last candidate passed two test files / 201 tests and a runtime build. A completed 76-byte source-built fixture independently verifies function source position, parameter ownership and CommonJS export; its entire normalized result agrees before/after (`8324b78c85bf55455c61dd3a2c143ed285063283b80e4f89a30c41356a56d921`). Its one-run whole wall/CPU/peak is 7.170 s / 6.836 s / 753.3 MiB before and 16.509 s / 14.343 s / 740.1 MiB after: no established small-task gain either. Input-catalog + arguments + text-reply token proxies are unchanged at 198,894 (`tiktoken 0.12.0`, `cl100k_base`, all discovery and four calls counted). Actual model token usage is unmeasured. Large failed-attempt proxies differ because recovery calls were added, not because context became smaller. #2941 already addresses earlier lookup work. Open #5139 addresses typed-array member propagation at another owner; I did not modify or merge it. Default 1 GiB behavior, full large-file IR equality, other platforms and extended-deadline completion remain unverified. The three candidate patches, package manifests, raw transcripts, diagnostic profile, failed attempts and resource receipts are preserved locally; none has been committed or integrated. The final experimental patch SHA-256 is `1c3da903329c39a8b9aef307dffbd8db344365eecb500591d6de599a2a7aac86`.

bsakaguchi commented

2026-11-03 23:47:04 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Nov 3, 2026):

TL;DR: This input still does not finish on main (c3647ade1) in my VM. At the default 1,024 MiB heap, the worker ran out of heap during the static phase after about 275 s, in both runs. With 3,072 MiB it hit the 300 s deadline instead, and no files were parsed either way.

I ran this input on current main (c3647ade1) in a Fedora 44 VM, with the CLI built from that tree. The input is wasm/openjpeg_nowasm_fallback.js from pdf.js (Apache-2.0), pdfjs-dist@5.3.93. It has 462,316 bytes, and its hash matches the issue's. It was parsed as inert text and never executed.

Run	Host, Node, commit	Setting	Outcome	Wall	Peak memory
Mine, 2 runs	Fedora 44 VM, 4 vCPUs, 8 GB; Node 24.22.1; `c3647ade1`	defaults (heap 1,024 MiB, deadline 300 s)	CLI exit 0, coverage `partial`; the heap ran out "during static"; 0 files parsed, 1 semantic node, 12 unknowns	275.1 s and 276.9 s	1,107 and 1,111 MiB (GNU time)
Mine, 1 run	same VM; Node 24.22.1; `c3647ade1`	`--max-heap-mb 3072`, deadline 300 s (the default)	CLI exit 1, `provider_timeout` at the deadline; coverage `partial`, 0 files parsed, 1 semantic node, 12 unknowns	301.5 s	2,387 MiB (GNU time)
kaoru0822-kitauji, issue body (secondhand)	macOS 26, Apple silicon; Node 26.5.0; `bb540826d`	`--source-path` on this file, defaults	finished; 31 semantic nodes, 372 unknowns	84.0 s	1.24 GB RSS
gongfpp, comment 6151435720 (secondhand)	macOS 26, Apple silicon, loaded host; Node 24.19.0; `3e7c6302`	defaults	`resource_constraint` "during semantic", coverage `partial`	144.4 s (median of 3)	about 1.14 GiB
gongfpp, same comment (secondhand)	same	`--max-heap-mb 3072`	`provider_timeout` at the 300 s deadline	302.6 s	not given
gensokyo-dev, comment 6290057244 (secondhand)	Linux x86-64, one shared CPU; Node 24.18.0; `c23e16162`	MCP, `max_heap_mb: 3072`, deadline 600 s	provider timeout, partial summary, 12 unknowns	605.6 s	2,331 MiB (kernel family peak)

The earlier rows are other people's numbers, from other hosts and commits. I did not rerun them. My peak is GNU time's maximum RSS for the command. The last row's is a kernel family peak, so the two differ. The 31/372 row is a finished run; mine is not, so the node and unknown counts do not compare.

What the code history shows

0c6c571a0 (#2941) says "Refs #2498". Its message reports a faster semantic phase on this input, 177.8 s instead of 272.9 s. Those are gongfpp's macOS numbers, not mine. #5139, #7131 and #8103 later changed the same file.

The stop has moved. The default run from gongfpp stopped "during semantic". Mine stopped "during static". An earlier comment describes a heap stop in the static phase. It names #4321 as the fix, and that PR closed unmerged. I did not check whether the cause is the same.

Not run

A repeat of the 3,072 MiB run.
A deadline longer than 300 s, such as the 600 s in the audit.
A heap above 3,072 MiB, a build of any earlier commit, a CPU profile, and per-phase timings.
Any other host or platform.
Any diff: I read the messages of #2941, #5139, #7131 and #8103, so I make no claim about which commit changed which number.

Test setup

Box: Fedora 44 VM (4 vCPUs, 8 GB) on my machine, kernel 7.2.11-200.fc44; host CPU AMD Ryzen 9 5900X
Node: 24.22.1
Head: c3647ade1 (main)
Input: pdfjs-dist@5.3.93 (Apache-2.0), npm archive, SHA-256 53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9, checked in the VM
Build: npm ci --offline (54 s), then npm run build:cached (12 s), no test:prepare
Shared box: one run at a time, nothing else running in the VM

Commands

All in the VM, Node 24.22.1 first on PATH, timeout in front, output saved. ~/i2498-main is the extracted tree.

npm --prefix ~/i2498-main/src ci --offline                          # exit 0, 54 s
npm --prefix ~/i2498-main/src run build:cached                      # exit 0, 12 s
/usr/bin/time -v timeout 400 node ~/i2498-main/src/scripts/rea.mjs analyze-javascript-application ~/targets/app --format json
  # twice: exit 0 both times, 275.14 s and 276.87 s wall, max RSS 1,133,928 and 1,137,700 KB
/usr/bin/time -v timeout 330 node ~/i2498-main/src/scripts/rea.mjs analyze-javascript-application ~/targets/app --format json --max-heap-mb 3072 --analysis-timeout-ms 300000
  # once: exit 1 (provider_timeout), 301.53 s wall, max RSS 2,443,920 KB

Flag names are from rea analyze-javascript-application --help in the same build. The CLI help gives defaults of 1,024 MiB for --max-heap-mb and 300,000 ms for --analysis-timeout-ms.

🤖 Written and tested with Claude Code (Sonnet 5.5, verified by Opus 5.5), posted with my OK 🦊

@CassieFox23 commented on GitHub (Nov 3, 2026): **TL;DR:** This input still does not finish on main (c3647ade1) in my VM. At the default 1,024 MiB heap, the worker ran out of heap during the static phase after about 275 s, in both runs. With 3,072 MiB it hit the 300 s deadline instead, and no files were parsed either way. I ran this input on current main (c3647ade1) in a Fedora 44 VM, with the CLI built from that tree. The input is `wasm/openjpeg_nowasm_fallback.js` from pdf.js (Apache-2.0), `pdfjs-dist@5.3.93`. It has 462,316 bytes, and its hash matches the issue's. It was parsed as inert text and never executed. | Run | Host, Node, commit | Setting | Outcome | Wall | Peak memory | | --- | --- | --- | --- | ---: | ---: | | Mine, 2 runs | Fedora 44 VM, 4 vCPUs, 8 GB; Node 24.22.1; c3647ade1 | defaults (heap 1,024 MiB, deadline 300 s) | CLI exit 0, coverage `partial`; the heap ran out "during static"; 0 files parsed, 1 semantic node, 12 unknowns | 275.1 s and 276.9 s | 1,107 and 1,111 MiB (GNU time) | | Mine, 1 run | same VM; Node 24.22.1; c3647ade1 | `--max-heap-mb 3072`, deadline 300 s (the default) | CLI exit 1, `provider_timeout` at the deadline; coverage `partial`, 0 files parsed, 1 semantic node, 12 unknowns | 301.5 s | 2,387 MiB (GNU time) | | kaoru0822-kitauji, issue body (secondhand) | macOS 26, Apple silicon; Node 26.5.0; bb540826d | `--source-path` on this file, defaults | finished; 31 semantic nodes, 372 unknowns | 84.0 s | 1.24 GB RSS | | gongfpp, comment 6151435720 (secondhand) | macOS 26, Apple silicon, loaded host; Node 24.19.0; 3e7c6302 | defaults | `resource_constraint` "during semantic", coverage `partial` | 144.4 s (median of 3) | about 1.14 GiB | | gongfpp, same comment (secondhand) | same | `--max-heap-mb 3072` | `provider_timeout` at the 300 s deadline | 302.6 s | not given | | gensokyo-dev, comment 6290057244 (secondhand) | Linux x86-64, one shared CPU; Node 24.18.0; c23e16162 | MCP, `max_heap_mb: 3072`, deadline 600 s | provider timeout, partial summary, 12 unknowns | 605.6 s | 2,331 MiB (kernel family peak) | The earlier rows are other people's numbers, from other hosts and commits. I did not rerun them. My peak is GNU time's maximum RSS for the command. The last row's is a kernel family peak, so the two differ. The 31/372 row is a finished run; mine is not, so the node and unknown counts do not compare. ### What the code history shows 0c6c571a0 (#2941) says "Refs #2498". Its message reports a faster semantic phase on this input, 177.8 s instead of 272.9 s. Those are gongfpp's macOS numbers, not mine. #5139, #7131 and #8103 later changed the same file. The stop has moved. The default run from gongfpp stopped "during semantic". Mine stopped "during static". An [earlier comment](https://github.com/morluto/rea/issues/2498#issuecomment-6206600543) describes a heap stop in the static phase. It names #4321 as the fix, and that PR closed unmerged. I did not check whether the cause is the same. ### Not run - A repeat of the 3,072 MiB run. - A deadline longer than 300 s, such as the 600 s in the audit. - A heap above 3,072 MiB, a build of any earlier commit, a CPU profile, and per-phase timings. - Any other host or platform. - Any diff: I read the messages of #2941, #5139, #7131 and #8103, so I make no claim about which commit changed which number. <details> <summary>Test setup</summary> - Box: Fedora 44 VM (4 vCPUs, 8 GB) on my machine, kernel 7.2.11-200.fc44; host CPU AMD Ryzen 9 5900X - Node: 24.22.1 - Head: c3647ade1 (main) - Input: pdfjs-dist@5.3.93 (Apache-2.0), npm archive, SHA-256 53c228428a8861035a34738966af263fc9d4cee6df25a0ad942af947469a0bd9, checked in the VM - Build: `npm ci --offline` (54 s), then `npm run build:cached` (12 s), no `test:prepare` - Shared box: one run at a time, nothing else running in the VM </details> <details> <summary>Commands</summary> All in the VM, Node 24.22.1 first on PATH, `timeout` in front, output saved. `~/i2498-main` is the extracted tree. ``` npm --prefix ~/i2498-main/src ci --offline # exit 0, 54 s npm --prefix ~/i2498-main/src run build:cached # exit 0, 12 s /usr/bin/time -v timeout 400 node ~/i2498-main/src/scripts/rea.mjs analyze-javascript-application ~/targets/app --format json # twice: exit 0 both times, 275.14 s and 276.87 s wall, max RSS 1,133,928 and 1,137,700 KB /usr/bin/time -v timeout 330 node ~/i2498-main/src/scripts/rea.mjs analyze-javascript-application ~/targets/app --format json --max-heap-mb 3072 --analysis-timeout-ms 300000 # once: exit 1 (provider_timeout), 301.53 s wall, max RSS 2,443,920 KB ``` Flag names are from `rea analyze-javascript-application --help` in the same build. The CLI help gives defaults of 1,024 MiB for `--max-heap-mb` and 300,000 ms for `--analysis-timeout-ms`. </details> 🤖 Written and tested with Claude Code (Sonnet 5.5, verified by Opus 5.5), posted with my OK 🦊

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#828

No description provided.

Rows
Columns