morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7736] [Bug] analyze_javascript_application fails and drops static facts for the remaining files when the estimated parent budget runs out, while the parent heap is mostly free #4629

New issue

Closed

opened 2026-10-31 00:51:34 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:51:34 +00:00

Owner

Originally created by @jgt87 on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7736

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Reproduced on rea-agents 6.8.0 (released) and on main 6a226bd09.

Same fixture (301 files, 8.5 MB, 456 KB chunk sorted late, `--max-heap-mb 4096`)	6.8.0 (npm, Node 26.12.1), 2 runs	main `6a226bd09` (source, Node 24.22.1), 2 runs
Exit code	1, 1	1, 1
File where the budget ran out	chunk283, chunk264	chunk284, chunk284
Files without static facts (`skipped_javascript_files`)	88, 109 of 301	87, 87 of 301
Largest chunk (467,444 bytes) has static facts	no, no	no, no

The point of failure depends on live heap readings, so it moves between runs.
A smaller 201-file build (5.8 MB) is borderline: 6.8.0 completed it once
(exit 0, all files parsed, 184 files with a refused projection); main
failed it in 2 of 3 runs (skipped_javascript_files 7 and 8).

Area

JavaScript application analysis (analyze_javascript_application), parent result admission.

Observed behavior

The command ends with exit code 1:

{"code":"resource_constraint","message":"The parent's result capacity was exhausted after retaining static facts for chunk283-1s6nj5.js; remaining sources were not analyzed","details":{"reported_limits":{"parent_result_budget_bytes":...,"parent_result_budget_exhausted":true}}}

The partial_observation has static facts only for the files analyzed before
the failure. The rest, including the largest chunk, are reported with
syntax.status: "not-examined" and no static facts. In an instrumented run of
main on the 201-file build, the parent heap held about 351 MiB
(heapUsed) when the failure was raised; the parent V8 heap limit is about
4.2 GB.

The remediation text is correct that max_heap_mb does not help. Raising the
parent heap with NODE_OPTIONS=--max-old-space-size would move the point of
failure, but the budget is not being used by retained data.

Analysis

Same root cause as #7735: the admission charges accumulate estimates,
and estimateJavaScriptStaticApplicationProjection charges about 25x the
serialized application relationships it admits. Once the estimated total passes
parent_result_budget_bytes, the next static checkpoint sets
#parentCapacityExhausted in JavaScriptAnalysisWorker.#readCheckpoint, and the
remaining sources are not analyzed. Static facts are cheap (6x checkpoint
bytes, a few hundred KB per file here), but they are refused because the
budget was consumed by projection estimates of earlier files.

Reproduction

Synthetic fixture only (generator gen-vite-like.mjs, deterministic):

node gen-vite-like.mjs ./fx/vite300 300 456 last   # 301 files, 8,500,279 bytes
rea analyze-javascript-application ./fx/vite300/assets \
  --artifact-format directory --max-heap-mb 4096 --format json > out.json
echo $?   # 1

See .details.partial_observation.normalized_result.statistics
(parsed_javascript_files, skipped_javascript_files).

Expected behavior

Static facts for every file are retained when the parent heap can hold them.
The analysis completes with partial semantic coverage instead of failing.

Impact and scope

In the original (non-synthetic) case on 6.4.0, 25 of 87 chunks had no static
facts, including the largest chunk (456 KB). Which files lose their facts
depends on processing order and on the live heap.

REA version

rea-agents 6.8.0 (npm) and upstream main at 6a226bd09. First observed on 6.4.0 (MCP).

Runtime and platform

Windows 11 Pro (10.0.26300). 6.8.0: global npm install on Node.js 26.12.1.
main: source checkout on Node.js 24.22.1.

Interface

CLI (rea analyze-javascript-application); first seen through MCP.

Typed outcome or recovery expected

With the change proposed for #7735 (fix/application-projection-estimate):
the 301-file build still fails (exit 1, budget ran out at chunk65), but 266 files keep static facts instead of 192 to 214 (35 skipped instead of 87 to 109). On the 201-file build it completed (exit 0, all 201 files with static facts). So that change reduces this failure but does not close it: once the estimated budget is used, the remaining files still lose their static facts. A complete fix needs static facts to be admitted on their own (they cost about 6x the checkpoint bytes), separately from the projection and semantic budgets. That is a design decision for the maintainers; no PR is proposed for it.

gen-vite-like.mjs (the synthetic fixture generator)

// Generates a synthetic Vite-like build output. No real application code.
// Usage: node gen-vite-like.mjs <outDir> [chunkCount] [bigChunkKB] [first|last]
// Writes <outDir>/index.html and <outDir>/assets/*.js. Each chunk has an
// absolute-base __vite__mapDeps table, real dynamic imports, localStorage
// get/set calls and api.rpc("proc_N_M") calls. Deterministic output.
import { mkdirSync, writeFileSync, rmSync } from "node:fs";
import { join } from "node:path";

const outDir = process.argv[2];
const chunkCount = Number(process.argv[3] ?? 87);
const bigKB = Number(process.argv[4] ?? 456);
if (!outDir) throw new Error("outDir required");
rmSync(outDir, { recursive: true, force: true });
const assets = join(outDir, "assets");
mkdirSync(assets, { recursive: true });

let seed = 7;
const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
const names = Array.from({ length: chunkCount }, (_, i) => `chunk${String(i).padStart(2, "0")}-${(i * 2654435761 >>> 0).toString(36).slice(0, 6)}.js`);
const entry = "index-a1b2c3.js";

function body(i, targetBytes) {
  const parts = [];
  let n = 0;
  let size = 0;
  while (size < targetBytes) {
    const f = `f${i}_${n}`;
    const key = `demo.pref.${i}.${n % 5}`;
    const fn = `export function ${f}(a,b){const c=a*${n}+b;` +
      `if(c>${Math.floor(rand() * 1000)}){localStorage.setItem("${key}",String(c));}` +
      `const d=localStorage.getItem("${key}");` +
      `return api.rpc("proc_${i}_${n % 7}",{v:c,d}).then(r=>r.data);}\n`;
    parts.push(fn);
    size += fn.length;
    n++;
  }
  return parts.join("");
}

const total = [];
const deps = names.map((n) => `assets/${n}`);
for (let i = 0; i < chunkCount; i++) {
  const bigIndex = process.argv[5] === "last" ? chunkCount - 1 : 0;
  const target = i === bigIndex ? bigKB * 1024 : 20 * 1024 + Math.floor(rand() * 12 * 1024);
  const next = (i + 1) % chunkCount;
  const other = (i + 7) % chunkCount;
  const head =
    `import{api}from"./${entry}";\n` +
    `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify([deps[next], deps[other]])})))=>i.map(i=>d[i]);\n` +
    `export const load${i}=()=>import("./${names[next]}");\n` +
    `export const pre${i}=()=>api.preload(()=>import("./${names[other]}"),__vite__mapDeps([0,1]));\n`;
  const text = head + body(i, target);
  writeFileSync(join(assets, names[i]), text);
  total.push(text.length);
}
const entryText =
  `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify(deps)})))=>i.map(i=>d[i]);\n` +
  `export const api={rpc(name,args){return fetch("/rpc/"+name,{method:"POST",body:JSON.stringify(args)}).then(r=>r.json());},preload(f,d){return f();}};\n` +
  names.map((n, i) => `export const route${i}=()=>api.preload(()=>import("./${n}"),__vite__mapDeps([${i}]));\n`).join("") +
  `const saved=localStorage.getItem("demo.session.token");if(saved){route0();}\n`;
writeFileSync(join(assets, entry), entryText);
writeFileSync(join(outDir, "index.html"), `<!doctype html><script type="module" src="/assets/${entry}"></script>`);
const sum = total.reduce((a, b) => a + b, 0) + entryText.length;
console.log(JSON.stringify({ files: chunkCount + 1, bytes: sum, biggest: Math.max(...total) }));

Originally created by @jgt87 on GitHub (Oct 28, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7736 Originally assigned to: @kaoru0822-kitauji on GitHub. ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. Reproduced on rea-agents 6.8.0 (released) and on main 6a226bd09. | Same fixture (301 files, 8.5 MB, 456 KB chunk sorted late, `--max-heap-mb 4096`) | 6.8.0 (npm, Node 26.12.1), 2 runs | main 6a226bd09 (source, Node 24.22.1), 2 runs | | --- | --- | --- | | Exit code | 1, 1 | 1, 1 | | File where the budget ran out | chunk283, chunk264 | chunk284, chunk284 | | Files without static facts (`skipped_javascript_files`) | 88, 109 of 301 | 87, 87 of 301 | | Largest chunk (467,444 bytes) has static facts | no, no | no, no | The point of failure depends on live heap readings, so it moves between runs. A smaller 201-file build (5.8 MB) is borderline: 6.8.0 completed it once (exit 0, all files parsed, 184 files with a refused projection); main failed it in 2 of 3 runs (`skipped_javascript_files` 7 and 8). ## Area JavaScript application analysis (`analyze_javascript_application`), parent result admission. ## Observed behavior The command ends with exit code 1: ```json {"code":"resource_constraint","message":"The parent's result capacity was exhausted after retaining static facts for chunk283-1s6nj5.js; remaining sources were not analyzed","details":{"reported_limits":{"parent_result_budget_bytes":...,"parent_result_budget_exhausted":true}}} ``` The `partial_observation` has static facts only for the files analyzed before the failure. The rest, including the largest chunk, are reported with `syntax.status: "not-examined"` and no static facts. In an instrumented run of main on the 201-file build, the parent heap held about 351 MiB (`heapUsed`) when the failure was raised; the parent V8 heap limit is about 4.2 GB. The remediation text is correct that `max_heap_mb` does not help. Raising the parent heap with `NODE_OPTIONS=--max-old-space-size` would move the point of failure, but the budget is not being used by retained data. ## Analysis Same root cause as #7735: the admission charges accumulate estimates, and `estimateJavaScriptStaticApplicationProjection` charges about 25x the serialized application relationships it admits. Once the estimated total passes `parent_result_budget_bytes`, the next static checkpoint sets `#parentCapacityExhausted` in `JavaScriptAnalysisWorker.#readCheckpoint`, and the remaining sources are not analyzed. Static facts are cheap (6x checkpoint bytes, a few hundred KB per file here), but they are refused because the budget was consumed by projection estimates of earlier files. ## Reproduction Synthetic fixture only (generator `gen-vite-like.mjs`, deterministic): ```sh node gen-vite-like.mjs ./fx/vite300 300 456 last # 301 files, 8,500,279 bytes rea analyze-javascript-application ./fx/vite300/assets \ --artifact-format directory --max-heap-mb 4096 --format json > out.json echo $? # 1 ``` See `.details.partial_observation.normalized_result.statistics` (`parsed_javascript_files`, `skipped_javascript_files`). ## Expected behavior Static facts for every file are retained when the parent heap can hold them. The analysis completes with partial semantic coverage instead of failing. ## Impact and scope In the original (non-synthetic) case on 6.4.0, 25 of 87 chunks had no static facts, including the largest chunk (456 KB). Which files lose their facts depends on processing order and on the live heap. ## REA version rea-agents 6.8.0 (npm) and upstream `main` at 6a226bd09. First observed on 6.4.0 (MCP). ## Runtime and platform Windows 11 Pro (10.0.26300). 6.8.0: global npm install on Node.js 26.12.1. main: source checkout on Node.js 24.22.1. ## Interface CLI (`rea analyze-javascript-application`); first seen through MCP. ## Typed outcome or recovery expected With the change proposed for #7735 (fix/application-projection-estimate): the 301-file build still fails (exit 1, budget ran out at chunk65), but 266 files keep static facts instead of 192 to 214 (35 skipped instead of 87 to 109). On the 201-file build it completed (exit 0, all 201 files with static facts). So that change reduces this failure but does not close it: once the estimated budget is used, the remaining files still lose their static facts. A complete fix needs static facts to be admitted on their own (they cost about 6x the checkpoint bytes), separately from the projection and semantic budgets. That is a design decision for the maintainers; no PR is proposed for it. <details><summary><code>gen-vite-like.mjs</code> (the synthetic fixture generator)</summary> ```js // Generates a synthetic Vite-like build output. No real application code. // Usage: node gen-vite-like.mjs <outDir> [chunkCount] [bigChunkKB] [first|last] // Writes <outDir>/index.html and <outDir>/assets/*.js. Each chunk has an // absolute-base __vite__mapDeps table, real dynamic imports, localStorage // get/set calls and api.rpc("proc_N_M") calls. Deterministic output. import { mkdirSync, writeFileSync, rmSync } from "node:fs"; import { join } from "node:path"; const outDir = process.argv[2]; const chunkCount = Number(process.argv[3] ?? 87); const bigKB = Number(process.argv[4] ?? 456); if (!outDir) throw new Error("outDir required"); rmSync(outDir, { recursive: true, force: true }); const assets = join(outDir, "assets"); mkdirSync(assets, { recursive: true }); let seed = 7; const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648); const names = Array.from({ length: chunkCount }, (_, i) => `chunk${String(i).padStart(2, "0")}-${(i * 2654435761 >>> 0).toString(36).slice(0, 6)}.js`); const entry = "index-a1b2c3.js"; function body(i, targetBytes) { const parts = []; let n = 0; let size = 0; while (size < targetBytes) { const f = `f${i}_${n}`; const key = `demo.pref.${i}.${n % 5}`; const fn = `export function ${f}(a,b){const c=a*${n}+b;` + `if(c>${Math.floor(rand() * 1000)}){localStorage.setItem("${key}",String(c));}` + `const d=localStorage.getItem("${key}");` + `return api.rpc("proc_${i}_${n % 7}",{v:c,d}).then(r=>r.data);}\n`; parts.push(fn); size += fn.length; n++; } return parts.join(""); } const total = []; const deps = names.map((n) => `assets/${n}`); for (let i = 0; i < chunkCount; i++) { const bigIndex = process.argv[5] === "last" ? chunkCount - 1 : 0; const target = i === bigIndex ? bigKB * 1024 : 20 * 1024 + Math.floor(rand() * 12 * 1024); const next = (i + 1) % chunkCount; const other = (i + 7) % chunkCount; const head = `import{api}from"./${entry}";\n` + `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify([deps[next], deps[other]])})))=>i.map(i=>d[i]);\n` + `export const load${i}=()=>import("./${names[next]}");\n` + `export const pre${i}=()=>api.preload(()=>import("./${names[other]}"),__vite__mapDeps([0,1]));\n`; const text = head + body(i, target); writeFileSync(join(assets, names[i]), text); total.push(text.length); } const entryText = `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify(deps)})))=>i.map(i=>d[i]);\n` + `export const api={rpc(name,args){return fetch("/rpc/"+name,{method:"POST",body:JSON.stringify(args)}).then(r=>r.json());},preload(f,d){return f();}};\n` + names.map((n, i) => `export const route${i}=()=>api.preload(()=>import("./${n}"),__vite__mapDeps([${i}]));\n`).join("") + `const saved=localStorage.getItem("demo.session.token");if(saved){route0();}\n`; writeFileSync(join(assets, entry), entryText); writeFileSync(join(outDir, "index.html"), `<!doctype html><script type="module" src="/assets/${entry}"></script>`); const sum = total.reduce((a, b) => a + b, 0) + entryText.length; console.log(JSON.stringify({ files: chunkCount + 1, bytes: sum, biggest: Math.max(...total) })); ``` </details>

bsakaguchi referenced this issue

2026-10-31 01:14:07 +00:00

[PR #4757] [MERGED] fix(javascript): report oversized recovery inputs as file-size limits #6391

bsakaguchi closed this issue

2026-11-03 00:29:37 +00:00

bsakaguchi commented

2026-11-03 00:29:38 +00:00

Author

Owner

@kaoru0822-kitauji commented on GitHub (Nov 2, 2026):

Checked on 8250bf232 (macOS, Node 24.22.1, parent heap limit 4288 MiB).

The exact recipe no longer fails here. The 301-file fixture exited 0 with 301/301 files in 4 runs, and the 201-file fixture exited 0 with 201/201 in 1 run. #7744 and #7986 cover that shape.
The root cause is still there for earlier charges that cannot be reclaimed. A 501-file fixture with vendor-style chunks sorted first exited 1 with The parent's result capacity was exhausted in 3 of 3 runs, with 90 to 98 files losing their static facts and sampled parent heap at 1.2 to 1.3 GiB.

#8648 addresses that for CLI stdout, with the generator and measurements in its description. It does not make MCP or --evidence_output deliver these fixtures. They still stop at later Evidence stages on both builds (record_evidence retention, and #8411). I did not run Windows, which you reported on.

@kaoru0822-kitauji commented on GitHub (Nov 2, 2026): Checked on `8250bf232` (macOS, Node 24.22.1, parent heap limit 4288 MiB). - The exact recipe no longer fails here. The 301-file fixture exited 0 with 301/301 files in 4 runs, and the 201-file fixture exited 0 with 201/201 in 1 run. #7744 and #7986 cover that shape. - The root cause is still there for earlier charges that cannot be reclaimed. A 501-file fixture with vendor-style chunks sorted first exited 1 with `The parent's result capacity was exhausted` in 3 of 3 runs, with 90 to 98 files losing their static facts and sampled parent heap at 1.2 to 1.3 GiB. #8648 addresses that for CLI stdout, with the generator and measurements in its description. It does not make MCP or `--evidence_output` deliver these fixtures. They still stop at later Evidence stages on both builds (`record_evidence` retention, and #8411). I did not run Windows, which you reported on.

bsakaguchi commented

2026-11-03 00:29:38 +00:00

Author

Owner

@jgt87 commented on GitHub (Nov 2, 2026):

Additional real-world data point for this failure, on MCP + Windows (the combination not yet checked above). Observed once, on released 6.8.0; I have not retested on main / a build containing #7744 or #7986.

Environment: rea-agents 6.8.0 (npm), MCP interface, Windows 11 x64. Server Node version and startup heap flags were not recorded.

Input: a directory of exported browser JavaScript sources from a large production web app (bundled Webpack-style chunks): 205 files, 33,554,325 bytes (~32 MiB), 0 invalid UTF-8. The sources are private and can't be shared, and there is no public fixture.

Request:

{
  "format": "directory",
  "detail": "summary",
  "max_semantic_nodes": 50000,
  "max_semantic_nodes_per_file": 1000,
  "max_semantic_unknowns": 5000,
  "max_semantic_unknowns_per_file": 100,
  "analysis_timeout_ms": 60000
}

(source_paths and max_heap_mb omitted.)

Result: the same error as this issue:

{
  "code": "resource_constraint",
  "retryable": false,
  "message": "The parent's result capacity was exhausted after retaining static facts for <chunk>.js; remaining sources were not analyzed",
  "details": {
    "resource": "memory",
    "reported_limits": {
      "worker_old_space_mb": 1024,
      "worker_heap_limit_bytes": 1098907648,
      "worker_timeout_ms": 60000,
      "parent_result_budget_bytes": 1946859724,
      "parent_result_budget_exhausted": true
    },
    "partial_observation": { "kind": "retained-evidence" }
  }
}

The retained partial evidence summary showed:

Field	Value
Parsed JavaScript files	109
Skipped JavaScript files	96
Parse failures	0
Complete / limited / not-examined source examinations	87 / 22 / 96
Visited AST nodes	3,285,555
Findings	40,828
Modules	6,500

Several file-level limitations said that complete static facts had been retained on the file node while the expanded application relationships exceeded the parent's result budget. That matches the #7735 / #7744 symptom. The javascript-routes → sources view correctly listed all 205 rows: 109 complete, 96 unavailable with reasons attributing the omission to the interrupted run. That coverage reporting was useful for recovery.

No worker timeout or deadline was involved in the original failure. I did not sample the parent heap, so I can't confirm the "budget used up while the heap is mostly free" part directly.

Workaround that worked: re-running only the 96 unavailable paths via source_paths in batches of 12 (smaller semantic limits). Batches that finished did not hit the parent-capacity error again. The remaining gaps were different limits: per-source provider_timeouts on a few large vendor/framework chunks (even at 180000 ms) and one chunk exhausting the 1024 MiB worker old-space heap. Final coverage was 200/205 parsed. Those are separate from this issue and are mentioned only for context.

If useful, I can retest the same capture over MCP on Windows once a release includes #7744 / #7986.

@jgt87 commented on GitHub (Nov 2, 2026): Additional real-world data point for this failure, on **MCP + Windows** (the combination not yet checked above). Observed once, on released 6.8.0; I have not retested on main / a build containing #7744 or #7986. **Environment:** rea-agents 6.8.0 (npm), MCP interface, Windows 11 x64. Server Node version and startup heap flags were not recorded. **Input:** a directory of exported browser JavaScript sources from a large production web app (bundled Webpack-style chunks): 205 files, 33,554,325 bytes (~32 MiB), 0 invalid UTF-8. The sources are private and can't be shared, and there is no public fixture. **Request:** ```json { "format": "directory", "detail": "summary", "max_semantic_nodes": 50000, "max_semantic_nodes_per_file": 1000, "max_semantic_unknowns": 5000, "max_semantic_unknowns_per_file": 100, "analysis_timeout_ms": 60000 } ``` (`source_paths` and `max_heap_mb` omitted.) **Result:** the same error as this issue: ```json { "code": "resource_constraint", "retryable": false, "message": "The parent's result capacity was exhausted after retaining static facts for <chunk>.js; remaining sources were not analyzed", "details": { "resource": "memory", "reported_limits": { "worker_old_space_mb": 1024, "worker_heap_limit_bytes": 1098907648, "worker_timeout_ms": 60000, "parent_result_budget_bytes": 1946859724, "parent_result_budget_exhausted": true }, "partial_observation": { "kind": "retained-evidence" } } } ``` The retained partial evidence summary showed: | Field | Value | |---|---:| | Parsed JavaScript files | 109 | | Skipped JavaScript files | 96 | | Parse failures | 0 | | Complete / limited / not-examined source examinations | 87 / 22 / 96 | | Visited AST nodes | 3,285,555 | | Findings | 40,828 | | Modules | 6,500 | Several file-level limitations said that complete static facts had been retained on the file node while the expanded application relationships exceeded the parent's result budget. That matches the #7735 / #7744 symptom. The `javascript-routes` → `sources` view correctly listed all 205 rows: 109 `complete`, 96 `unavailable` with reasons attributing the omission to the interrupted run. That coverage reporting was useful for recovery. No worker timeout or deadline was involved in the original failure. I did not sample the parent heap, so I can't confirm the "budget used up while the heap is mostly free" part directly. **Workaround that worked:** re-running only the 96 unavailable paths via `source_paths` in batches of 12 (smaller semantic limits). Batches that finished did not hit the parent-capacity error again. The remaining gaps were *different* limits: per-source `provider_timeout`s on a few large vendor/framework chunks (even at 180000 ms) and one chunk exhausting the 1024 MiB worker old-space heap. Final coverage was 200/205 parsed. Those are separate from this issue and are mentioned only for context. If useful, I can retest the same capture over MCP on Windows once a release includes #7744 / #7986.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4629

No description provided.

Rows
Columns