[GH-ISSUE #7735] [Bug] analyze_javascript_application: application-projection estimate uses up the parent result budget after ~10 files; later files get a 0-byte semantic budget #4627

New issue

Closed

opened 2026-10-31 00:51:33 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:51:33 +00:00

Owner

Originally created by @jgt87 on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7735

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Reproduced on rea-agents 6.8.0 (released) and on main 6a226bd09.

Same fixture (88 files, 456 KB chunk first, `--max-heap-mb 4096`)	6.8.0 (npm, Node 26.12.1)	main `6a226bd09` (source, Node 24.22.1)
`parent_result_budget_bytes`	1,995,849,968	2,082,939,876
Files with refused application projection	77 of 88	78 of 88
Files offered `semantic_result_budget_bytes: 0`	77	78
Lowest / highest `parent_available_result_budget_bytes` reported	0 / 37,098,550	793,106 / 23,723,138
Files with a semantic graph	8	10

The estimate formula below is the same in the 6.8.0 package and on main. The
per-file charge table further down comes from a logging-only instrumented build
of main baec375c3; the commits up to 6a226bd09 do not touch src/javascript,
src/domain/javascript or src/application/javascript.

Area

JavaScript application analysis (analyze_javascript_application), parent result admission.

Observed behavior

On a synthetic Vite-style build of 88 files (2.8 MB, one 456 KB chunk), only the
first 10 files are projected into the application graph and get a semantic
graph. For the other 78 files REA reports:

Resource constraint during analyze_javascript_application: Expanded application relationships for chunkNN.js exceed the parent's result budget; complete static facts were retained on the file node
chunkNN.js: Resource constraint during analyze_javascript_application: The parent reserved its remaining result capacity for static observations; semantic analysis of chunkNN.js was not started

with reported_limits such as:

{"parent_result_budget_bytes":2083223972,"parent_available_result_budget_bytes":6432706,"application_projection_required_bytes":43126016}
{"worker_old_space_mb":4096,"worker_heap_limit_bytes":4320133120,"worker_timeout_ms":300000,"semantic_result_budget_bytes":0}

So 2.08 GB of parent budget is reported, but only 3 to 23 MB is available after
10 files, and the next 78 files are offered a semantic budget of 0 bytes. At
that point the parent process heap holds about 300 MiB (process.memoryUsage().heapUsed),
and its peak RSS for the whole run is 1,199 MiB. The parent V8 heap limit is
about 4.2 GB.

Analysis

I instrumented the built CLI locally (logging only) to print each admission
charge. The largest single charge is estimateJavaScriptStaticApplicationProjection
(src/domain/javascript/javascriptApplicationProjectionResources.ts):

const metadataBytes = 2048 + Buffer.byteLength(sourcePath) * 4;
return (checkpointBytes * 4 + repeatedBytes + records * metadataBytes) * 8;

records is 8 per finding, so each finding is charged about
8 × 2,100 × 8 = 134 KB. For the 456 KB chunk (855,843-byte static checkpoint)
the estimate is 645,656,672 bytes. Analyzed alone, the same chunk produces an
application graph of 25,274,686 bytes of JSON (2,320 nodes, 6,923 edges), so
the charge is about 25x the serialized relationships it admits. The 2 KB
per-record allowance is already about 4.5x the measured ~0.46 KB per record;
the outer * 8 expands it a second time.

Charges for the first files (bytes):

File	Static checkpoint	Projection charge	Transfer charge (module + graph)	Total charged after file
chunk00 (456 KB)	855,843	645,656,672	27,503,464 + 296,630,464	974,925,658
chunk01 (26 KB)	46,944	35,764,480	1,477,912 + 74,388,400	1,086,838,114
chunk08 (31 KB)	61,704	47,070,720	1,948,952 + 98,232,496	1,861,349,050
chunk09 onward		refused (avail 24 MB)	not started	~2,009,000,000

The 0-byte semantic budget is deliberate in JavaScriptAnalysisWorker.analyze
(a file whose application projection was refused is offered 0 while sources
remain), so the visible symptom follows from the estimate, not from a separate
accounting error.

Related: #2097 (open PR, recalibrates the separate transfer-retention factor),
#2092, #1387, #3671 (same "required bytes far below the budget" pattern on
6.4.0), #3550.

Reproduction

Synthetic fixture only (generator gen-vite-like.mjs below, deterministic):

node gen-vite-like.mjs ./fx/vite87 87 456      # 88 files, 2,809,718 bytes
rea analyze-javascript-application ./fx/vite87/assets \
  --artifact-format directory --max-heap-mb 4096 --format json > out.json

Count the two limitation messages above in out.json .limitations (78 each),
and the files with a semantic function node (10).

Expected behavior

The admission estimate should stay a bounded multiple of what the parent
retains, so that a build whose retained results fit comfortably in the parent
heap is not refused after a few files.

Impact and scope

Every multi-file bundle with many storage, request or import findings per
file. In the original (non-synthetic) case on 6.4.0, 67 of 87 chunks were
refused with available budgets of 0, 10,151 and 23,463 bytes. #7736 is the same root cause reaching the static-fact stage.

REA version

rea-agents 6.8.0 (npm, Node 26.12.1) and upstream main at 6a226bd09 (source, Node 24.22.1). First observed on 6.4.0 (MCP).

Runtime and platform

Windows 11 Pro (10.0.26300). 6.8.0: global npm install on Node.js 26.12.1. main: source checkout on Node.js 24.22.1.

Interface

CLI (rea analyze-javascript-application); first seen through MCP analyze_javascript_application (detail summary, integrity_policy: record-and-continue, max_heap_mb: 4096).

Typed outcome or recovery expected

A fix is proposed in a PR (fix/application-projection-estimate): count the
per-record allowance once and keep the 8x expansion for checkpoint and
repeated bytes.

gen-vite-like.mjs (the synthetic fixture generator)

// Generates a synthetic Vite-like build output. No real application code.
// Usage: node gen-vite-like.mjs <outDir> [chunkCount] [bigChunkKB] [first|last]
// Writes <outDir>/index.html and <outDir>/assets/*.js. Each chunk has an
// absolute-base __vite__mapDeps table, real dynamic imports, localStorage
// get/set calls and api.rpc("proc_N_M") calls. Deterministic output.
import { mkdirSync, writeFileSync, rmSync } from "node:fs";
import { join } from "node:path";

const outDir = process.argv[2];
const chunkCount = Number(process.argv[3] ?? 87);
const bigKB = Number(process.argv[4] ?? 456);
if (!outDir) throw new Error("outDir required");
rmSync(outDir, { recursive: true, force: true });
const assets = join(outDir, "assets");
mkdirSync(assets, { recursive: true });

let seed = 7;
const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
const names = Array.from({ length: chunkCount }, (_, i) => `chunk${String(i).padStart(2, "0")}-${(i * 2654435761 >>> 0).toString(36).slice(0, 6)}.js`);
const entry = "index-a1b2c3.js";

function body(i, targetBytes) {
  const parts = [];
  let n = 0;
  let size = 0;
  while (size < targetBytes) {
    const f = `f${i}_${n}`;
    const key = `demo.pref.${i}.${n % 5}`;
    const fn = `export function ${f}(a,b){const c=a*${n}+b;` +
      `if(c>${Math.floor(rand() * 1000)}){localStorage.setItem("${key}",String(c));}` +
      `const d=localStorage.getItem("${key}");` +
      `return api.rpc("proc_${i}_${n % 7}",{v:c,d}).then(r=>r.data);}\n`;
    parts.push(fn);
    size += fn.length;
    n++;
  }
  return parts.join("");
}

const total = [];
const deps = names.map((n) => `assets/${n}`);
for (let i = 0; i < chunkCount; i++) {
  const bigIndex = process.argv[5] === "last" ? chunkCount - 1 : 0;
  const target = i === bigIndex ? bigKB * 1024 : 20 * 1024 + Math.floor(rand() * 12 * 1024);
  const next = (i + 1) % chunkCount;
  const other = (i + 7) % chunkCount;
  const head =
    `import{api}from"./${entry}";\n` +
    `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify([deps[next], deps[other]])})))=>i.map(i=>d[i]);\n` +
    `export const load${i}=()=>import("./${names[next]}");\n` +
    `export const pre${i}=()=>api.preload(()=>import("./${names[other]}"),__vite__mapDeps([0,1]));\n`;
  const text = head + body(i, target);
  writeFileSync(join(assets, names[i]), text);
  total.push(text.length);
}
const entryText =
  `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify(deps)})))=>i.map(i=>d[i]);\n` +
  `export const api={rpc(name,args){return fetch("/rpc/"+name,{method:"POST",body:JSON.stringify(args)}).then(r=>r.json());},preload(f,d){return f();}};\n` +
  names.map((n, i) => `export const route${i}=()=>api.preload(()=>import("./${n}"),__vite__mapDeps([${i}]));\n`).join("") +
  `const saved=localStorage.getItem("demo.session.token");if(saved){route0();}\n`;
writeFileSync(join(assets, entry), entryText);
writeFileSync(join(outDir, "index.html"), `<!doctype html><script type="module" src="/assets/${entry}"></script>`);
const sum = total.reduce((a, b) => a + b, 0) + entryText.length;
console.log(JSON.stringify({ files: chunkCount + 1, bytes: sum, biggest: Math.max(...total) }));

Originally created by @jgt87 on GitHub (Oct 28, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7735 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. Reproduced on rea-agents 6.8.0 (released) and on main 6a226bd09. | Same fixture (88 files, 456 KB chunk first, `--max-heap-mb 4096`) | 6.8.0 (npm, Node 26.12.1) | main 6a226bd09 (source, Node 24.22.1) | | --- | ---: | ---: | | `parent_result_budget_bytes` | 1,995,849,968 | 2,082,939,876 | | Files with refused application projection | 77 of 88 | 78 of 88 | | Files offered `semantic_result_budget_bytes: 0` | 77 | 78 | | Lowest / highest `parent_available_result_budget_bytes` reported | 0 / 37,098,550 | 793,106 / 23,723,138 | | Files with a semantic graph | 8 | 10 | The estimate formula below is the same in the 6.8.0 package and on main. The per-file charge table further down comes from a logging-only instrumented build of main baec375c3; the commits up to 6a226bd09 do not touch `src/javascript`, `src/domain/javascript` or `src/application/javascript`. ## Area JavaScript application analysis (`analyze_javascript_application`), parent result admission. ## Observed behavior On a synthetic Vite-style build of 88 files (2.8 MB, one 456 KB chunk), only the first 10 files are projected into the application graph and get a semantic graph. For the other 78 files REA reports: ```text Resource constraint during analyze_javascript_application: Expanded application relationships for chunkNN.js exceed the parent's result budget; complete static facts were retained on the file node chunkNN.js: Resource constraint during analyze_javascript_application: The parent reserved its remaining result capacity for static observations; semantic analysis of chunkNN.js was not started ``` with `reported_limits` such as: ```json {"parent_result_budget_bytes":2083223972,"parent_available_result_budget_bytes":6432706,"application_projection_required_bytes":43126016} {"worker_old_space_mb":4096,"worker_heap_limit_bytes":4320133120,"worker_timeout_ms":300000,"semantic_result_budget_bytes":0} ``` So 2.08 GB of parent budget is reported, but only 3 to 23 MB is available after 10 files, and the next 78 files are offered a semantic budget of 0 bytes. At that point the parent process heap holds about 300 MiB (`process.memoryUsage().heapUsed`), and its peak RSS for the whole run is 1,199 MiB. The parent V8 heap limit is about 4.2 GB. ## Analysis I instrumented the built CLI locally (logging only) to print each admission charge. The largest single charge is `estimateJavaScriptStaticApplicationProjection` (`src/domain/javascript/javascriptApplicationProjectionResources.ts`): ```ts const metadataBytes = 2048 + Buffer.byteLength(sourcePath) * 4; return (checkpointBytes * 4 + repeatedBytes + records * metadataBytes) * 8; ``` `records` is 8 per finding, so each finding is charged about 8 × 2,100 × 8 = 134 KB. For the 456 KB chunk (855,843-byte static checkpoint) the estimate is 645,656,672 bytes. Analyzed alone, the same chunk produces an application graph of 25,274,686 bytes of JSON (2,320 nodes, 6,923 edges), so the charge is about 25x the serialized relationships it admits. The 2 KB per-record allowance is already about 4.5x the measured ~0.46 KB per record; the outer `* 8` expands it a second time. Charges for the first files (bytes): | File | Static checkpoint | Projection charge | Transfer charge (module + graph) | Total charged after file | | --- | ---: | ---: | ---: | ---: | | chunk00 (456 KB) | 855,843 | 645,656,672 | 27,503,464 + 296,630,464 | 974,925,658 | | chunk01 (26 KB) | 46,944 | 35,764,480 | 1,477,912 + 74,388,400 | 1,086,838,114 | | chunk08 (31 KB) | 61,704 | 47,070,720 | 1,948,952 + 98,232,496 | 1,861,349,050 | | chunk09 onward | | refused (avail 24 MB) | not started | ~2,009,000,000 | The 0-byte semantic budget is deliberate in `JavaScriptAnalysisWorker.analyze` (a file whose application projection was refused is offered 0 while sources remain), so the visible symptom follows from the estimate, not from a separate accounting error. Related: #2097 (open PR, recalibrates the separate transfer-retention factor), #2092, #1387, #3671 (same "required bytes far below the budget" pattern on 6.4.0), #3550. ## Reproduction Synthetic fixture only (generator `gen-vite-like.mjs` below, deterministic): ```sh node gen-vite-like.mjs ./fx/vite87 87 456 # 88 files, 2,809,718 bytes rea analyze-javascript-application ./fx/vite87/assets \ --artifact-format directory --max-heap-mb 4096 --format json > out.json ``` Count the two limitation messages above in `out.json` `.limitations` (78 each), and the files with a semantic `function` node (10). ## Expected behavior The admission estimate should stay a bounded multiple of what the parent retains, so that a build whose retained results fit comfortably in the parent heap is not refused after a few files. ## Impact and scope Every multi-file bundle with many storage, request or import findings per file. In the original (non-synthetic) case on 6.4.0, 67 of 87 chunks were refused with available budgets of 0, 10,151 and 23,463 bytes. #7736 is the same root cause reaching the static-fact stage. ## REA version rea-agents 6.8.0 (npm, Node 26.12.1) and upstream `main` at 6a226bd09 (source, Node 24.22.1). First observed on 6.4.0 (MCP). ## Runtime and platform Windows 11 Pro (10.0.26300). 6.8.0: global npm install on Node.js 26.12.1. main: source checkout on Node.js 24.22.1. ## Interface CLI (`rea analyze-javascript-application`); first seen through MCP `analyze_javascript_application` (detail summary, `integrity_policy: record-and-continue`, `max_heap_mb: 4096`). ## Typed outcome or recovery expected A fix is proposed in a PR (fix/application-projection-estimate): count the per-record allowance once and keep the 8x expansion for checkpoint and repeated bytes. <details><summary><code>gen-vite-like.mjs</code> (the synthetic fixture generator)</summary> ```js // Generates a synthetic Vite-like build output. No real application code. // Usage: node gen-vite-like.mjs <outDir> [chunkCount] [bigChunkKB] [first|last] // Writes <outDir>/index.html and <outDir>/assets/*.js. Each chunk has an // absolute-base __vite__mapDeps table, real dynamic imports, localStorage // get/set calls and api.rpc("proc_N_M") calls. Deterministic output. import { mkdirSync, writeFileSync, rmSync } from "node:fs"; import { join } from "node:path"; const outDir = process.argv[2]; const chunkCount = Number(process.argv[3] ?? 87); const bigKB = Number(process.argv[4] ?? 456); if (!outDir) throw new Error("outDir required"); rmSync(outDir, { recursive: true, force: true }); const assets = join(outDir, "assets"); mkdirSync(assets, { recursive: true }); let seed = 7; const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648); const names = Array.from({ length: chunkCount }, (_, i) => `chunk${String(i).padStart(2, "0")}-${(i * 2654435761 >>> 0).toString(36).slice(0, 6)}.js`); const entry = "index-a1b2c3.js"; function body(i, targetBytes) { const parts = []; let n = 0; let size = 0; while (size < targetBytes) { const f = `f${i}_${n}`; const key = `demo.pref.${i}.${n % 5}`; const fn = `export function ${f}(a,b){const c=a*${n}+b;` + `if(c>${Math.floor(rand() * 1000)}){localStorage.setItem("${key}",String(c));}` + `const d=localStorage.getItem("${key}");` + `return api.rpc("proc_${i}_${n % 7}",{v:c,d}).then(r=>r.data);}\n`; parts.push(fn); size += fn.length; n++; } return parts.join(""); } const total = []; const deps = names.map((n) => `assets/${n}`); for (let i = 0; i < chunkCount; i++) { const bigIndex = process.argv[5] === "last" ? chunkCount - 1 : 0; const target = i === bigIndex ? bigKB * 1024 : 20 * 1024 + Math.floor(rand() * 12 * 1024); const next = (i + 1) % chunkCount; const other = (i + 7) % chunkCount; const head = `import{api}from"./${entry}";\n` + `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify([deps[next], deps[other]])})))=>i.map(i=>d[i]);\n` + `export const load${i}=()=>import("./${names[next]}");\n` + `export const pre${i}=()=>api.preload(()=>import("./${names[other]}"),__vite__mapDeps([0,1]));\n`; const text = head + body(i, target); writeFileSync(join(assets, names[i]), text); total.push(text.length); } const entryText = `const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=${JSON.stringify(deps)})))=>i.map(i=>d[i]);\n` + `export const api={rpc(name,args){return fetch("/rpc/"+name,{method:"POST",body:JSON.stringify(args)}).then(r=>r.json());},preload(f,d){return f();}};\n` + names.map((n, i) => `export const route${i}=()=>api.preload(()=>import("./${n}"),__vite__mapDeps([${i}]));\n`).join("") + `const saved=localStorage.getItem("demo.session.token");if(saved){route0();}\n`; writeFileSync(join(assets, entry), entryText); writeFileSync(join(outDir, "index.html"), `<!doctype html><script type="module" src="/assets/${entry}"></script>`); const sum = total.reduce((a, b) => a + b, 0) + entryText.length; console.log(JSON.stringify({ files: chunkCount + 1, bytes: sum, biggest: Math.max(...total) })); ``` </details>

bsakaguchi closed this issue

2026-10-31 00:51:35 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471