morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7787] [Bug] analyze-javascript-application does not write --evidence_output when a source reaches its deadline; partial Evidence goes inline to stdout #4642

New issue

Open

opened 2026-10-31 00:51:43 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:51:43 +00:00

Owner

Originally created by @jgt87 on GitHub (Oct 28, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7787

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

rea analyze-javascript-application --detail summary --evidence_output FILE does not write FILE when one source reaches the per-source deadline (analysis_timeout_ms). The command exits 1 with provider_timeout, and stdout carries the complete partial Evidence inline, not the compact summary that was requested.

The facts of the sources that finished are kept, but only inline on stdout in details.partial_observation. In this run that is 6 storage nodes, 6 endpoint nodes and 217 semantic nodes, with graph and semantic coverage partial, plus the limitation vendor.js: Analysis operation timed out after 5000ms: analyze_javascript_application. No file at the requested path holds them.

On a synthetic 7-file build (six small modules and one 1 MB string-array-style module):

	rea-agents 6.8.0 (npm, Node.js 26.12.1)	main `7aa943c14` (source, Node.js 24.22.1)
`--analysis-timeout-ms 5000`: exit code	1 (`provider_timeout`)	1 (`provider_timeout`)
`--analysis-timeout-ms 5000`: stdout	510,839 bytes, complete partial Evidence	511,150 bytes, complete partial Evidence
`--analysis-timeout-ms 5000`: `--evidence_output` file	not written	not written
`--analysis-timeout-ms 5000`: `parsed_javascript_files` in the partial result	7 of 7	6 of 7
Deadline raised (6.8.0: 60000; main: 300000, the default): exit code	0	0
Deadline raised: stdout	15,068 bytes, summary	15,072 bytes, summary
Deadline raised: `--evidence_output` file	77,836,431 bytes	77,836,432 bytes

parsed_javascript_files differs because the deadline stopped the slow source in a different phase on each run.

In the original (non-synthetic) case, a 2.8 MB build with one large obfuscated vendor chunk reached the default 300 s deadline. stdout carried more than 64 MB of partial Evidence, and the requested Evidence file was not written.

Reproduction

The generator and the commands are below. The generator writes six small modules and vendor.js. On this machine the whole run takes about 33 s with the default deadline; a 5 s deadline stops vendor.js.

node gen-timeout.mjs ./fx/timeout 6 6000      # 7 files; vendor.js is 1,036,432 bytes
rea analyze-javascript-application ./fx/timeout --artifact-format directory \
  --detail summary --evidence_output "$PWD/ev.json" --analysis-timeout-ms 5000 \
  --format json > out.json
echo $?                                       # 1
ls ev.json                                    # No such file or directory
wc -c out.json                                # about 511 KB: error with details.partial_observation

With --analysis-timeout-ms 60000 (or the default), the same command exits 0, writes ev.json (about 78 MB) and prints a 15 KB summary.

gen-timeout.mjs (synthetic, deterministic)

// Synthetic build: small modules plus one large string-array-style block that
// takes the analyzer much longer than the others. No real application code.
// Usage: node gen-timeout.mjs <outDir> [smallCount] [slowFunctions]
import { mkdirSync, rmSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const [outDir, smallArg = "6", slowArg = "6000"] = process.argv.slice(2);
if (!outDir) throw new Error("outDir required");
const smallCount = Number(smallArg);
const slowFunctions = Number(slowArg);
rmSync(outDir, { recursive: true, force: true });
mkdirSync(outDir, { recursive: true });

for (let i = 0; i < smallCount; i++)
  writeFileSync(
    join(outDir, `small${i}.js`),
    `export function load${i}(){return fetch("/api/item/${i}").then(r=>r.json());}\n` +
      `export function save${i}(v){localStorage.setItem("demo.key.${i}",JSON.stringify(v));}\n`,
  );

const words = Array.from({ length: 4000 }, (_, i) => `w${i.toString(36)}`);
const table = `const _t=${JSON.stringify(words)};function _s(i){return _t[(i*7+3)%_t.length];}\n`;
const body = Array.from({ length: slowFunctions }, (_, i) => {
  const k = (n) => `_s(${(i * 31 + n) % 4000})`;
  return `export function f${i}(a,b){const o={[${k(1)}]:a,[${k(2)}]:b};` +
    `if(o[${k(3)}]>o[${k(4)}]){return o[${k(5)}]+${k(6)};}` +
    `return window[${k(7)}](o[${k(8)}],o[${k(9)}]);}\n`;
}).join("");
writeFileSync(join(outDir, "vendor.js"), table + body);
console.log(JSON.stringify({ files: smallCount + 1, vendorBytes: (table + body).length }));

Expected behavior

When the caller passes --evidence_output, a failure that has partial Evidence publishes that partial Evidence to the requested file, in the same atomic way as a success. The failure output then carries the compact view and the publication receipt (path and bytes), as the success path does. Without --evidence_output, the current documented behavior can stay: "CLI failure output contains the complete partial Evidence" (docs/javascript-artifact-reconstruction.md, "Recover partial analysis").

docs/cli.md says that --detail summary --evidence_output FILE "exports the full result to a new file and returns a compact overview". The timeout's own remediation says "Reuse accessible partial_observation Evidence to avoid recomputation". On the CLI with --evidence_output, the only copy is inline on stdout, so it is not a file the caller can reuse.

Impact and scope

Large builds, where --detail summary --evidence_output matters most, are also the ones most likely to reach a per-source deadline.
A caller that reads only the requested file finds nothing, and the completed facts reach the caller only as a large error document on stdout (more than 64 MB in the original case).
Reproduced on both versions above.
Not checked: other failure codes that carry partial_observation for this command, for example the resource_constraint in #7736, probably take the same path; MCP delivery.

REA version

rea-agents 6.8.0 (npm) and upstream main at 7aa943c14.

Runtime and platform

Windows 11 Pro (10.0.26300).
6.8.0: global npm install, Node.js 26.12.1.
main: source checkout, Node.js 24.22.1.

Target and artifact context

A synthetic directory of seven JavaScript files (generator above), selected with --artifact-format directory. No real application code.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The start of stdout on main 7aa943c14:

{"error":"JavaScript application analysis failed","code":"provider_timeout","category":"timeout","message":"JavaScript source analysis reached its 5000ms per-source server/provider deadline (analysis_timeout_ms), not the MCP client deadline.","retryable":true,"remediation":{"action":"Reuse accessible partial_observation Evidence to avoid recomputation; ..."}, "details":{"operation":"analyze_javascript_application","timeout_ms":5000,"limit":{...},"captured_output":{...},"partial_observation":{"evidence_id":"ev_e6bcbde33a915a90e...", ...}}}

details has no evidence_bundle, published_path or bytes entry.

Typed outcome or recovery expected

Keep provider_timeout, exit 1 and the partial_observation identity. Publish the partial Evidence to the --evidence_output path and report the publication receipt in details, and keep stdout compact when --detail summary is set.

Analysis or suspected cause

Source reading, not verified in a debugger: javascriptApplicationAnalysis.ts passes the finalized result to publishCliAnalysisEvidence (src/cli/analysisEvidenceDelivery.ts), which returns a failed result unchanged (if (!result.ok) return result.error;) before any evidenceOutput handling. So --evidence_output publication and the summary view apply only to success, and the failure prints the error document with its inline partial_observation. The bundle code already records partial Evidence with an evidence_id (see #4276 for raw, non-Evidence observations), so the partial result here has an ID and could be published.

Additional context

Related: #7736, the same command failing with resource_constraint and a partial_observation; #5808, the audit of recovery controls for provider timeouts and limits.
docs/cli.md states that cancellation before publication leaves no new file. It does not state what happens to --evidence_output when the analysis fails with partial Evidence.

Originally created by @jgt87 on GitHub (Oct 28, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7787 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Evidence, artifacts, or reconstruction ## Observed behavior `rea analyze-javascript-application --detail summary --evidence_output FILE` does not write `FILE` when one source reaches the per-source deadline (`analysis_timeout_ms`). The command exits 1 with `provider_timeout`, and stdout carries the complete partial Evidence inline, not the compact summary that was requested. The facts of the sources that finished are kept, but only inline on stdout in `details.partial_observation`. In this run that is 6 storage nodes, 6 endpoint nodes and 217 semantic nodes, with graph and semantic coverage `partial`, plus the limitation `vendor.js: Analysis operation timed out after 5000ms: analyze_javascript_application`. No file at the requested path holds them. On a synthetic 7-file build (six small modules and one 1 MB string-array-style module): | | rea-agents 6.8.0 (npm, Node.js 26.12.1) | main 7aa943c14 (source, Node.js 24.22.1) | | --- | --- | --- | | `--analysis-timeout-ms 5000`: exit code | 1 (`provider_timeout`) | 1 (`provider_timeout`) | | `--analysis-timeout-ms 5000`: stdout | 510,839 bytes, complete partial Evidence | 511,150 bytes, complete partial Evidence | | `--analysis-timeout-ms 5000`: `--evidence_output` file | not written | not written | | `--analysis-timeout-ms 5000`: `parsed_javascript_files` in the partial result | 7 of 7 | 6 of 7 | | Deadline raised (6.8.0: 60000; main: 300000, the default): exit code | 0 | 0 | | Deadline raised: stdout | 15,068 bytes, summary | 15,072 bytes, summary | | Deadline raised: `--evidence_output` file | 77,836,431 bytes | 77,836,432 bytes | `parsed_javascript_files` differs because the deadline stopped the slow source in a different phase on each run. In the original (non-synthetic) case, a 2.8 MB build with one large obfuscated vendor chunk reached the default 300 s deadline. stdout carried more than 64 MB of partial Evidence, and the requested Evidence file was not written. ## Reproduction The generator and the commands are below. The generator writes six small modules and `vendor.js`. On this machine the whole run takes about 33 s with the default deadline; a 5 s deadline stops `vendor.js`. ```sh node gen-timeout.mjs ./fx/timeout 6 6000 # 7 files; vendor.js is 1,036,432 bytes rea analyze-javascript-application ./fx/timeout --artifact-format directory \ --detail summary --evidence_output "$PWD/ev.json" --analysis-timeout-ms 5000 \ --format json > out.json echo $? # 1 ls ev.json # No such file or directory wc -c out.json # about 511 KB: error with details.partial_observation ``` With `--analysis-timeout-ms 60000` (or the default), the same command exits 0, writes `ev.json` (about 78 MB) and prints a 15 KB summary. <details> <summary><code>gen-timeout.mjs</code> (synthetic, deterministic)</summary> ```js // Synthetic build: small modules plus one large string-array-style block that // takes the analyzer much longer than the others. No real application code. // Usage: node gen-timeout.mjs <outDir> [smallCount] [slowFunctions] import { mkdirSync, rmSync, writeFileSync } from "node:fs"; import { join } from "node:path"; const [outDir, smallArg = "6", slowArg = "6000"] = process.argv.slice(2); if (!outDir) throw new Error("outDir required"); const smallCount = Number(smallArg); const slowFunctions = Number(slowArg); rmSync(outDir, { recursive: true, force: true }); mkdirSync(outDir, { recursive: true }); for (let i = 0; i < smallCount; i++) writeFileSync( join(outDir, `small${i}.js`), `export function load${i}(){return fetch("/api/item/${i}").then(r=>r.json());}\n` + `export function save${i}(v){localStorage.setItem("demo.key.${i}",JSON.stringify(v));}\n`, ); const words = Array.from({ length: 4000 }, (_, i) => `w${i.toString(36)}`); const table = `const _t=${JSON.stringify(words)};function _s(i){return _t[(i*7+3)%_t.length];}\n`; const body = Array.from({ length: slowFunctions }, (_, i) => { const k = (n) => `_s(${(i * 31 + n) % 4000})`; return `export function f${i}(a,b){const o={[${k(1)}]:a,[${k(2)}]:b};` + `if(o[${k(3)}]>o[${k(4)}]){return o[${k(5)}]+${k(6)};}` + `return window[${k(7)}](o[${k(8)}],o[${k(9)}]);}\n`; }).join(""); writeFileSync(join(outDir, "vendor.js"), table + body); console.log(JSON.stringify({ files: smallCount + 1, vendorBytes: (table + body).length })); ``` </details> ## Expected behavior When the caller passes `--evidence_output`, a failure that has partial Evidence publishes that partial Evidence to the requested file, in the same atomic way as a success. The failure output then carries the compact view and the publication receipt (path and bytes), as the success path does. Without `--evidence_output`, the current documented behavior can stay: "CLI failure output contains the complete partial Evidence" (`docs/javascript-artifact-reconstruction.md`, "Recover partial analysis"). `docs/cli.md` says that `--detail summary --evidence_output FILE` "exports the full result to a new file and returns a compact overview". The timeout's own remediation says "Reuse accessible partial_observation Evidence to avoid recomputation". On the CLI with `--evidence_output`, the only copy is inline on stdout, so it is not a file the caller can reuse. ## Impact and scope - Large builds, where `--detail summary --evidence_output` matters most, are also the ones most likely to reach a per-source deadline. - A caller that reads only the requested file finds nothing, and the completed facts reach the caller only as a large error document on stdout (more than 64 MB in the original case). - Reproduced on both versions above. - Not checked: other failure codes that carry `partial_observation` for this command, for example the `resource_constraint` in #7736, probably take the same path; MCP delivery. ## REA version rea-agents 6.8.0 (npm) and upstream `main` at 7aa943c14. ## Runtime and platform Windows 11 Pro (10.0.26300). 6.8.0: global npm install, Node.js 26.12.1. main: source checkout, Node.js 24.22.1. ## Target and artifact context A synthetic directory of seven JavaScript files (generator above), selected with `--artifact-format directory`. No real application code. ## Interface CLI ## Earliest failure stage Transport or result projection ## Relevant output or evidence The start of stdout on main 7aa943c14: ```json {"error":"JavaScript application analysis failed","code":"provider_timeout","category":"timeout","message":"JavaScript source analysis reached its 5000ms per-source server/provider deadline (analysis_timeout_ms), not the MCP client deadline.","retryable":true,"remediation":{"action":"Reuse accessible partial_observation Evidence to avoid recomputation; ..."}, "details":{"operation":"analyze_javascript_application","timeout_ms":5000,"limit":{...},"captured_output":{...},"partial_observation":{"evidence_id":"ev_e6bcbde33a915a90e...", ...}}} ``` `details` has no `evidence_bundle`, `published_path` or `bytes` entry. ## Typed outcome or recovery expected Keep `provider_timeout`, exit 1 and the `partial_observation` identity. Publish the partial Evidence to the `--evidence_output` path and report the publication receipt in `details`, and keep stdout compact when `--detail summary` is set. ## Analysis or suspected cause Source reading, not verified in a debugger: `javascriptApplicationAnalysis.ts` passes the finalized result to `publishCliAnalysisEvidence` (`src/cli/analysisEvidenceDelivery.ts`), which returns a failed result unchanged (`if (!result.ok) return result.error;`) before any `evidenceOutput` handling. So `--evidence_output` publication and the summary view apply only to success, and the failure prints the error document with its inline `partial_observation`. The bundle code already records partial Evidence with an `evidence_id` (see #4276 for raw, non-Evidence observations), so the partial result here has an ID and could be published. ## Additional context - Related: #7736, the same command failing with `resource_constraint` and a `partial_observation`; #5808, the audit of recovery controls for provider timeouts and limits. - `docs/cli.md` states that cancellation before publication leaves no new file. It does not state what happens to `--evidence_output` when the analysis fails with partial Evidence.

bsakaguchi referenced this issue

2026-10-31 01:13:40 +00:00

[PR #4642] [MERGED] fix(setup): offer Hopper installation only where setup can install it #6336

bsakaguchi referenced this issue

2026-11-01 21:46:57 +00:00

[PR #6368] [MERGED] docs(limits): describe how to design admission recovery #7314

bsakaguchi referenced this issue

2026-11-01 21:47:00 +00:00

[PR #6375] [MERGED] test: repair remaining bulk-merge regression fixtures #7319