[GH-ISSUE #4629] [Bug] JavaScript recovery reports input-size exhaustion as invalid_value without structured limits #2408

New issue

Closed

opened 2026-10-17 21:04:15 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:04:15 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4629

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

recover-javascript-sources refuses a stable 67108865-byte UTF-8 JavaScript file with invalid_request / invalid_input, issue reason invalid_value, and no structured byte allowance. The response advertises retryable: true and generic argument-correction guidance. The 67108864-byte limit survives only in message text, which also calls this input-size refusal an invalid analysis output. A malformed UTF-8 input receives the same issue reason.

This reproduced through the real public CLI in 3/3 attempts on each tested build, on Linux x64 with the documented real Wakaru binary configured. Both builds successfully recovered a small source with the same engine. This report concerns the typed refusal; the size limit remains enforced.

Reproduction

Install the release below in a disposable directory, or install the source checkout's locked dependencies and run npm run build:cached. On Linux x64, install Wakaru with npm install wakaru@1.14.0 in a separate disposable directory. Use its absolute node_modules/@wakaru/cli-linux-x64/wakaru path, rather than the Node shim. Ensure /usr/bin/prlimit is available.
Save this as reproduce.mjs in a disposable directory. Run node reproduce.mjs /absolute/path/to/rea/scripts/rea.mjs /absolute/path/to/wakaru against each build. These are the checks actually run; every CLI subprocess has a fresh HOME and runtime directory and an allowlisted environment. The script removes its fixtures and successful recovery output.

import assert from 'node:assert/strict';
import { mkdtemp, mkdir, writeFile, rm, readdir, stat } from 'node:fs/promises';
import { dirname, join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
import { spawnSync } from 'node:child_process';
const entry = resolve(process.argv[2]);
const engine = resolve(process.argv[3]);
const packageRoot = dirname(dirname(entry));
const { snapshotRecoveryInputs } = await import(pathToFileURL(join(packageRoot, 'dist/javascript/recovery/RecoveryFiles.js')));
const root = await mkdtemp(join(process.cwd(), 'recovery-size-'));
const maximum = 64 * 1024 * 1024;
const clean = value => JSON.stringify(value).replaceAll(root, '<fixture>');
try {
  await writeFile(join(root, 'over.js'), Buffer.alloc(maximum + 1, 32));
  await writeFile(join(root, 'exact.js'), Buffer.alloc(maximum, 32));
  await writeFile(join(root, 'invalid.js'), Buffer.from([255]));
  await writeFile(join(root, 'small.js'), 'console.log("fixture");\n');
  const har = Buffer.from('{"log":{"version":"1.2","creator":{"name":"fixture","version":"1"},"entries":[]}}');
  await writeFile(join(root, 'over.har'), Buffer.concat([har, Buffer.alloc(33554433 - har.length, 32)]));
  let sequence = 0;
  for (const name of ['over.js', 'over.js', 'over.js', 'invalid.js', 'small.js', 'over.har']) {
    const callRoot = join(root, String(sequence++));
    await mkdir(join(callRoot, 'home'), { recursive: true });
    await mkdir(join(callRoot, 'runtime'));
    const output = join(callRoot, 'output');
    const env = { PATH: process.env.PATH, HOME: join(callRoot, 'home'), TMPDIR: join(callRoot, 'runtime'), LANG: 'C', REA_LOG_LEVEL: 'silent', REA_WAKARU_COMMAND: engine };
    const args = name.endsWith('.har')
      ? ['inspect-web-network-capture', join(root, name), 'har', '--json']
      : ['recover-javascript-sources', join(root, name), output, '--json'];
    const call = spawnSync(process.execPath, [entry, ...args], { env, encoding: 'utf8', timeout: 180000, maxBuffer: 10 * 1024 * 1024 });
    assert.equal(call.error, undefined);
    assert.equal(call.stderr, '');
    const reply = JSON.parse(call.stdout);
    const residuals = await readdir(join(callRoot, 'runtime'));
    const published = await stat(output).then(() => true, () => false);
    console.log(name, clean({ exit: call.status, code: reply.code, category: reply.category, retryable: reply.retryable, remediation: reply.remediation, details: reply.details, status: reply.normalized_result?.status, residuals, published }));
    assert.equal(call.status, name === 'small.js' ? 0 : 1);
    assert.deepEqual(residuals, []);
    assert.equal(published, name === 'small.js');
    if (name === 'over.js') {
      assert.equal(reply.code, 'invalid_request');
      assert.equal(reply.details.issues[0].reason, 'invalid_value');
      assert.equal(reply.details.issues[0].expected, undefined);
      assert.equal(reply.details.issues[0].maximum, undefined);
      assert.equal(reply.details.reported_limits, undefined);
      assert.match(reply.details.issues[0].message, /67108864/);
    }
    if (name === 'invalid.js') assert.equal(reply.details.issues[0].reason, 'invalid_value');
    if (name === 'over.har') {
      assert.equal(reply.details.issues[0].reason, 'out_of_range');
      assert.equal(reply.details.issues[0].expected.maximum_capture_bytes, 33554432);
    }
  }
  const exact = await snapshotRecoveryInputs([join(root, 'exact.js')], join(root, 'snapshot'));
  assert.equal(exact[0].bytes, maximum);
  console.log('exact.js snapshot accepted_bytes=' + exact[0].bytes);
} finally {
  await rm(root, { recursive: true, force: true });
}

Observe the three oversized recovery calls return invalid_value without expected, maximum, or reported_limits. The malformed UTF-8 call also returns invalid_value; small-source recovery succeeds. The sibling HAR CLI reports out_of_range with a structured allowance. The exact-limit recovery snapshot succeeds through the production snapshot function; that last control is an internal boundary check, not an exact-limit public recovery run.

Expected behavior

Keep enforcing the total selected-input allowance while exposing a machine-readable distinction between input-size exhaustion and malformed source, with the configured byte budget in structured fields. The recovery guide promises explicit failure and rollback when the limit is exceeded, without mandating one exact code. The shared error algebra distinguishes measured resource constraints and bounded admission refusals from malformed input and supports structured file-size limits. The sibling capture workflow uses out_of_range plus structured expected-limit metadata. Either established representation can preserve the distinction missing here.

Impact and scope

Clients handling limits through typed fields must parse recovery's message to distinguish oversized valid source from malformed input. No limit bypass, source corruption, or downstream client incident was observed. The single stable input CLI case reproduced consistently in this bounded sample; natural workload frequency is unknown. MCP, multi-input exhaustion, and growth during reading were not exercised.

Impact level: P2
Assessment status: supported
Impact basis: The measured size refusal loses its structured classification and byte allowance, preventing consistent typed limit handling without parsing message text.
Workaround status: available
Workaround basis: Callers can identify the refusal from the retained size-limit message and documented allowance. The budget is still enforced. Truncating required source bytes would not preserve the original analysis and is not proposed as a workaround.

REA version

Published npm rea-agents 6.5.0 and fresh main source commit 8451d283920e86936576fdf67615807d1b4ca4a9. Both public entrypoints report 6.5.0; source dependencies and build used the pinned toolchain.

Runtime and platform

Node.js v24.18.0; npm 11.16.0; Debian GNU/Linux 12 x86_64 in a Linux x64 Docker container with network disabled during the checks; Wakaru 1.14.0, SHA-256 664fa06782a61eda7ed15885134b52525fd565dd77607e642062ceb6cd927c38, matching the recovery guide. The container ran on a Darwin arm64 host.

Target and artifact context

Self-generated JavaScript whitespace files of 67108865 and 67108864 ASCII spaces; a one-byte invalid UTF-8 file (ff); and a 24-byte small source. The HAR comparison is valid empty HAR JSON padded with trailing JSON whitespace to 33554433 bytes. The selected application code is never executed by recovery.

Interface

CLI

Earliest failure stage

Setup or provider admission

Relevant output or evidence

Expected source: docs/javascript-recovery.md:132-137; src/domain/analysisErrorCore.ts:23-54,80-94; src/domain/analysisErrorProjection.ts:258-263,287-297; and src/browser/history/CaptureFailures.ts:124-138 in the tested main tree.
Failure source: The public CLI results below; source breadcrumbs are src/javascript/recovery/RecoveryFiles.ts:55-60,139-148,287-294 and src/javascript/recovery/WakaruProvider.ts:109-120,236-242.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Meaningful output from both builds, with the generated fixture prefix replaced by <fixture>:

recover-javascript-sources over.js <absent-output> --json, 3/3 each:
  exit=1; code=invalid_request; category=invalid_input; retryable=true
  remediation.action="Correct the listed arguments and retry."
  issue.path=["path"]; issue.reason=invalid_value
  issue.message="Cannot snapshot UTF-8 JavaScript input <fixture>/over.js: Invalid analysis output for recover_javascript_sources: Expected a regular file no larger than 67108864 bytes: <fixture>/over.js"
  no issue.expected, issue.maximum, details.resource or details.reported_limits
recover-javascript-sources invalid.js <absent-output> --json:
  exit=1; issue.reason=invalid_value
  issue.message ends with "The encoded data was not valid for encoding utf-8"
recover-javascript-sources small.js <absent-output> --json:
  exit=0; normalized_result.status=complete
inspect-web-network-capture over.har har --json:
  exit=1; issue.reason=out_of_range
  issue.expected={"maximum_capture_bytes":33554432}
Production snapshotRecoveryInputs, exact.js:
  accepted_bytes=67108864

All exercised CLI stderr streams were empty. Every per-call runtime directory was empty afterward; failed calls left the requested output absent, while small-source recovery created its output. The exact-limit snapshot and each non-oversized control ran once per build. No full exact-limit engine recovery is claimed.

Typed outcome or recovery expected

For the same 67108865-byte valid UTF-8 source, retain refusal while exposing its input-size nature and 67108864-byte total allowance in structured fields, with recovery guidance appropriate to that deterministic bound. Keep the exact-limit snapshot admitted, small-source recovery successful, malformed UTF-8 distinguishable, and failure cleanup intact. These are proposed restoration criteria; no fix was applied.

Analysis or suspected cause

Source-derived reasoning: scanRecoveryFile raises AnalysisOutputError on measured input-size exhaustion. snapshotRecoveryInput catches non-cancellation errors and calls recoveryInputFailure, which hard-codes invalid_value and carries the limit only inside its message. The provider preserves this typed error after cleanup, and the shared service and CLI projection cannot restore discarded structured size facts.

Additional context

Related #4621 concerns oversized WASM admission through the stable artifact reader and WABT translator. The tested main build already includes a WABT file-size translation fix in src/wasm/WabtFailures.ts:142-163, yet the public recovery size failure above still reproduces. Recovery uses its own scanner and wrapper. A recovery-specific passing size-classification check is independently required. Open and closed issues and pull requests were searched; no matching recovery report was found. No last-known-good revision is established.

Originally created by @coygeek on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4629 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `recover-javascript-sources` refuses a stable 67108865-byte UTF-8 JavaScript file with `invalid_request` / `invalid_input`, issue reason `invalid_value`, and no structured byte allowance. The response advertises `retryable: true` and generic argument-correction guidance. The 67108864-byte limit survives only in message text, which also calls this input-size refusal an invalid analysis output. A malformed UTF-8 input receives the same issue reason. This reproduced through the real public CLI in 3/3 attempts on each tested build, on Linux x64 with the documented real Wakaru binary configured. Both builds successfully recovered a small source with the same engine. This report concerns the typed refusal; the size limit remains enforced. ### Reproduction 1. Install the release below in a disposable directory, or install the source checkout's locked dependencies and run `npm run build:cached`. On Linux x64, install Wakaru with `npm install wakaru@1.14.0` in a separate disposable directory. Use its absolute `node_modules/@wakaru/cli-linux-x64/wakaru` path, rather than the Node shim. Ensure `/usr/bin/prlimit` is available. 2. Save this as `reproduce.mjs` in a disposable directory. Run `node reproduce.mjs /absolute/path/to/rea/scripts/rea.mjs /absolute/path/to/wakaru` against each build. These are the checks actually run; every CLI subprocess has a fresh HOME and runtime directory and an allowlisted environment. The script removes its fixtures and successful recovery output. ```js import assert from 'node:assert/strict'; import { mkdtemp, mkdir, writeFile, rm, readdir, stat } from 'node:fs/promises'; import { dirname, join, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; import { spawnSync } from 'node:child_process'; const entry = resolve(process.argv[2]); const engine = resolve(process.argv[3]); const packageRoot = dirname(dirname(entry)); const { snapshotRecoveryInputs } = await import(pathToFileURL(join(packageRoot, 'dist/javascript/recovery/RecoveryFiles.js'))); const root = await mkdtemp(join(process.cwd(), 'recovery-size-')); const maximum = 64 * 1024 * 1024; const clean = value => JSON.stringify(value).replaceAll(root, '<fixture>'); try { await writeFile(join(root, 'over.js'), Buffer.alloc(maximum + 1, 32)); await writeFile(join(root, 'exact.js'), Buffer.alloc(maximum, 32)); await writeFile(join(root, 'invalid.js'), Buffer.from([255])); await writeFile(join(root, 'small.js'), 'console.log("fixture");\n'); const har = Buffer.from('{"log":{"version":"1.2","creator":{"name":"fixture","version":"1"},"entries":[]}}'); await writeFile(join(root, 'over.har'), Buffer.concat([har, Buffer.alloc(33554433 - har.length, 32)])); let sequence = 0; for (const name of ['over.js', 'over.js', 'over.js', 'invalid.js', 'small.js', 'over.har']) { const callRoot = join(root, String(sequence++)); await mkdir(join(callRoot, 'home'), { recursive: true }); await mkdir(join(callRoot, 'runtime')); const output = join(callRoot, 'output'); const env = { PATH: process.env.PATH, HOME: join(callRoot, 'home'), TMPDIR: join(callRoot, 'runtime'), LANG: 'C', REA_LOG_LEVEL: 'silent', REA_WAKARU_COMMAND: engine }; const args = name.endsWith('.har') ? ['inspect-web-network-capture', join(root, name), 'har', '--json'] : ['recover-javascript-sources', join(root, name), output, '--json']; const call = spawnSync(process.execPath, [entry, ...args], { env, encoding: 'utf8', timeout: 180000, maxBuffer: 10 * 1024 * 1024 }); assert.equal(call.error, undefined); assert.equal(call.stderr, ''); const reply = JSON.parse(call.stdout); const residuals = await readdir(join(callRoot, 'runtime')); const published = await stat(output).then(() => true, () => false); console.log(name, clean({ exit: call.status, code: reply.code, category: reply.category, retryable: reply.retryable, remediation: reply.remediation, details: reply.details, status: reply.normalized_result?.status, residuals, published })); assert.equal(call.status, name === 'small.js' ? 0 : 1); assert.deepEqual(residuals, []); assert.equal(published, name === 'small.js'); if (name === 'over.js') { assert.equal(reply.code, 'invalid_request'); assert.equal(reply.details.issues[0].reason, 'invalid_value'); assert.equal(reply.details.issues[0].expected, undefined); assert.equal(reply.details.issues[0].maximum, undefined); assert.equal(reply.details.reported_limits, undefined); assert.match(reply.details.issues[0].message, /67108864/); } if (name === 'invalid.js') assert.equal(reply.details.issues[0].reason, 'invalid_value'); if (name === 'over.har') { assert.equal(reply.details.issues[0].reason, 'out_of_range'); assert.equal(reply.details.issues[0].expected.maximum_capture_bytes, 33554432); } } const exact = await snapshotRecoveryInputs([join(root, 'exact.js')], join(root, 'snapshot')); assert.equal(exact[0].bytes, maximum); console.log('exact.js snapshot accepted_bytes=' + exact[0].bytes); } finally { await rm(root, { recursive: true, force: true }); } ``` 3. Observe the three oversized recovery calls return `invalid_value` without `expected`, `maximum`, or `reported_limits`. The malformed UTF-8 call also returns `invalid_value`; small-source recovery succeeds. The sibling HAR CLI reports `out_of_range` with a structured allowance. The exact-limit recovery snapshot succeeds through the production snapshot function; that last control is an internal boundary check, not an exact-limit public recovery run. ### Expected behavior Keep enforcing the total selected-input allowance while exposing a machine-readable distinction between input-size exhaustion and malformed source, with the configured byte budget in structured fields. The recovery guide promises explicit failure and rollback when the limit is exceeded, without mandating one exact code. The shared error algebra distinguishes measured resource constraints and bounded admission refusals from malformed input and supports structured file-size limits. The sibling capture workflow uses `out_of_range` plus structured expected-limit metadata. Either established representation can preserve the distinction missing here. ### Impact and scope Clients handling limits through typed fields must parse recovery's message to distinguish oversized valid source from malformed input. No limit bypass, source corruption, or downstream client incident was observed. The single stable input CLI case reproduced consistently in this bounded sample; natural workload frequency is unknown. MCP, multi-input exhaustion, and growth during reading were not exercised. - Impact level: P2 - Assessment status: supported - Impact basis: The measured size refusal loses its structured classification and byte allowance, preventing consistent typed limit handling without parsing message text. - Workaround status: available - Workaround basis: Callers can identify the refusal from the retained size-limit message and documented allowance. The budget is still enforced. Truncating required source bytes would not preserve the original analysis and is not proposed as a workaround. ### REA version Published npm `rea-agents` 6.5.0 and fresh main source commit `8451d283920e86936576fdf67615807d1b4ca4a9`. Both public entrypoints report `6.5.0`; source dependencies and build used the pinned toolchain. ### Runtime and platform Node.js v24.18.0; npm 11.16.0; Debian GNU/Linux 12 x86_64 in a Linux x64 Docker container with network disabled during the checks; Wakaru 1.14.0, SHA-256 `664fa06782a61eda7ed15885134b52525fd565dd77607e642062ceb6cd927c38`, matching the recovery guide. The container ran on a Darwin arm64 host. ### Target and artifact context Self-generated JavaScript whitespace files of 67108865 and 67108864 ASCII spaces; a one-byte invalid UTF-8 file (`ff`); and a 24-byte small source. The HAR comparison is valid empty HAR JSON padded with trailing JSON whitespace to 33554433 bytes. The selected application code is never executed by recovery. ### Interface CLI ### Earliest failure stage Setup or provider admission ### Relevant output or evidence - Expected source: `docs/javascript-recovery.md:132-137`; `src/domain/analysisErrorCore.ts:23-54,80-94`; `src/domain/analysisErrorProjection.ts:258-263,287-297`; and `src/browser/history/CaptureFailures.ts:124-138` in the tested main tree. - Failure source: The public CLI results below; source breadcrumbs are `src/javascript/recovery/RecoveryFiles.ts:55-60,139-148,287-294` and `src/javascript/recovery/WakaruProvider.ts:109-120,236-242`. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Meaningful output from both builds, with the generated fixture prefix replaced by `<fixture>`: ```text recover-javascript-sources over.js <absent-output> --json, 3/3 each: exit=1; code=invalid_request; category=invalid_input; retryable=true remediation.action="Correct the listed arguments and retry." issue.path=["path"]; issue.reason=invalid_value issue.message="Cannot snapshot UTF-8 JavaScript input <fixture>/over.js: Invalid analysis output for recover_javascript_sources: Expected a regular file no larger than 67108864 bytes: <fixture>/over.js" no issue.expected, issue.maximum, details.resource or details.reported_limits recover-javascript-sources invalid.js <absent-output> --json: exit=1; issue.reason=invalid_value issue.message ends with "The encoded data was not valid for encoding utf-8" recover-javascript-sources small.js <absent-output> --json: exit=0; normalized_result.status=complete inspect-web-network-capture over.har har --json: exit=1; issue.reason=out_of_range issue.expected={"maximum_capture_bytes":33554432} Production snapshotRecoveryInputs, exact.js: accepted_bytes=67108864 ``` All exercised CLI stderr streams were empty. Every per-call runtime directory was empty afterward; failed calls left the requested output absent, while small-source recovery created its output. The exact-limit snapshot and each non-oversized control ran once per build. No full exact-limit engine recovery is claimed. ### Typed outcome or recovery expected For the same 67108865-byte valid UTF-8 source, retain refusal while exposing its input-size nature and 67108864-byte total allowance in structured fields, with recovery guidance appropriate to that deterministic bound. Keep the exact-limit snapshot admitted, small-source recovery successful, malformed UTF-8 distinguishable, and failure cleanup intact. These are proposed restoration criteria; no fix was applied. ### Analysis or suspected cause Source-derived reasoning: `scanRecoveryFile` raises `AnalysisOutputError` on measured input-size exhaustion. `snapshotRecoveryInput` catches non-cancellation errors and calls `recoveryInputFailure`, which hard-codes `invalid_value` and carries the limit only inside its message. The provider preserves this typed error after cleanup, and the shared service and CLI projection cannot restore discarded structured size facts. ### Additional context Related [#4621](https://github.com/morluto/rea/issues/4621) concerns oversized WASM admission through the stable artifact reader and WABT translator. The tested main build already includes a WABT file-size translation fix in `src/wasm/WabtFailures.ts:142-163`, yet the public recovery size failure above still reproduces. Recovery uses its own scanner and wrapper. A recovery-specific passing size-classification check is independently required. Open and closed issues and pull requests were searched; no matching recovery report was found. No last-known-good revision is established.

bsakaguchi closed this issue

2026-10-17 21:04:15 +00:00

bsakaguchi referenced this issue

2026-10-31 00:30:55 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471