morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1331] [Bug] CLI JSON input above the V8 string limit is reported as invalid-json, so large Evidence cannot be traced #466

New issue

Closed

opened 2026-10-15 18:25:18 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:25:18 +00:00

Owner

Originally created by @jopemachine on GitHub (Oct 9, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1331

Before submitting

I searched the open and closed issues for a related report. Related but distinct: #623 masked a RangeError as io during analysis, and #1053 / #1065 cover delivering large results over MCP. None covers CLI follow-up input.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

rea trace-application-feature <file.json> rejects a valid 768,736,012-byte JSON file within about one second with invalid_request and input_reason: "invalid-json". The file wraps Evidence that rea analyze-javascript-application --format json had just written (733 MB). The same command succeeds when the wrapped Evidence is 496 MB.

parseCliJsonInput reads the file as bytes and decodes it with TextDecoder("utf-8", { fatal: true }). Above V8's maximum string length the decode throws Error: Cannot create a string longer than 0x1fffffe8 characters. The catch in parseJson treats every decoding failure as invalid UTF-8 ("Decoding failure means the bytes are not valid UTF-8 JSON input") and returns undefined, so the caller reports invalid-json.

Reproduction

Decoder behavior alone:

// node limit.mjs 500000000  -> decodes
// node limit.mjs 600000000  -> Error: Cannot create a string longer than 0x1fffffe8 characters
const size = Number(process.argv[2])
const body = Buffer.alloc(size, 0x61)
body[0] = 0x22
body[size - 1] = 0x22
JSON.parse(new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(body))

With the CLI:

Produce Evidence above roughly 540 MB: rea analyze-javascript-application <dir> --artifact-format directory --format json > app.json. Our input was 33 minified webpack/rspack chunks totalling 7.0 MB, which produced 733 MB of Evidence.
Wrap it: { printf '{"application":'; cat app.json; printf ',"seed":{"kind":"module","value":"<module id>","match":"exact"},"direction":"incoming"}'; } > trace.json
rea trace-application-feature trace.json --format json returns input_reason: "invalid-json".

Expected behavior

The error names the actual cause, an input too large for the runtime to hold as one string, with a size-specific reason and remediation instead of claiming the JSON is invalid. Ideally, CLI follow-up workflows could consume Evidence the CLI itself produced, for example through a streaming parse or a reference to retained Evidence instead of inline Evidence.

Impact and scope

Every CLI workflow that takes inline Evidence, such as trace-application-feature, is unusable on an analysis whose Evidence exceeds about 512 MiB. Evidence grows much faster than its input (roughly 100–900× in our runs), so a few MB of minified JavaScript is enough to reach the limit. Evidence integrity is not affected; the problems are the misleading classification and the lack of a path for large inputs. Our workaround was to re-analyze a smaller subset: 0.55 MB of chunks produced 496 MB of Evidence, which traced normally.

REA version

6.1.0 (npm)

Runtime and platform

Node.js 26.10.0, macOS 27.0.1, arm64

Target and artifact context

A directory of minified webpack/rspack chunks from a public web application bundle, analyzed statically.

Interface

CLI

Earliest failure stage

Request parsing or validation

Originally created by @jopemachine on GitHub (Oct 9, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1331 ### Before submitting - [x] I searched the open and closed issues for a related report. Related but distinct: #623 masked a `RangeError` as `io` during analysis, and #1053 / #1065 cover delivering large results over MCP. None covers CLI follow-up input. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior `rea trace-application-feature <file.json>` rejects a valid 768,736,012-byte JSON file within about one second with `invalid_request` and `input_reason: "invalid-json"`. The file wraps Evidence that `rea analyze-javascript-application --format json` had just written (733 MB). The same command succeeds when the wrapped Evidence is 496 MB. `parseCliJsonInput` reads the file as bytes and decodes it with `TextDecoder("utf-8", { fatal: true })`. Above V8's maximum string length the decode throws `Error: Cannot create a string longer than 0x1fffffe8 characters`. The `catch` in `parseJson` treats every decoding failure as invalid UTF-8 ("Decoding failure means the bytes are not valid UTF-8 JSON input") and returns `undefined`, so the caller reports `invalid-json`. ### Reproduction Decoder behavior alone: ```js // node limit.mjs 500000000 -> decodes // node limit.mjs 600000000 -> Error: Cannot create a string longer than 0x1fffffe8 characters const size = Number(process.argv[2]) const body = Buffer.alloc(size, 0x61) body[0] = 0x22 body[size - 1] = 0x22 JSON.parse(new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(body)) ``` With the CLI: 1. Produce Evidence above roughly 540 MB: `rea analyze-javascript-application <dir> --artifact-format directory --format json > app.json`. Our input was 33 minified webpack/rspack chunks totalling 7.0 MB, which produced 733 MB of Evidence. 2. Wrap it: `{ printf '{"application":'; cat app.json; printf ',"seed":{"kind":"module","value":"<module id>","match":"exact"},"direction":"incoming"}'; } > trace.json` 3. `rea trace-application-feature trace.json --format json` returns `input_reason: "invalid-json"`. ### Expected behavior The error names the actual cause, an input too large for the runtime to hold as one string, with a size-specific reason and remediation instead of claiming the JSON is invalid. Ideally, CLI follow-up workflows could consume Evidence the CLI itself produced, for example through a streaming parse or a reference to retained Evidence instead of inline Evidence. ### Impact and scope Every CLI workflow that takes inline Evidence, such as `trace-application-feature`, is unusable on an analysis whose Evidence exceeds about 512 MiB. Evidence grows much faster than its input (roughly 100–900× in our runs), so a few MB of minified JavaScript is enough to reach the limit. Evidence integrity is not affected; the problems are the misleading classification and the lack of a path for large inputs. Our workaround was to re-analyze a smaller subset: 0.55 MB of chunks produced 496 MB of Evidence, which traced normally. ### REA version 6.1.0 (npm) ### Runtime and platform Node.js 26.10.0, macOS 27.0.1, arm64 ### Target and artifact context A directory of minified webpack/rspack chunks from a public web application bundle, analyzed statically. ### Interface CLI ### Earliest failure stage Request parsing or validation

bsakaguchi

2026-10-15 18:25:18 +00:00

closed this issue
added the
area: cli

P1: high

bug
labels

bsakaguchi commented

2026-10-15 18:25:19 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 9, 2026):

TL;DR: Same thing here on Linux with 6.2.0. The file isn't bad JSON. It's just bigger than Node can hold in one string, about 537 MB, and REA reports that as invalid-json. The two failures carry different error codes. So the catch can tell them apart and say what went wrong.

Reproduced on Linux with the published rea-agents@6.2.0. On main at 1c123493a, src/cliJsonInput.ts still has the same TextDecoder catch in parseJson.

Two synthetic files of the shape {"application":{"padding":"aaa…"},"seed":{"kind":"module","value":"x","match":"exact"},"direction":"incoming"}, differing only in padding length, through rea trace-application-feature <file> --format json:

file	result	wall	max RSS
500,000,000 bytes	parsed, then rejected on schema: `missing_argument` for `application.evidence_id`, `application.subject`, `application.provider`, …	2.0 s	about 1.8 GB
600,000,000 bytes	`input_reason: "invalid-json"`, one issue `{ "path": [], "reason": "invalid_format", "expected": "JSON" }`	1.6 s	about 0.9 GB

So the same valid JSON is reported as invalid once it crosses the string cap. The decoder-only script from the report behaves the same here: 500,000,000 decodes, 600,000,000 throws Cannot create a string longer than 0x1fffffe8 characters. The cliff is exact: 536,870,888 ASCII bytes (0x1fffffe8) decode, one more byte throws.

One detail that may help the fix: the throw is not the fatal decoder's TypeError. Invalid UTF-8 throws TypeError with code: "ERR_ENCODING_INVALID_ENCODED_DATA", while the oversized buffer throws a plain Error with code: "ERR_STRING_TOO_LONG" (Node 22). The catch could tell them apart by code and report a size-specific reason instead of invalid-json.

Test setup

Box: Linux x86-64
Node: 22.22.2

🤖 Generated with Claude Code (Fable 5.1) 🦊

@CassieFox23 commented on GitHub (Oct 9, 2026): **TL;DR:** Same thing here on Linux with 6.2.0. The file isn't bad JSON. It's just bigger than Node can hold in one string, about 537 MB, and REA reports that as `invalid-json`. The two failures carry different error codes. So the catch can tell them apart and say what went wrong. Reproduced on Linux with the published `rea-agents@6.2.0`. On main at 1c123493a, `src/cliJsonInput.ts` still has the same `TextDecoder` catch in `parseJson`. Two synthetic files of the shape `{"application":{"padding":"aaa…"},"seed":{"kind":"module","value":"x","match":"exact"},"direction":"incoming"}`, differing only in padding length, through `rea trace-application-feature <file> --format json`: | file | result | wall | max RSS | | --- | --- | --- | --- | | 500,000,000 bytes | parsed, then rejected on schema: `missing_argument` for `application.evidence_id`, `application.subject`, `application.provider`, … | 2.0 s | about 1.8 GB | | 600,000,000 bytes | `input_reason: "invalid-json"`, one issue `{ "path": [], "reason": "invalid_format", "expected": "JSON" }` | 1.6 s | about 0.9 GB | So the same valid JSON is reported as invalid once it crosses the string cap. The decoder-only script from the report behaves the same here: 500,000,000 decodes, 600,000,000 throws `Cannot create a string longer than 0x1fffffe8 characters`. The cliff is exact: 536,870,888 ASCII bytes (0x1fffffe8) decode, one more byte throws. One detail that may help the fix: the throw is not the fatal decoder's `TypeError`. Invalid UTF-8 throws `TypeError` with `code: "ERR_ENCODING_INVALID_ENCODED_DATA"`, while the oversized buffer throws a plain `Error` with `code: "ERR_STRING_TOO_LONG"` (Node 22). The catch could tell them apart by `code` and report a size-specific reason instead of `invalid-json`. <details> <summary>Test setup</summary> - Box: Linux x86-64 - Node: 22.22.2 </details> 🤖 Generated with Claude Code (Fable 5.1) 🦊

bsakaguchi commented

2026-10-15 18:25:19 +00:00

Author

Owner

@CassieFox23 commented on GitHub (Oct 9, 2026):

TL;DR: This is fixed on main. Thanks N0zoM1z0 for both fixes! A 564 MB trace request now goes through. On 6.2.0 the same file is still called "not JSON".

Edit: swapped an app name for a description.

What I ran:

I built main at 7738c1f, which has #1350 and #1360.
Main's analyze-javascript-application on the asar of a closed-source Electron app from Flathub wrote 563,647,505 bytes of Evidence. That's above V8's 536,870,888-character string limit.
I wrapped it the way the issue shows, with a module seed of app.js and incoming. The request file is 563,647,602 bytes.

`trace-application-feature` on that file	Result
Published 6.2.0	`invalid_request`, `invalid_format`, expected `JSON`, in about 2 s
Main at `7738c1f`	exit 0 in 63 s: 2 seeds matched, 6 nodes, 6 edges

Main's run peaked at about 2.8 GB of memory (/usr/bin/time -v).

Test setup

Box: Linux x86-64
Node: 22.23.1

🤖 Generated with Claude Code (Opus 5.5) 🦊

@CassieFox23 commented on GitHub (Oct 9, 2026): **TL;DR:** This is fixed on main. Thanks N0zoM1z0 for both fixes! A 564 MB trace request now goes through. On 6.2.0 the same file is still called "not JSON". Edit: swapped an app name for a description. What I ran: - I built main at 7738c1f, which has #1350 and #1360. - Main's `analyze-javascript-application` on the asar of a closed-source Electron app from Flathub wrote 563,647,505 bytes of Evidence. That's above V8's 536,870,888-character string limit. - I wrapped it the way the issue shows, with a module seed of `app.js` and `incoming`. The request file is 563,647,602 bytes. | `trace-application-feature` on that file | Result | |---|---| | Published 6.2.0 | `invalid_request`, `invalid_format`, expected `JSON`, in about 2 s | | Main at 7738c1f | exit 0 in 63 s: 2 seeds matched, 6 nodes, 6 edges | Main's run peaked at about 2.8 GB of memory (`/usr/bin/time -v`). <details> <summary>Test setup</summary> - Box: Linux x86-64 - Node: 22.23.1 </details> 🤖 Generated with Claude Code (Opus 5.5) 🦊

bsakaguchi referenced this issue

2026-10-16 21:13:21 +00:00

[PR #466] [MERGED] docs: clarify the agent reverse-engineering promise #2011

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#466

No description provided.

Rows
Columns