[PR #1350] [MERGED] fix(json): preserve runtime limits across JSON file readers #2817

New issue

Closed

opened 2026-10-17 21:10:37 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:10:37 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1350
Author: @N0zoM1z0
Created: 10/9/2026
Status: ✅ Merged
Merged: 10/9/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/cli-json-runtime-limits

📝 Commits (5)

16e58c5 fix(cli): preserve runtime JSON string-limit diagnostics
26a1a38 fix(cli): integrate string-limit follow-up with current main
7a22868 test(json): check actionable oversized Evidence recovery
6e920eb fix(json): decode large UTF-8 inputs by character length
773df0c test(json): reject truncated UTF-8 after large input decode

📊 Changes

8 files changed (+232 additions, -81 deletions)

View changed files

📝 src/application/EvidenceBundleFiles.ts (+8 -2)
📝 src/application/JsonFiles.ts (+13 -12)
➕ src/application/Utf8JsonInput.ts (+78 -0)
📝 src/application/binary/AnalysisSnapshotFiles.ts (+4 -1)
📝 src/cliJsonInput.ts (+50 -64)
📝 src/cliJsonInputStatus.test.ts (+1 -0)
➕ tests/boundary/cli/jsonInputSize.test.ts (+69 -0)
📝 tests/boundary/filesystem/cliJsonInput.test.ts (+9 -2)

📄 Description

Evidence JSON file reads still classify Node’s string limit as invalid-json, including bundle and snapshot imports. Share strict UTF-8 decoding with CLI file input and preserve this failure as the existing resource_constraint outcome, with the selected path, observed byte count and runtime string limit. Invalid UTF-8 and invalid JSON retain their classifications; unrelated decoder failures remain read failures.

Build on main’s CLI fix: retain inline-input handling, the too-large envelope/exit status and existing resource/limit fields. Add size observations and recovery by re-analyzing a smaller selection.

Also fix supported Node 22 rejecting valid multibyte JSON by encoded byte count before decoding. Only inputs whose bytes exceed the native string limit use bounded streaming UTF-8 decoding, preserving split characters and checking the actual decoded UTF-16 length before joining. JSON parsing remains whole-document and requires the decoded text to fit in one runtime string.

Verification: npm run build:cached; npm run check:ci (all 8 static tasks); prior seven-file integration run (95 cases passed), followed by the three affected boundary files on both Node 22.22.2 and pinned Node 24.18.0 (12 cases passed each). An official Node 22 binary independently reproduced ERR_STRING_TOO_LONG for 536,870,894 encoded bytes / 268,435,446 UTF-16 code units; the production parser accepts the same valid JSON after this fix. Compiled CLI and Evidence readers reject genuinely oversized decoded strings with status 1, and existing large valid JSON, multibyte, malformed input, UTF-8, filesystem diagnostics and bundle tests pass. A final reused large-file fixture also verifies an incomplete UTF-8 suffix is rejected after flushing on both runtimes (9 cases each). Large cases run sequentially; no large fixture is committed. Real Hopper/Ghidra verification was not performed because no provider behavior changes.

Fixes #1331 (remaining Evidence reader boundary, Node 22 compatibility and recovery diagnostics).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1350 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/9/2026 **Status:** ✅ Merged **Merged:** 10/9/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/cli-json-runtime-limits` --- ### 📝 Commits (5) - [`16e58c5`](https://github.com/morluto/rea/commit/16e58c5e5a68ac00503f566e06ff517729e8314e) fix(cli): preserve runtime JSON string-limit diagnostics - [`26a1a38`](https://github.com/morluto/rea/commit/26a1a38632e10225b87e628e8afcf23513a30dd6) fix(cli): integrate string-limit follow-up with current main - [`7a22868`](https://github.com/morluto/rea/commit/7a228686da31994a556fceb762347b7cdc0b4dd0) test(json): check actionable oversized Evidence recovery - [`6e920eb`](https://github.com/morluto/rea/commit/6e920ebf2160676ef0dd2bfca65a08e07338018e) fix(json): decode large UTF-8 inputs by character length - [`773df0c`](https://github.com/morluto/rea/commit/773df0cefe184f3948ce7230f9921c402582ded0) test(json): reject truncated UTF-8 after large input decode ### 📊 Changes **8 files changed** (+232 additions, -81 deletions) <details> <summary>View changed files</summary> 📝 `src/application/EvidenceBundleFiles.ts` (+8 -2) 📝 `src/application/JsonFiles.ts` (+13 -12) ➕ `src/application/Utf8JsonInput.ts` (+78 -0) 📝 `src/application/binary/AnalysisSnapshotFiles.ts` (+4 -1) 📝 `src/cliJsonInput.ts` (+50 -64) 📝 `src/cliJsonInputStatus.test.ts` (+1 -0) ➕ `tests/boundary/cli/jsonInputSize.test.ts` (+69 -0) 📝 `tests/boundary/filesystem/cliJsonInput.test.ts` (+9 -2) </details> ### 📄 Description Evidence JSON file reads still classify Node’s string limit as `invalid-json`, including bundle and snapshot imports. Share strict UTF-8 decoding with CLI file input and preserve this failure as the existing `resource_constraint` outcome, with the selected path, observed byte count and runtime string limit. Invalid UTF-8 and invalid JSON retain their classifications; unrelated decoder failures remain read failures. Build on main’s CLI fix: retain inline-input handling, the `too-large` envelope/exit status and existing resource/limit fields. Add size observations and recovery by re-analyzing a smaller selection. Also fix supported Node 22 rejecting valid multibyte JSON by encoded byte count before decoding. Only inputs whose bytes exceed the native string limit use bounded streaming UTF-8 decoding, preserving split characters and checking the actual decoded UTF-16 length before joining. JSON parsing remains whole-document and requires the decoded text to fit in one runtime string. Verification: `npm run build:cached`; `npm run check:ci` (all 8 static tasks); prior seven-file integration run (95 cases passed), followed by the three affected boundary files on both Node 22.22.2 and pinned Node 24.18.0 (12 cases passed each). An official Node 22 binary independently reproduced `ERR_STRING_TOO_LONG` for 536,870,894 encoded bytes / 268,435,446 UTF-16 code units; the production parser accepts the same valid JSON after this fix. Compiled CLI and Evidence readers reject genuinely oversized decoded strings with status 1, and existing large valid JSON, multibyte, malformed input, UTF-8, filesystem diagnostics and bundle tests pass. A final reused large-file fixture also verifies an incomplete UTF-8 suffix is rejected after flushing on both runtimes (9 cases each). Large cases run sequentially; no large fixture is committed. Real Hopper/Ghidra verification was not performed because no provider behavior changes. Fixes #1331 (remaining Evidence reader boundary, Node 22 compatibility and recovery diagnostics). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:10:37 +00:00