[PR #646] [MERGED] fix(javascript): hash graph and evidence JSON incrementally #2137

New issue

Closed

opened 2026-10-16 21:14:46 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:14:46 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/646
Author: @N0zoM1z0
Created: 10/6/2026
Status: ✅ Merged
Merged: 10/6/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/large-javascript-evidence-digests

📝 Commits (1)

9dd8630 fix(javascript): hash graph and evidence JSON incrementally

📊 Changes

9 files changed (+346 additions, -21 deletions)

View changed files

📝 docs/javascript-artifact-reconstruction.md (+17 -0)
📝 package.json (+1 -0)
➕ scripts/verify-large-javascript-digests.mjs (+38 -0)
➕ src/application/JavaScriptApplicationService.test.ts (+87 -0)
📝 src/application/JavaScriptApplicationService.ts (+39 -1)
➕ src/domain/canonicalDigest.test.ts (+83 -0)
➕ src/domain/canonicalDigest.ts (+75 -0)
📝 src/domain/comparisonSemantics.ts (+3 -5)
📝 src/domain/evidence.ts (+3 -15)

📄 Description

Changes

Hash canonical JSON incrementally for shared graph/comparison digests and Evidence IDs, preserving the canonicalize byte sequence and existing identifiers.
Preserve unexpected JavaScript analysis failures as typed provider errors with caller-visible input path, error name, and message; known filesystem failures retain the existing I/O classification.
Add compatibility/property regressions and an opt-in verify:javascript:digests lane that crosses Node's actual single-string limit without adding that workload to routine tests.

Refs #623. The maintainer's 4.0.1 comment prompted verification: the current main still fails both production hash paths with RangeError: Invalid string length for an aggregate beyond the engine limit.

Verification

Focused domain/service/CLI/MCP/acceptance suite: 9 files, 62 tests passed. Final added compatibility/negative cases: 2 files, 9 tests passed.
npm run check:fast: passed; typecheck, lint, formatting and metadata passed in the static gate; npm run knip passed after registering the opt-in verifier.
npm run docs:check: passed.
node scripts/verify-large-javascript-digests.mjs: passed for 537,921,040 serialized bytes against a 536,870,888 engine string limit on a 1 GiB heap; peak RSS 76,720 KiB. The unchanged production graph and Evidence digest paths both throw on the equivalent fixture.
Actual CLI on a generated public 32-file, 6,304,960-byte JavaScript fixture: complete 27,928,234-byte JSON result, no truncated scopes.
Actual stdio MCP with the pinned client SDK: complete 98,173,521-byte response, authenticated Evidence and the same ID as CLI. The SDK's default 10 MiB read buffer correctly rejects this response; the verification uses its explicit 128 MiB maxBufferSize option.
An unchanged smaller application retains its exact pre-fix Evidence ID.

No real Hopper/Ghidra run was performed for this provider-neutral hashing change.

Remaining coverage

This PR removes aggregate string assembly from hashing. Incur output formatting and MCP JSON serialization still assemble whole strings, so full results beyond the engine's string limit remain a separate output-boundary limitation; this PR leaves the issue open for the remaining output work. CLI field filtering happens before formatting and remains available. The repository's full dist/ reproduction is additionally blocked earlier by an existing empty-endpoint graph schema failure, which is recorded separately rather than folded into this change.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/646 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/6/2026 **Status:** ✅ Merged **Merged:** 10/6/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/large-javascript-evidence-digests` --- ### 📝 Commits (1) - [`9dd8630`](https://github.com/morluto/rea/commit/9dd863099422af50d0e04428e14b1477b8e4ce1f) fix(javascript): hash graph and evidence JSON incrementally ### 📊 Changes **9 files changed** (+346 additions, -21 deletions) <details> <summary>View changed files</summary> 📝 `docs/javascript-artifact-reconstruction.md` (+17 -0) 📝 `package.json` (+1 -0) ➕ `scripts/verify-large-javascript-digests.mjs` (+38 -0) ➕ `src/application/JavaScriptApplicationService.test.ts` (+87 -0) 📝 `src/application/JavaScriptApplicationService.ts` (+39 -1) ➕ `src/domain/canonicalDigest.test.ts` (+83 -0) ➕ `src/domain/canonicalDigest.ts` (+75 -0) 📝 `src/domain/comparisonSemantics.ts` (+3 -5) 📝 `src/domain/evidence.ts` (+3 -15) </details> ### 📄 Description ## Changes - Hash canonical JSON incrementally for shared graph/comparison digests and Evidence IDs, preserving the canonicalize byte sequence and existing identifiers. - Preserve unexpected JavaScript analysis failures as typed provider errors with caller-visible input path, error name, and message; known filesystem failures retain the existing I/O classification. - Add compatibility/property regressions and an opt-in `verify:javascript:digests` lane that crosses Node's actual single-string limit without adding that workload to routine tests. Refs #623. The maintainer's 4.0.1 comment prompted verification: the current main still fails both production hash paths with `RangeError: Invalid string length` for an aggregate beyond the engine limit. ## Verification - Focused domain/service/CLI/MCP/acceptance suite: 9 files, 62 tests passed. Final added compatibility/negative cases: 2 files, 9 tests passed. - `npm run check:fast`: passed; typecheck, lint, formatting and metadata passed in the static gate; `npm run knip` passed after registering the opt-in verifier. - `npm run docs:check`: passed. - `node scripts/verify-large-javascript-digests.mjs`: passed for 537,921,040 serialized bytes against a 536,870,888 engine string limit on a 1 GiB heap; peak RSS 76,720 KiB. The unchanged production graph and Evidence digest paths both throw on the equivalent fixture. - Actual CLI on a generated public 32-file, 6,304,960-byte JavaScript fixture: complete 27,928,234-byte JSON result, no truncated scopes. - Actual stdio MCP with the pinned client SDK: complete 98,173,521-byte response, authenticated Evidence and the same ID as CLI. The SDK's default 10 MiB read buffer correctly rejects this response; the verification uses its explicit 128 MiB `maxBufferSize` option. - An unchanged smaller application retains its exact pre-fix Evidence ID. No real Hopper/Ghidra run was performed for this provider-neutral hashing change. ## Remaining coverage This PR removes aggregate string assembly from hashing. Incur output formatting and MCP JSON serialization still assemble whole strings, so full results beyond the engine's string limit remain a separate output-boundary limitation; this PR leaves the issue open for the remaining output work. CLI field filtering happens before formatting and remains available. The repository's full `dist/` reproduction is additionally blocked earlier by an existing empty-endpoint graph schema failure, which is recorded separately rather than folded into this change. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:14:46 +00:00