[PR #1407] [MERGED] fix(javascript)!: stop copying literals into semantic metadata #2863

New issue

Closed

opened 2026-10-17 21:11:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:11:09 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/1407
Author: @N0zoM1z0
Created: 10/10/2026
Status: ✅ Merged
Merged: 10/10/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: fix/javascript-literal-identity-size

📝 Commits (1)

e456f9d fix(javascript)!: stop copying literals into semantic metadata

📊 Changes

3 files changed (+171 additions, -2 deletions)

View changed files

📝 docs/javascript-application-workflows.md (+12 -0)
📝 src/application/javascript/JavaScriptSemanticGraphValueProjection.ts (+5 -2)
➕ tests/boundary/mcp/javascriptLiteralPayload.test.ts (+154 -0)

📄 Description

Large JavaScript literals are copied into identity.role_key and the string label as well as properties.value. A source-owned 2.10 MB file with a 1,572,864-character literal produces 7.00 MB CLI JSON and an oversized default MCP response. Commit the canonical JSON value by SHA-256 in the role key and use string literal for the display label. Keep the exact value, binding/property role, artifact hash, source range, Evidence, and literal-query behavior.

Refs #723. This fixes producer-side repetition; the broader discovery/text/structured design remains open. It does not establish the private-bundle crash cause in #1387.

On the same owned fixture, bounded Linux x64:

Surface	Before	After
Complete CLI JSON	6,999,140 bytes	2,149,559 bytes (69% smaller)
Literal role key	2,031,674 characters	133 characters
String label	2,031,618 characters	14 characters
Default stdio MCP	typed receive-budget constraint	complete 4,591,550-byte response

Both runs retain the exact literal and the same four semantic nodes/two relations. Node 24.18.0 and 22.22.2 MCP runs verify full value preservation, retained-reference semantic tracing, summary, and subsequent ping. These are serialized-size measurements, not observed model-host token counts or billing.

Validation:

npm run test:focused -- tests/boundary/mcp/javascriptLiteralPayload.test.ts src/application/javascript/JavaScriptSemanticGraphBuilder.test.ts tests/boundary/javascriptSemanticTraceEvidence.test.ts: 56 passed; the final E2E also checks typed 1/"1" seeds and the empty string.
npm run check:fast.
Public CLI accepts the previously produced Evidence and its original literal node ID after this change.
No Hopper/Ghidra verification required; static JavaScript analysis and real public CLI/stdio MCP are exercised.

Compatibility: breaking metadata change, marked !. New literal node IDs use value-sha256 role keys; string labels become string literal. Read exact values from properties.value and obtain IDs from the selected parent graph. Existing stored graphs remain valid. Request/output schema shapes, complete Evidence, and text/structured equivalence are preserved; no caps or truncation are added.

Follow-up: the shorter-encoding refinement was not part of this merged head. It is tracked in #1408 to retain short literal keys/labels while preserving the large-payload reduction.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/1407 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/10/2026 **Status:** ✅ Merged **Merged:** 10/10/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `fix/javascript-literal-identity-size` --- ### 📝 Commits (1) - [`e456f9d`](https://github.com/morluto/rea/commit/e456f9d1a841013ed2fd61069e1aaa41a129cb9a) fix(javascript)!: stop copying literals into semantic metadata ### 📊 Changes **3 files changed** (+171 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `docs/javascript-application-workflows.md` (+12 -0) 📝 `src/application/javascript/JavaScriptSemanticGraphValueProjection.ts` (+5 -2) ➕ `tests/boundary/mcp/javascriptLiteralPayload.test.ts` (+154 -0) </details> ### 📄 Description Large JavaScript literals are copied into `identity.role_key` and the string label as well as `properties.value`. A source-owned 2.10 MB file with a 1,572,864-character literal produces 7.00 MB CLI JSON and an oversized default MCP response. Commit the canonical JSON value by SHA-256 in the role key and use `string literal` for the display label. Keep the exact value, binding/property role, artifact hash, source range, Evidence, and literal-query behavior. Refs #723. This fixes producer-side repetition; the broader discovery/text/structured design remains open. It does not establish the private-bundle crash cause in #1387. On the same owned fixture, bounded Linux x64: | Surface | Before | After | | --- | --- | --- | | Complete CLI JSON | 6,999,140 bytes | 2,149,559 bytes (69% smaller) | | Literal role key | 2,031,674 characters | 133 characters | | String label | 2,031,618 characters | 14 characters | | Default stdio MCP | typed receive-budget constraint | complete 4,591,550-byte response | Both runs retain the exact literal and the same four semantic nodes/two relations. Node 24.18.0 and 22.22.2 MCP runs verify full value preservation, retained-reference semantic tracing, summary, and subsequent ping. These are serialized-size measurements, not observed model-host token counts or billing. Validation: - `npm run test:focused -- tests/boundary/mcp/javascriptLiteralPayload.test.ts src/application/javascript/JavaScriptSemanticGraphBuilder.test.ts tests/boundary/javascriptSemanticTraceEvidence.test.ts`: 56 passed; the final E2E also checks typed `1`/`"1"` seeds and the empty string. - `npm run check:fast`. - Public CLI accepts the **previously produced** Evidence and its original literal node ID after this change. - No Hopper/Ghidra verification required; static JavaScript analysis and real public CLI/stdio MCP are exercised. Compatibility: **breaking metadata change**, marked `!`. New literal node IDs use `value-sha256` role keys; string labels become `string literal`. Read exact values from `properties.value` and obtain IDs from the selected parent graph. Existing stored graphs remain valid. Request/output schema shapes, complete Evidence, and text/structured equivalence are preserved; no caps or truncation are added. Follow-up: the shorter-encoding refinement was not part of this merged head. It is tracked in #1408 to retain short literal keys/labels while preserving the large-payload reduction. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-17 21:11:09 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-10-31 01:06:42 +00:00

[PR #2863] [CLOSED] fix(saz): accept U+FFFD in well-formed metadata #5447

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2863

No description provided.

Rows
Columns