[PR #4140] [MERGED] fix(javascript): enforce semantic byte bounds for literals #6080

Closed
opened 2026-10-31 01:11:41 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4140
Author: @N0zoM1z0
Created: 10/15/2026
Status: ✅ Merged
Merged: 10/15/2026
Merged by: @N0zoM1z0

Base: main ← Head: fix/javascript-literal-byte-budget


📝 Commits (4)

  • 40b9d3c fix(javascript): enforce semantic byte bounds for literals
  • 38d263a fix(javascript): retain resource observations across semantic projections
  • 5808872 fix(javascript): normalize retained request endpoint values
  • edaeb24 fix(javascript): preserve complete endpoint lookup identities

📊 Changes

17 files changed (+1024 additions, -57 deletions)

View changed files

📝 docs/javascript-application-workflows.md (+3 -0)
📝 docs/javascript-artifact-reconstruction.md (+12 -1)
➕ docs/verification/javascript-literal-byte-budget-2026-10-15.json (+547 -0)
➕ docs/verification/javascript-literal-byte-budget-2026-10-15.md (+86 -0)
📝 src/application/javascript/JavaScriptSemanticGraphDataProjection.ts (+17 -2)
📝 src/domain/javascript/javascriptSemanticAnalysis.ts (+2 -7)
📝 src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts (+114 -1)
📝 src/domain/javascript/javascriptSemanticDataEffectHelpers.ts (+0 -9)
📝 src/domain/javascript/javascriptSemanticDataEffects.ts (+14 -3)
📝 src/domain/javascript/javascriptSemanticIr.ts (+5 -1)
📝 src/domain/javascript/javascriptSemanticPrimitives.ts (+10 -1)
📝 src/domain/javascript/javascriptSemanticResourceLimits.ts (+0 -19)
📝 src/domain/javascript/javascriptSemanticState.ts (+15 -0)
📝 src/domain/javascript/javascriptSemanticValues.ts (+13 -5)
📝 tests/acceptance/applications/javascriptEndpointPayload.test.ts (+18 -4)
➕ tests/boundary/mcp/javascriptLiteralBudget.test.ts (+137 -0)
📝 tests/boundary/mcp/javascriptLiteralPayload.test.ts (+31 -4)

📄 Description

Summary

Direct/cooked string literals could bypass the documented primitive-value byte estimate. Normalize them and configuration defaults through the shared primitive owner, preserve limiting values through member/destructuring projection, and record encountered bounds independently of later selected values.

Fixes #3933. Part of #4027 and decision #4091.

Problem and expected behavior

A 200,000-unit normalized primitive must obey the existing 1 MiB worst-case JSON estimate. Retain original source/digests and exact bounded values. A selected healthy sibling can remain exactly 7 while whole-file coverage records its discarded sibling's limit.

Change and scope

  • Validate primitive singletons without Set/sort/canonical-key allocation.
  • Record encountered classifications in per-analysis state; remove the recursive retained-value scan.
  • Preserve the shared slot owner's tagged unknown through projection.
  • Normalize configuration defaults once, remove the duplicate raw reader, preserve known null/false/0/empty, and project value_resource_limit with an anchored typed unknown.
  • Preserve full endpoint lookup identities for query/trace/comparison. The same large string is now tested as both a bounded primitive and an exact endpoint.
  • Keep aggregate default-wire capacity with 16 independently admitted values and at least the former total payload.

Shared domain/application owners serve CLI/MCP. Inputs and advertised schemas are unchanged. The optional configuration limiting fact is projected into existing generic graph/coverage records; no worker protocol variant, dependency or runtime capability is added. This static-analysis correction requires no new maintainer scope decision. Ignored catalogs/skills are regenerated and checked.

Expected existing effects:

  • Read existing files or captured data.
  • Launch an owned REA analysis worker.
  • Write private staged source files.

Evidence and regression coverage

Original main reproduced both public direct-literal failures. Review head 40b9d3c5 reproduced projection/default gaps (8 failed/25 passed); those valid findings are fixed. Exact boundary arithmetic is corrected: 174,763 units estimate 1,048,580 bytes.

An attempted extension at 5808872 applied the primitive policy to endpoint identities. Although its new local tests and real Obsidian control passed, existing CI endpoint acceptance failed (1 failed/1,573 passed/19 skipped in shard 4). That was our compatibility regression. The extension is reverted; the original large endpoint input and its full lookup/feature-trace/semantic-trace/version-comparison assertions remain, now alongside the identical string as a bounded primitive. The observation behind that review was real, but the requested policy does not apply to this separate identity representation.

Final production src is byte-identical to validated 38d263a. Existing 66 focused and 932 semantic passes apply to that state. Final corrected public selection passed 7/7 in two files, adding one strengthened acceptance workflow: 67 distinct focused cases across invocations, not one 67-case run. Prior/rejected attempts are preserved in the verification records.

Real Obsidian 1.14.4, selected package.json/main.js: fresh final run preserves exact baseline parent/view Evidence, 23 facts, source coverage/unknowns and CLI/MCP parity. Complete verifier before/final: 33.394/32.949 s wall, 33.017/32.559 s CPU, 1,979,863,040/1,880,096,768 B peak cgroup memory. One CPU/8 GiB, clean scope/no OOM. Uncontrolled caches/scheduling and one control pair establish no speedup or equivalence bound.

Static reference-task BPE stays 16,548 with selected name/description/inputSchema contracts (204,093 with the full catalog projection). This is a control, not a new token-saving claim; annotations/output schemas, client replay/rendering, reasoning and hosted-model use are unmeasured.

Reproduction, source hashes, measurements and rejected attempts.

  • Observed, derived and inferred claims are distinguishable.
  • Artifact identity, source provenance and failed attempts are retained.
  • Partial, unsupported and unknown outcomes remain visible.

Validation performed

  • Focused/public and semantic tests as described above; original aggregate-wire and large-endpoint capacities retained.
  • npm run check: passed typecheck, lint, format, knip, module boundaries and architecture guards.
  • npm run docs:check: passed, including five documentation consistency tests.
  • Fresh real Obsidian verifier, three rounds, CLI parity and tiktoken 0.12.0: passed.
  • git diff --check: passed.

Local verification is Linux x64, Node 24.18.0/npm 11.16.0, static shipped ASAR. Live Electron and other-platform runtime behavior were not exercised locally. Final head edaeb247 passed 18 checks; 15 lanes were conditionally skipped. The merged source/built test report has 15,614 passed and 139 skipped. Both automated reviews completed on this head with no new findings or unresolved threads. Merged as 9f8659e0536e4e50a438f5b9d5661f3c9624f6ac; #3933 is closed.

Author readiness

Human owner: @N0zoM1z0

The session authorized implementation, validation, ready PRs and merging, including waiving manual declarations. The agent reviewed the diff/evidence. A human final-diff review is not asserted; declarations remain unchecked.

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • Concrete problem, scope, authority, related issue and conventional title identified.
  • Tests preserve limiting outcomes, precision, source evidence, caller choices and aggregate capacity.
  • Owning docs/contracts updated; generated artifacts checked and excluded from commits.
  • Final diff checked for secrets, unrelated changes and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4140 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/15/2026 **Status:** ✅ Merged **Merged:** 10/15/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `fix/javascript-literal-byte-budget` --- ### 📝 Commits (4) - [`40b9d3c`](https://github.com/morluto/rea/commit/40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931) fix(javascript): enforce semantic byte bounds for literals - [`38d263a`](https://github.com/morluto/rea/commit/38d263a369e7853ccfae1c8e0a3bd387f920c7cf) fix(javascript): retain resource observations across semantic projections - [`5808872`](https://github.com/morluto/rea/commit/580887209c4fceda432eac4083fac82a0dd78f15) fix(javascript): normalize retained request endpoint values - [`edaeb24`](https://github.com/morluto/rea/commit/edaeb247540291695ebdfdec955cd0c671819134) fix(javascript): preserve complete endpoint lookup identities ### 📊 Changes **17 files changed** (+1024 additions, -57 deletions) <details> <summary>View changed files</summary> 📝 `docs/javascript-application-workflows.md` (+3 -0) 📝 `docs/javascript-artifact-reconstruction.md` (+12 -1) ➕ `docs/verification/javascript-literal-byte-budget-2026-10-15.json` (+547 -0) ➕ `docs/verification/javascript-literal-byte-budget-2026-10-15.md` (+86 -0) 📝 `src/application/javascript/JavaScriptSemanticGraphDataProjection.ts` (+17 -2) 📝 `src/domain/javascript/javascriptSemanticAnalysis.ts` (+2 -7) 📝 `src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts` (+114 -1) 📝 `src/domain/javascript/javascriptSemanticDataEffectHelpers.ts` (+0 -9) 📝 `src/domain/javascript/javascriptSemanticDataEffects.ts` (+14 -3) 📝 `src/domain/javascript/javascriptSemanticIr.ts` (+5 -1) 📝 `src/domain/javascript/javascriptSemanticPrimitives.ts` (+10 -1) 📝 `src/domain/javascript/javascriptSemanticResourceLimits.ts` (+0 -19) 📝 `src/domain/javascript/javascriptSemanticState.ts` (+15 -0) 📝 `src/domain/javascript/javascriptSemanticValues.ts` (+13 -5) 📝 `tests/acceptance/applications/javascriptEndpointPayload.test.ts` (+18 -4) ➕ `tests/boundary/mcp/javascriptLiteralBudget.test.ts` (+137 -0) 📝 `tests/boundary/mcp/javascriptLiteralPayload.test.ts` (+31 -4) </details> ### 📄 Description ## Summary Direct/cooked string literals could bypass the documented primitive-value byte estimate. Normalize them and configuration defaults through the shared primitive owner, preserve limiting values through member/destructuring projection, and record encountered bounds independently of later selected values. Fixes #3933. Part of #4027 and decision #4091. ## Problem and expected behavior A 200,000-unit normalized primitive must obey the existing 1 MiB worst-case JSON estimate. Retain original source/digests and exact bounded values. A selected healthy sibling can remain exactly `7` while whole-file coverage records its discarded sibling's limit. ## Change and scope - Validate primitive singletons without Set/sort/canonical-key allocation. - Record encountered classifications in per-analysis state; remove the recursive retained-value scan. - Preserve the shared slot owner's tagged unknown through projection. - Normalize configuration defaults once, remove the duplicate raw reader, preserve known null/false/0/empty, and project `value_resource_limit` with an anchored typed unknown. - Preserve full endpoint lookup identities for query/trace/comparison. The same large string is now tested as both a bounded primitive and an exact endpoint. - Keep aggregate default-wire capacity with 16 independently admitted values and at least the former total payload. Shared domain/application owners serve CLI/MCP. Inputs and advertised schemas are unchanged. The optional configuration limiting fact is projected into existing generic graph/coverage records; no worker protocol variant, dependency or runtime capability is added. This static-analysis correction requires no new maintainer scope decision. Ignored catalogs/skills are regenerated and checked. Expected existing effects: - [x] Read existing files or captured data. - [x] Launch an owned REA analysis worker. - [x] Write private staged source files. ## Evidence and regression coverage Original main reproduced both public direct-literal failures. Review head `40b9d3c5` reproduced projection/default gaps (8 failed/25 passed); those valid findings are fixed. Exact boundary arithmetic is corrected: 174,763 units estimate 1,048,580 bytes. An attempted extension at `5808872` applied the primitive policy to endpoint identities. Although its new local tests and real Obsidian control passed, existing CI endpoint acceptance failed (1 failed/1,573 passed/19 skipped in shard 4). That was our compatibility regression. The extension is reverted; the original large endpoint input and its full lookup/feature-trace/semantic-trace/version-comparison assertions remain, now alongside the identical string as a bounded primitive. The observation behind that review was real, but the requested policy does not apply to this separate identity representation. Final production `src` is byte-identical to validated `38d263a`. Existing 66 focused and 932 semantic passes apply to that state. Final corrected public selection passed **7/7** in two files, adding one strengthened acceptance workflow: **67 distinct focused cases across invocations**, not one 67-case run. Prior/rejected attempts are preserved in the verification records. Real Obsidian 1.14.4, selected package.json/main.js: fresh final run preserves exact baseline parent/view Evidence, 23 facts, source coverage/unknowns and CLI/MCP parity. Complete verifier before/final: **33.394/32.949 s wall**, **33.017/32.559 s CPU**, **1,979,863,040/1,880,096,768 B peak cgroup memory**. One CPU/8 GiB, clean scope/no OOM. Uncontrolled caches/scheduling and one control pair establish no speedup or equivalence bound. Static reference-task BPE stays **16,548** with selected name/description/inputSchema contracts (**204,093** with the full catalog projection). This is a control, not a new token-saving claim; annotations/output schemas, client replay/rendering, reasoning and hosted-model use are unmeasured. [Reproduction, source hashes, measurements and rejected attempts](https://github.com/morluto/rea/blob/edaeb247540291695ebdfdec955cd0c671819134/docs/verification/javascript-literal-byte-budget-2026-10-15.md). - [x] Observed, derived and inferred claims are distinguishable. - [x] Artifact identity, source provenance and failed attempts are retained. - [x] Partial, unsupported and unknown outcomes remain visible. ## Validation performed - Focused/public and semantic tests as described above; original aggregate-wire and large-endpoint capacities retained. - `npm run check`: passed typecheck, lint, format, knip, module boundaries and architecture guards. - `npm run docs:check`: passed, including five documentation consistency tests. - Fresh real Obsidian verifier, three rounds, CLI parity and tiktoken 0.12.0: passed. - `git diff --check`: passed. Local verification is Linux x64, Node 24.18.0/npm 11.16.0, static shipped ASAR. Live Electron and other-platform runtime behavior were not exercised locally. Final head `edaeb247` passed 18 checks; 15 lanes were conditionally skipped. The merged source/built test report has 15,614 passed and 139 skipped. Both automated reviews completed on this head with no new findings or unresolved threads. Merged as `9f8659e0536e4e50a438f5b9d5661f3c9624f6ac`; #3933 is closed. ## Author readiness Human owner: @N0zoM1z0 The session authorized implementation, validation, ready PRs and merging, including waiving manual declarations. The agent reviewed the diff/evidence. A human final-diff review is not asserted; declarations remain unchecked. - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [x] Concrete problem, scope, authority, related issue and conventional title identified. - [x] Tests preserve limiting outcomes, precision, source evidence, caller choices and aggregate capacity. - [x] Owning docs/contracts updated; generated artifacts checked and excluded from commits. - [x] Final diff checked for secrets, unrelated changes and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 01:11:41 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6080
No description provided.