morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3574] [Bug] Valid ASAR headers with leading JSON whitespace lose their member inventory #1324

New issue

Closed

opened 2026-10-16 21:04:17 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-16 21:04:17 +00:00

Owner

Originally created by @xu91102 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3574

Before submitting

Searched open and closed issues and pull requests for this exact ASAR-header behavior.
This report contains only self-owned inert fixtures and no credentials or private target data.

Area / interface / earliest failure stage

Evidence, artifacts, or reconstruction; public CLI and stdio MCP; artifact inventory classification.

Observed behavior

A valid ASAR whose serialized header JSON begins with one ASCII space or newline is misclassified as file. inspect-artifact exits 0 but retains only the root occurrence, silently losing its two members. analyze-javascript-application exits 1 with artifact_operation_failed, reason format, detail Artifact inventory classified <fixture>.asar as file, not asar. The same bytes fail through real stdio MCP, including explicit format: asar.

Reproduction actually executed

Current source 1579b6865e8ee87873036a39af3e257cc70d6f21, pinned Node 24.18.0/npm 11.16.0, macOS arm64, locked @electron/asar 4.2.0:

Use official createPackage on an owned directory containing inert main.js and package.json.
Retain the exact member body, prepend one space (or newline) to the original serialized header JSON, and update the Chromium pickle payload/string extents with four-byte alignment.
Official listPackage, getRawHeader and extractFile successfully read the same header and both exact member contents.
Run rea inspect-artifact fixture.asar --json and rea analyze-javascript-application fixture.asar --json, then the MCP analyze_javascript_application with {input_path: fixture, format: auto} and explicit ASAR control.

Both leading-space and leading-newline variants fail, each reproduced on the old discovery revision and fresh current main. Compact official output and internal JSON pretty-printing pass both interfaces; ordinary non-ASAR bytes named .asar are correctly refused. Fresh compiled-runtime and MCP raw output were captured separately. Only the official reader executes format parsing; the inert fixture JavaScript is never executed by REA.

Expected behavior and references

Accept this same supported ASAR JSON-object header and preserve member inventory, content digests, and existing integrity/size controls. JSON permits space, tab, CR and LF before a value (RFC 8259 section 2); the pinned official ASAR reader decodes the pickle string with JSON.parse (upstream reader).

Small evidence / impact / scope

The leading-space carrier is 591 bytes, SHA-256 c02668707519ef04abe45bb65ad2a189b5a1660dc23948cf25fc21f8334d2c68; the unchanged main content is export const answer = 42;\n. Official reader sees /main.js and /package.json; REA reports root_format file and only .. Compact, space, newline and pretty variants retain identical parsed header/member content and unchanged carrier bytes after all calls. This loses supported static inventory and prevents analysis of such a selected archive.

Source-derived cause and proposed checks

The shared inventory signature currently requires prefix[16] === 0x7b immediately after the fixed pickle fields. This is stricter than JSON syntax. Root classification and nested stream inventory consume this shared result. A fix should normalize the signature at this owner and preserve fixed extents, JSON-only whitespace, bounded reads, and upstream complete decode/integrity validation. Nested archives and long whitespace spanning the 8192-byte initial prefix were part of the acceptance scope; the completed draft verification below now covers them.

Limits

Fixtures are reserialized from an official producer archive. Third-party occurrence/frequency and affected shipped apps are NOT VERIFIED. Windows/Linux execution, native engines, real Electron GUI/loading, installed npm-package behavior, naturally occurring OS/network short reads, OS process-family quiescence, controlled cold/warm costs and model tokens are NOT RUN. MCP SDK close resolved and the exact owned server PID disappeared; this is not full family quiescence proof.

Verified draft follow-up

A locally validated draft fix is preserved on xu91102/rea at 69a80c35e. This is a fork commit, not an upstream PR or upstream CI result. Normal fork-to-upstream draft creation has been rejected by GitHub; human final-diff review remains pending.

The final byte package passed independent local Standards/Spec review, 12 related files / 126 tests, npm run check, documentation checks and discovery. Real 14 CLI / 9 stdio MCP calls preserve six official-reader-equivalent carriers, exact source/carrier SHA and eight full normalized CLI/MCP positive equalities; opaque .asar stays refused. Root/nested and >8192-prefix whitespace, non-JSON whitespace, declared JSON extent, budget, integrity, cleanup and other format controls remain covered. Controlled 1/7-byte reads from owned real descriptors and a compiled 2 MiB lazy one-byte stream confirm bounded prefix storage; these controls are not naturally occurring network/OS short-read reproductions.

Four completed public analysis calls before and four after used the same baseline/dependencies and immutable compact/pretty carriers; all full normalized results were equal. Elapsed, CPU and sampled family RSS were recorded, with host caches uncontrolled. No isolated cost or performance improvement is claimed; tokens were not measured.

Originally created by @xu91102 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3574 ## Before submitting - [x] Searched open and closed issues and pull requests for this exact ASAR-header behavior. - [x] This report contains only self-owned inert fixtures and no credentials or private target data. ## Area / interface / earliest failure stage Evidence, artifacts, or reconstruction; public CLI and stdio MCP; artifact inventory classification. ## Observed behavior A valid ASAR whose serialized header JSON begins with one ASCII space or newline is misclassified as `file`. `inspect-artifact` exits 0 but retains only the root occurrence, silently losing its two members. `analyze-javascript-application` exits 1 with `artifact_operation_failed`, reason `format`, detail `Artifact inventory classified <fixture>.asar as file, not asar`. The same bytes fail through real stdio MCP, including explicit `format: asar`. ## Reproduction actually executed Current source `1579b6865e8ee87873036a39af3e257cc70d6f21`, pinned Node 24.18.0/npm 11.16.0, macOS arm64, locked `@electron/asar` 4.2.0: 1. Use official `createPackage` on an owned directory containing inert `main.js` and `package.json`. 2. Retain the exact member body, prepend one space (or newline) to the original serialized header JSON, and update the Chromium pickle payload/string extents with four-byte alignment. 3. Official `listPackage`, `getRawHeader` and `extractFile` successfully read the same header and both exact member contents. 4. Run `rea inspect-artifact fixture.asar --json` and `rea analyze-javascript-application fixture.asar --json`, then the MCP `analyze_javascript_application` with `{input_path: fixture, format: auto}` and explicit ASAR control. Both leading-space and leading-newline variants fail, each reproduced on the old discovery revision and fresh current main. Compact official output and internal JSON pretty-printing pass both interfaces; ordinary non-ASAR bytes named `.asar` are correctly refused. Fresh compiled-runtime and MCP raw output were captured separately. Only the official reader executes format parsing; the inert fixture JavaScript is never executed by REA. ## Expected behavior and references Accept this same supported ASAR JSON-object header and preserve member inventory, content digests, and existing integrity/size controls. JSON permits space, tab, CR and LF before a value ([RFC 8259 section 2](https://www.rfc-editor.org/rfc/rfc8259.txt)); the pinned official ASAR reader decodes the pickle string with `JSON.parse` ([upstream reader](https://github.com/electron/asar/blob/v4.2.0/src/disk.ts)). ## Small evidence / impact / scope The leading-space carrier is 591 bytes, SHA-256 `c02668707519ef04abe45bb65ad2a189b5a1660dc23948cf25fc21f8334d2c68`; the unchanged main content is `export const answer = 42;\n`. Official reader sees `/main.js` and `/package.json`; REA reports root_format `file` and only `.`. Compact, space, newline and pretty variants retain identical parsed header/member content and unchanged carrier bytes after all calls. This loses supported static inventory and prevents analysis of such a selected archive. ## Source-derived cause and proposed checks The shared inventory signature currently requires `prefix[16] === 0x7b` immediately after the fixed pickle fields. This is stricter than JSON syntax. Root classification and nested stream inventory consume this shared result. A fix should normalize the signature at this owner and preserve fixed extents, JSON-only whitespace, bounded reads, and upstream complete decode/integrity validation. Nested archives and long whitespace spanning the 8192-byte initial prefix were part of the acceptance scope; the completed draft verification below now covers them. ## Limits Fixtures are reserialized from an official producer archive. Third-party occurrence/frequency and affected shipped apps are NOT VERIFIED. Windows/Linux execution, native engines, real Electron GUI/loading, installed npm-package behavior, naturally occurring OS/network short reads, OS process-family quiescence, controlled cold/warm costs and model tokens are NOT RUN. MCP SDK close resolved and the exact owned server PID disappeared; this is not full family quiescence proof. ## Verified draft follow-up A locally validated draft fix is preserved on [xu91102/rea at 69a80c35e](https://github.com/xu91102/rea/commit/69a80c35e6a118eede00dbabfe59967506ee39a5). This is a fork commit, **not an upstream PR or upstream CI result**. Normal fork-to-upstream draft creation has been rejected by GitHub; human final-diff review remains pending. The final byte package passed independent local Standards/Spec review, 12 related files / 126 tests, `npm run check`, documentation checks and discovery. Real 14 CLI / 9 stdio MCP calls preserve six official-reader-equivalent carriers, exact source/carrier SHA and eight full normalized CLI/MCP positive equalities; opaque `.asar` stays refused. Root/nested and >8192-prefix whitespace, non-JSON whitespace, declared JSON extent, budget, integrity, cleanup and other format controls remain covered. Controlled 1/7-byte reads from owned real descriptors and a compiled 2 MiB lazy one-byte stream confirm bounded prefix storage; these controls are not naturally occurring network/OS short-read reproductions. Four completed public analysis calls before and four after used the same baseline/dependencies and immutable compact/pretty carriers; all full normalized results were equal. Elapsed, CPU and sampled family RSS were recorded, with host caches uncontrolled. No isolated cost or performance improvement is claimed; tokens were not measured.

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:10:26 +00:00

[PR #1324] [CLOSED] fix(javascript): preserve distinct namespace provenance components #2802

bsakaguchi closed this issue

2026-10-31 00:30:37 +00:00

bsakaguchi commented

2026-10-31 00:30:37 +00:00

Author

Owner

@morluto commented on GitHub (Oct 19, 2026):

Fixed on main in e8da7e1edd075feec0e6beedec93d5645fed8125.

The producer required { at byte 16 and root/nested acquisition only extended for XML. A shared ASAR parser now owns Pickle extents and a bounded streaming signature observation. Classification consumes those facts, scans only SP/HTAB/CR/LF inside jsonBytes, discards leading whitespace, and preserves the existing decode budget. Fixed block storage also avoids retaining arbitrarily large stream backing buffers or one prefix object per tiny chunk. Superseded signature parsing and separate root acquisition logic were removed.

Reproduced on baseline 2e5171e39 using an official @electron/asar 4.2.0 archive: upstream listed/extracted identical members after a 10,000-byte legal prefix; REA CLI rejected it as file. The fixed compiled CLI succeeds and preserves the exact archive SHA-256. Compiled CLI and real stdio MCP regressions cover long/root/nested whitespace, selected-source identity and JSON-looking pickle padding. The nearby short-read counterexample is tracked and fixed as #5188. The hash EOF cancellation check remains covered.

Validation: 24 focused files, 170 tests passed; npm run check, npm run docs:check (including 5 consistency tests), and npm run verify:test-discovery (1,634 files exactly once) passed. Coverage includes malformed whitespace/padding, limits, integrity, descriptor close failures/retry, XML, PE/MZ and FAT/Mach-O consumers. Independent source review found no remaining concrete counterexample; it does not establish real-provider behavior.

Equivalent compact-archive CLI tasks returned identical normalized results and estimated reply tokens (14,370 cl100k tokens). Two repeated-process medians per version: elapsed 1.270 → 1.361 s; process-family CPU 1.353 → 1.370 s; sampled family RSS 523,730,944 → 519,700,480 bytes. These small alternating samples are not a speedup claim, true cold runs, or complete agent-token measurements. The long-whitespace case is a separate red/green because baseline cannot complete it.

Context7 and external DeepWiki were used for navigation; conclusions were checked against the pinned upstream package and RFC 8259's JSON whitespace grammar. The fork draft linked above informed review rather than being cherry-picked; its proposed removal of the final EOF cancellation check was not retained. Thanks to @xu91102 for that draft.

No real Electron process execution, Windows qualification, or naturally occurring OS short-read reproduction was performed. No directly related cleanup remains identified in this slice.

@morluto commented on GitHub (Oct 19, 2026): Fixed on `main` in [e8da7e1edd075feec0e6beedec93d5645fed8125](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125). The producer required `{` at byte 16 and root/nested acquisition only extended for XML. A shared ASAR parser now owns Pickle extents and a bounded streaming signature observation. Classification consumes those facts, scans only SP/HTAB/CR/LF inside `jsonBytes`, discards leading whitespace, and preserves the existing decode budget. Fixed block storage also avoids retaining arbitrarily large stream backing buffers or one prefix object per tiny chunk. Superseded signature parsing and separate root acquisition logic were removed. Reproduced on baseline `2e5171e39` using an official `@electron/asar` 4.2.0 archive: upstream listed/extracted identical members after a 10,000-byte legal prefix; REA CLI rejected it as `file`. The fixed compiled CLI succeeds and preserves the exact archive SHA-256. Compiled CLI and real stdio MCP regressions cover long/root/nested whitespace, selected-source identity and JSON-looking pickle padding. The nearby short-read counterexample is tracked and fixed as #5188. The hash EOF cancellation check remains covered. Validation: 24 focused files, 170 tests passed; `npm run check`, `npm run docs:check` (including 5 consistency tests), and `npm run verify:test-discovery` (1,634 files exactly once) passed. Coverage includes malformed whitespace/padding, limits, integrity, descriptor close failures/retry, XML, PE/MZ and FAT/Mach-O consumers. Independent source review found no remaining concrete counterexample; it does not establish real-provider behavior. Equivalent compact-archive CLI tasks returned identical normalized results and estimated reply tokens (14,370 cl100k tokens). Two repeated-process medians per version: elapsed 1.270 → 1.361 s; process-family CPU 1.353 → 1.370 s; sampled family RSS 523,730,944 → 519,700,480 bytes. These small alternating samples are not a speedup claim, true cold runs, or complete agent-token measurements. The long-whitespace case is a separate red/green because baseline cannot complete it. Context7 and external DeepWiki were used for navigation; conclusions were checked against the pinned upstream package and [RFC 8259's JSON whitespace grammar](https://www.rfc-editor.org/rfc/rfc8259.html#section-2). The fork draft linked above informed review rather than being cherry-picked; its proposed removal of the final EOF cancellation check was not retained. Thanks to @xu91102 for that draft. No real Electron process execution, Windows qualification, or naturally occurring OS short-read reproduction was performed. No directly related cleanup remains identified in this slice.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1324

No description provided.

Rows
Columns