morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #4770] [Bug] DOS MZ load-image attestation retains full overlay copies in memory #2480

New issue

Closed

opened 2026-10-17 21:05:26 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-17 21:05:26 +00:00

Owner

Originally created by @morluto on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4770

Before submitting

Searched related open/closed issues (load-image/overlay/memory); no matching report in the completed search. A broader follow-up search timed out.
Uses only authored fixture data and public source references.

Area

Native analysis or decompilation

Observed behavior

DOS MZ inspect_native_load_image captures the entire immutable source and then clones it for relocation comparison, including appended overlay bytes that are outside the load module. The extra clone grows with overlay length although the initialized image is unchanged.

Reproduction

At 3fc007534b, ran a small Node probe against built attestGhidraDosLoadImage with the existing authored fixture's 160-byte image and overlays of 23 and 65,536 bytes. Updated the complete original/modified source digests independently; the mapping and relocation facts stayed fixed. Counted calls to Buffer.from with the exact source Buffer, restoring the function immediately afterward.

source bytes | overlay bytes | additional whole-source clone | result
183          | 23            | 183                           | verified
65696        | 65536         | 65696                         | verified

This executes the production comparison with authored observations. The preceding uncapped snapshot capture is source-derived, not measured by this probe. No large allocation, OOM, target or Ghidra process was run.

Expected behavior

Preserve full original/modified source identity, overlay length, complete mapping digests and relocation evidence while bounding retained data by the format-required image/header bytes and streaming hash buffers. Do not merely cap source size and refuse previously admitted overlays. AGENTS.md performance rules require accounting for allocation before it occurs; docs/ghidra-dos.md says overlays participate in source identity but are excluded from initialized module coverage.

Impact and scope

Observed: avoidable overlay-sized extra clone on two tiny inputs. Source-derived: the initial capture also grows with the complete source, so appended overlay size is an unbounded memory input to this workflow. Actual OOM frequency, production peak memory and real-provider impact: NOT_ENOUGH_INFO; no large-input or real-engine experiment was performed. COM has a format length bound and raw-binary attestation already uses streaming snapshot hashing; this report concerns MZ overlays.

REA version

6.5.0, source 3fc007534b

Runtime and platform

Node 24.18.0, macOS arm64; simulated provider observations only.

Target and artifact context

Authored DOS MZ header/module, same mapped bytes with two overlay sizes.

Interface

Other: production provider comparison; shared CLI/MCP path inspected in source.

Earliest failure stage

Native analysis or reconstruction (snapshot capture and expected-image derivation).

Relevant output or evidence

GhidraLoadImageAttest.ts: DOS calls readTargetSnapshot(undefined, signal).
GhidraClient.ts: absent cap becomes initial.size, passed as expected capture size.
BoundedFileBytes.ts: allocates expected size.
GhidraLoadImageValues.ts: Buffer.from(bytes) clones all source bytes before fixups; source and mapped digests consume that representation.
dosMz.ts: image extent derives from unsigned16 page count, while overlay extent derives from complete file size.

Typed outcome or recovery expected

Complete normal verified/mismatch results with the same source identity and Evidence coverage, bounded retention, exact cancellation reason, completed facts and retained cleanup ownership. Preserve detection of changes in the overlay as well as mapped bytes.

Analysis or suspected cause

The snapshot capture contract exposes a whole Buffer, so the consumer has no distinct streaming identity and bounded image facts. Correct this representation at its owner; removing only the second clone leaves the initial full-source capture. A bounded prefix must still handle a Windows-header signature outside that prefix as unknown/requested rather than absent.

Additional context

#3981 / PRs #4101 and #4156 address relocation-window timing separately. Do not conflate that existing defect with overlay memory ownership. Heavy profiling/stress remains stopped.

Originally created by @morluto on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4770 ### Before submitting - [x] Searched related open/closed issues (load-image/overlay/memory); no matching report in the completed search. A broader follow-up search timed out. - [x] Uses only authored fixture data and public source references. ### Area Native analysis or decompilation ### Observed behavior DOS MZ `inspect_native_load_image` captures the entire immutable source and then clones it for relocation comparison, including appended overlay bytes that are outside the load module. The extra clone grows with overlay length although the initialized image is unchanged. ### Reproduction At 3fc007534bb2104d662b43cc712281f9d1d6a7d3, ran a small Node probe against built `attestGhidraDosLoadImage` with the existing authored fixture's 160-byte image and overlays of 23 and 65,536 bytes. Updated the complete original/modified source digests independently; the mapping and relocation facts stayed fixed. Counted calls to `Buffer.from` with the exact source Buffer, restoring the function immediately afterward. ``` source bytes | overlay bytes | additional whole-source clone | result 183 | 23 | 183 | verified 65696 | 65536 | 65696 | verified ``` This executes the production comparison with authored observations. The preceding uncapped snapshot capture is source-derived, not measured by this probe. No large allocation, OOM, target or Ghidra process was run. ### Expected behavior Preserve full original/modified source identity, overlay length, complete mapping digests and relocation evidence while bounding retained data by the format-required image/header bytes and streaming hash buffers. Do not merely cap source size and refuse previously admitted overlays. AGENTS.md performance rules require accounting for allocation before it occurs; docs/ghidra-dos.md says overlays participate in source identity but are excluded from initialized module coverage. ### Impact and scope Observed: avoidable overlay-sized extra clone on two tiny inputs. Source-derived: the initial capture also grows with the complete source, so appended overlay size is an unbounded memory input to this workflow. Actual OOM frequency, production peak memory and real-provider impact: NOT_ENOUGH_INFO; no large-input or real-engine experiment was performed. COM has a format length bound and raw-binary attestation already uses streaming snapshot hashing; this report concerns MZ overlays. ### REA version 6.5.0, source 3fc007534bb2104d662b43cc712281f9d1d6a7d3 ### Runtime and platform Node 24.18.0, macOS arm64; simulated provider observations only. ### Target and artifact context Authored DOS MZ header/module, same mapped bytes with two overlay sizes. ### Interface Other: production provider comparison; shared CLI/MCP path inspected in source. ### Earliest failure stage Native analysis or reconstruction (snapshot capture and expected-image derivation). ### Relevant output or evidence - [GhidraLoadImageAttest.ts](https://github.com/morluto/rea/blob/3fc007534bb2104d662b43cc712281f9d1d6a7d3/src/ghidra/GhidraLoadImageAttest.ts#L163): DOS calls `readTargetSnapshot(undefined, signal)`. - [GhidraClient.ts](https://github.com/morluto/rea/blob/3fc007534bb2104d662b43cc712281f9d1d6a7d3/src/ghidra/GhidraClient.ts#L387): absent cap becomes `initial.size`, passed as expected capture size. - [BoundedFileBytes.ts](https://github.com/morluto/rea/blob/3fc007534bb2104d662b43cc712281f9d1d6a7d3/src/process/BoundedFileBytes.ts#L49): allocates expected size. - [GhidraLoadImageValues.ts](https://github.com/morluto/rea/blob/3fc007534bb2104d662b43cc712281f9d1d6a7d3/src/ghidra/GhidraLoadImageValues.ts#L41): `Buffer.from(bytes)` clones all source bytes before fixups; source and mapped digests consume that representation. - [dosMz.ts](https://github.com/morluto/rea/blob/3fc007534bb2104d662b43cc712281f9d1d6a7d3/src/domain/dosMz.ts): image extent derives from unsigned16 page count, while overlay extent derives from complete file size. ### Typed outcome or recovery expected Complete normal `verified`/`mismatch` results with the same source identity and Evidence coverage, bounded retention, exact cancellation reason, completed facts and retained cleanup ownership. Preserve detection of changes in the overlay as well as mapped bytes. ### Analysis or suspected cause The snapshot capture contract exposes a whole Buffer, so the consumer has no distinct streaming identity and bounded image facts. Correct this representation at its owner; removing only the second clone leaves the initial full-source capture. A bounded prefix must still handle a Windows-header signature outside that prefix as unknown/requested rather than absent. ### Additional context #3981 / PRs #4101 and #4156 address relocation-window timing separately. Do not conflate that existing defect with overlay memory ownership. Heavy profiling/stress remains stopped.

bsakaguchi

2026-10-17 21:05:26 +00:00

closed this issue
added the
triage

bug
labels

bsakaguchi commented

2026-10-17 21:05:28 +00:00

Author

Owner

@morluto commented on GitHub (Oct 17, 2026):

Fixed on main in 2f9b9c48b8.

The snapshot owner now captures the declared MZ image plus the bounded Windows-pointer header, derives relocation/mapping facts once, and streams appended overlay bytes into both complete source digests. It reads a requested Windows signature separately when it lies beyond that image. Overlay size no longer drives a whole-source capture or modified-image clone. The old whole-Buffer production comparison seam is removed. Completed comparisons, exact cancellation, explicit source refusal reasons and descriptor cleanup ownership survive finalization and retry.

Validation: 119 parser/comparison/MCP checks passed; 40 provider checks passed (82 deselected); 14 file/lifecycle checks passed (74 deselected), including cancellation after confirmed close and failed-close retry with completed MZ facts. Static checks, build, test discovery and docs checks passed (5 README consistency tests). Two scoped source reviews found no blocker. Small allocation controls retained only the 160-byte image and chunks at most 65,536 bytes with a 65,539-byte overlay; source and overlay corruption remain rejected. Distant non-Windows/NE signature controls passed.

Twelve completed source-comparison tasks used real private descriptor reads and the same authored 160-byte image, with overlays 23/65,539 bytes. Full replies were byte-identical at 2199/2180 cl100k_base token estimates. Baseline/candidate warm medians:

Overlay bytes	Elapsed seconds	CPU seconds	OS peak RSS bytes
23	0.061145 /0.060359	0.050996 /0.052581	70,107,136 /70,361,088
65,539	0.061098 /0.061057	0.050879 /0.052639	69,992,448 /70,483,968

First processes were recorded separately from two alternating warm repeats. One Node process/no children: wait4 peak process RSS is the process-family peak; periodic samples were incomplete. These tiny samples establish output parity and no speed or peak-memory improvement. Whole-agent/catalog tokens, full public CLI/MCP performance, real Ghidra, large/maximal MZ inputs, OOM behavior and failure-path costs remain unmeasured. Allocation bounds follow the format fields; this is not a measured large-file memory claim. Heavy profiling/stress remains stopped.

#3981's intermediate relocation-window timing remains separate, with open PRs #4101/#4156. This change does not qualify their real-engine overlap behavior. The broader hydration and codebase audits remain open.

@morluto commented on GitHub (Oct 17, 2026): Fixed on main in 2f9b9c48b81ffb7ecd6240e26f26bd859e11edd3. The snapshot owner now captures the declared MZ image plus the bounded Windows-pointer header, derives relocation/mapping facts once, and streams appended overlay bytes into both complete source digests. It reads a requested Windows signature separately when it lies beyond that image. Overlay size no longer drives a whole-source capture or modified-image clone. The old whole-Buffer production comparison seam is removed. Completed comparisons, exact cancellation, explicit source refusal reasons and descriptor cleanup ownership survive finalization and retry. Validation: 119 parser/comparison/MCP checks passed; 40 provider checks passed (82 deselected); 14 file/lifecycle checks passed (74 deselected), including cancellation after confirmed close and failed-close retry with completed MZ facts. Static checks, build, test discovery and docs checks passed (5 README consistency tests). Two scoped source reviews found no blocker. Small allocation controls retained only the 160-byte image and chunks at most 65,536 bytes with a 65,539-byte overlay; source and overlay corruption remain rejected. Distant non-Windows/NE signature controls passed. Twelve completed source-comparison tasks used real private descriptor reads and the same authored 160-byte image, with overlays 23/65,539 bytes. Full replies were byte-identical at 2199/2180 cl100k_base token estimates. Baseline/candidate warm medians: | Overlay bytes | Elapsed seconds | CPU seconds | OS peak RSS bytes | |---|---|---|---| |23|0.061145 /0.060359|0.050996 /0.052581|70,107,136 /70,361,088| |65,539|0.061098 /0.061057|0.050879 /0.052639|69,992,448 /70,483,968| First processes were recorded separately from two alternating warm repeats. One Node process/no children: wait4 peak process RSS is the process-family peak; periodic samples were incomplete. These tiny samples establish output parity and no speed or peak-memory improvement. Whole-agent/catalog tokens, full public CLI/MCP performance, real Ghidra, large/maximal MZ inputs, OOM behavior and failure-path costs remain unmeasured. Allocation bounds follow the format fields; this is not a measured large-file memory claim. Heavy profiling/stress remains stopped. #3981's intermediate relocation-window timing remains separate, with open PRs #4101/#4156. This change does not qualify their real-engine overlap behavior. The broader hydration and codebase audits remain open.

bsakaguchi referenced this issue

2026-10-31 01:05:14 +00:00

[PR #2480] [MERGED] fix(browser): distinguish SVG links from HTML XLink attributes #5260

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2480

No description provided.

Rows
Columns