morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4192] [Bug] PE input byte-budget refusals report memory constraints #1643

New issue

Closed

opened 2026-10-16 21:08:21 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-16 21:08:21 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4192

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-pe-resources rejects a valid PE image above the caller's max_file_bytes with details.resource: "memory" and advice to review worker/host memory. The limit actually compares the input file's byte length. Both compiled CLI and production stdio MCP reproduced this for the PE32 and PE32+ fixtures below. Raising only the input byte budget returned complete inventories of unchanged images.

Reproduction

From the built source checkout below, create two standalone 1536-byte static PE resource images. Each has one type 10, name 1, language 1033 resource with payload DATA. No selected image is executed.

fixture_root="$(mktemp -d)"
python3 - "$fixture_root" <<'PY'
from pathlib import Path
import struct
import sys

def make_pe(plus=False):
    b = bytearray(1536)
    def u16(at, value): struct.pack_into('<H', b, at, value)
    def u32(at, value): struct.pack_into('<I', b, at, value)
    b[0:2] = b'MZ'
    u32(60, 0x80)
    b[0x80:0x84] = b'PE\0\0'
    u16(0x84, 0x8664 if plus else 0x14c)
    u16(0x86, 1)
    u16(0x94, 240 if plus else 224)
    u16(0x96, 0x2022 if plus else 0x2102)
    o = 0x98
    u16(o, 0x20b if plus else 0x10b)
    u32(o + 8, 512)
    if plus: struct.pack_into('<Q', b, o + 24, 0x180000000)
    else: u32(o + 28, 0x10000000)
    u32(o + 32, 4096)
    u32(o + 36, 512)
    u16(o + 40, 6)
    u16(o + 48, 6)
    u32(o + 56, 0x3000)
    u32(o + 60, 1024)
    u16(o + 68, 3)
    u32(o + (108 if plus else 92), 16)
    d = o + (112 if plus else 96) + 16
    u32(d, 0x2000)
    u32(d + 4, 92)
    s = o + (240 if plus else 224)
    b[s:s+5] = b'.rsrc'
    for at, value in [(8,92),(12,0x2000),(16,512),(20,1024),(36,0x40000040)]:
        u32(s+at,value)
    r = 1024
    for offset, identity, child in [(0,10,0x80000018),(24,1,0x80000030),(48,1033,72)]:
        u16(r+offset+14,1)
        u32(r+offset+16,identity)
        u32(r+offset+20,child)
    u32(r+72,0x2058)
    u32(r+76,4)
    b[r+88:r+92]=b'DATA'
    return bytes(b)

root = Path(sys.argv[1])
(root / 'resource32.dll').write_bytes(make_pe())
(root / 'resource64.dll').write_bytes(make_pe(True))
PY

Run the insufficient-budget cases and sufficient-budget controls:

for image in resource32.dll resource64.dll; do
  node scripts/rea.mjs inspect-pe-resources "$fixture_root/$image" --max-file-bytes 64 --json
  node scripts/rea.mjs inspect-pe-resources "$fixture_root/$image" --max-file-bytes 1536 --json
done
node scripts/rea.mjs inspect-pe-resources "$fixture_root/resource32.dll" --max-file-bytes 1536 --max-entries 1 --json

Both 64-byte calls exit 1 with the memory classification below. Both 1536-byte calls exit 0 with complete one-resource inventories. The last command is a separate entry-budget control and exits 1 with PE resource directory entry budget exceeded; no partial inventory was returned. The value 64 is the supported minimum file-byte budget. This exact fixture and these CLI cases were replayed independently with unchanged input hashes.

Production stdio MCP calls to inspect_pe_resources with each absolute fixture path and max_file_bytes: 64 returned isError: true and the same memory classification. Changing only max_file_bytes to 1536 returned complete Evidence for each image. A subsequent ping succeeded and client/transport close completed.

Expected behavior

Refuse an input above the caller-selected byte budget as a file-size resource constraint, retain the selected limit, and give recovery guidance about the input byte allowance. Preserve the no-partial-success refusal. A sufficient budget should still return complete inventories of these unchanged images.

This expectation follows from the PE input schema describing max_file_bytes as maximum stable artifact bytes to read, the documented separate input, entry and metadata budgets, and the shared taxonomy distinguishing file-size from memory. This is a source-derived interpretation of those contracts; the PE documentation does not contain an explicit sample file-size error.

Impact and scope

P2: both public interfaces give callers the wrong machine-readable resource kind and recovery advice for this bounded input-byte refusal. The requested byte bound still holds; no partial inventory or input corruption was observed. No downstream incident or prevalence claim is made.

A verified workaround for these valid fixtures is to raise max_file_bytes from 64 to 1536. That preserves complete results and input identity. It applies only to valid inputs within the supported maximum and does not resolve independent entry or metadata allocation limits.

REA version

Tested source revision 62bf09809ef7bb224028916424f8f07652a20f4f, freshly built with npm run build:cached; CLI version output 6.5.0. This identifies a source build, not a reproduced installed-package release.

Runtime and platform

Node.js v24.18.0; npm 11.16.0; macOS arm64. Built-in static PE parser; no active target or external native engine.

Target and artifact context

The self-authored PE32 and PE32+ resource images above, each 1536 bytes with one opaque four-byte resource payload. These are supported static-parser fixtures; Windows loader acceptance is not claimed.

Interface

CLI and MCP

Earliest failure stage

Native analysis or reconstruction, at stable-file byte admission before PE parsing.

Relevant output or evidence

The 64-byte CLI results have these meaningful fields:

{
  "code": "resource_constraint",
  "category": "resource_constraint",
  "remediation": {
    "action": "Review the reported worker memory limits and available host memory. Retry with sufficient memory or a smaller artifact; REA retains tighter inherited limits."
  },
  "details": {
    "operation": "inspect_pe_resources",
    "resource": "memory",
    "reported_limits": { "max_file_bytes": 64, "max_entries": 16384 }
  }
}

The reason begins Selected artifact exceeds 64 bytes: and ends with the selected path. With budget 1536, each format reports {"status":"complete","examined_entries":3,"resources":1}. Checks confirmed all three resource IDs, payload offset 1112, length 4, SHA-256 of DATA, and unchanged input digests after every call. The MCP replay confirmed the same refusal/success pairs.

npm run test:focused -- tests/acceptance/applications/peResources.test.ts passed its one existing compiled CLI/real stdio MCP acceptance test. It checks budget refusal but does not assert this incorrect resource kind.

Typed outcome or recovery expected

Keep the 64-byte refusal without partial output, but report details.resource: "file-size" and guidance naming the caller-selected input byte budget. Keep complete inventory and unchanged identity at budget 1536 for both formats. Preserve separate entry-limit refusal and genuine metadata allocation-limit semantics. Selecting the generic file-size fallback alone would still give advice about worker output writes rather than this input allowance.

Analysis or suspected cause

Source inspection shows the stable reader throws ArtifactReaderFailure("limit", ...) before opening an oversized regular file. The PE provider maps every shared limit cause to AnalysisResourceConstraintError(..., "memory", ...). The error projection exposes that kind unchanged, and recovery presentation chooses the observed memory advice.

Current public main at c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb has unchanged PE provider/parser/layout, stable reader, schema, service, CLI/MCP contract, composition, error mapping, documentation and acceptance test. Applicability to that revision was inspected, not runtime-reproduced. The parser metadata accounting has an independent 8 MiB allocation budget and uses the same failure reason; this report does not request relabeling every limit cause as file size.

Additional context

Related reports have different triggers and closure signals: #1702 concerns the former default entry count, #3970 concerns zero-length resource mapping, and #3538 concerns JavaScript parent-result recovery.

NOT_ENOUGH_INFO: an actual metadata allocation-limit refusal was not executed; that separate limit is source-inspected only. Windows/Linux host behavior, Windows loader acceptance, and installed-package behavior were not verified. No last-known-good or first-bad revision was established.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4192 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `inspect-pe-resources` rejects a valid PE image above the caller's `max_file_bytes` with `details.resource: "memory"` and advice to review worker/host memory. The limit actually compares the input file's byte length. Both compiled CLI and production stdio MCP reproduced this for the PE32 and PE32+ fixtures below. Raising only the input byte budget returned complete inventories of unchanged images. ### Reproduction From the built source checkout below, create two standalone 1536-byte static PE resource images. Each has one type 10, name 1, language 1033 resource with payload `DATA`. No selected image is executed. ```sh fixture_root="$(mktemp -d)" python3 - "$fixture_root" <<'PY' from pathlib import Path import struct import sys def make_pe(plus=False): b = bytearray(1536) def u16(at, value): struct.pack_into('<H', b, at, value) def u32(at, value): struct.pack_into('<I', b, at, value) b[0:2] = b'MZ' u32(60, 0x80) b[0x80:0x84] = b'PE\0\0' u16(0x84, 0x8664 if plus else 0x14c) u16(0x86, 1) u16(0x94, 240 if plus else 224) u16(0x96, 0x2022 if plus else 0x2102) o = 0x98 u16(o, 0x20b if plus else 0x10b) u32(o + 8, 512) if plus: struct.pack_into('<Q', b, o + 24, 0x180000000) else: u32(o + 28, 0x10000000) u32(o + 32, 4096) u32(o + 36, 512) u16(o + 40, 6) u16(o + 48, 6) u32(o + 56, 0x3000) u32(o + 60, 1024) u16(o + 68, 3) u32(o + (108 if plus else 92), 16) d = o + (112 if plus else 96) + 16 u32(d, 0x2000) u32(d + 4, 92) s = o + (240 if plus else 224) b[s:s+5] = b'.rsrc' for at, value in [(8,92),(12,0x2000),(16,512),(20,1024),(36,0x40000040)]: u32(s+at,value) r = 1024 for offset, identity, child in [(0,10,0x80000018),(24,1,0x80000030),(48,1033,72)]: u16(r+offset+14,1) u32(r+offset+16,identity) u32(r+offset+20,child) u32(r+72,0x2058) u32(r+76,4) b[r+88:r+92]=b'DATA' return bytes(b) root = Path(sys.argv[1]) (root / 'resource32.dll').write_bytes(make_pe()) (root / 'resource64.dll').write_bytes(make_pe(True)) PY ``` Run the insufficient-budget cases and sufficient-budget controls: ```sh for image in resource32.dll resource64.dll; do node scripts/rea.mjs inspect-pe-resources "$fixture_root/$image" --max-file-bytes 64 --json node scripts/rea.mjs inspect-pe-resources "$fixture_root/$image" --max-file-bytes 1536 --json done node scripts/rea.mjs inspect-pe-resources "$fixture_root/resource32.dll" --max-file-bytes 1536 --max-entries 1 --json ``` Both 64-byte calls exit 1 with the memory classification below. Both 1536-byte calls exit 0 with complete one-resource inventories. The last command is a separate entry-budget control and exits 1 with `PE resource directory entry budget exceeded; no partial inventory was returned.` The value 64 is the supported minimum file-byte budget. This exact fixture and these CLI cases were replayed independently with unchanged input hashes. Production stdio MCP calls to `inspect_pe_resources` with each absolute fixture path and `max_file_bytes: 64` returned `isError: true` and the same memory classification. Changing only `max_file_bytes` to 1536 returned complete Evidence for each image. A subsequent ping succeeded and client/transport close completed. ### Expected behavior Refuse an input above the caller-selected byte budget as a `file-size` resource constraint, retain the selected limit, and give recovery guidance about the input byte allowance. Preserve the no-partial-success refusal. A sufficient budget should still return complete inventories of these unchanged images. This expectation follows from the [PE input schema](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/domain/native/peResources.ts#L17-L41) describing `max_file_bytes` as maximum stable artifact bytes to read, the [documented separate input, entry and metadata budgets](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/docs/binary-diagnostics.md#L152-L194), and the [shared taxonomy](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/domain/analysisErrorCore.ts#L23-L45) distinguishing `file-size` from `memory`. This is a source-derived interpretation of those contracts; the PE documentation does not contain an explicit sample `file-size` error. ### Impact and scope P2: both public interfaces give callers the wrong machine-readable resource kind and recovery advice for this bounded input-byte refusal. The requested byte bound still holds; no partial inventory or input corruption was observed. No downstream incident or prevalence claim is made. A verified workaround for these valid fixtures is to raise `max_file_bytes` from 64 to 1536. That preserves complete results and input identity. It applies only to valid inputs within the supported maximum and does not resolve independent entry or metadata allocation limits. ### REA version Tested [source revision 62bf09809ef7bb224028916424f8f07652a20f4f](https://github.com/morluto/rea/commit/62bf09809ef7bb224028916424f8f07652a20f4f), freshly built with `npm run build:cached`; CLI version output `6.5.0`. This identifies a source build, not a reproduced installed-package release. ### Runtime and platform Node.js v24.18.0; npm 11.16.0; macOS arm64. Built-in static PE parser; no active target or external native engine. ### Target and artifact context The self-authored PE32 and PE32+ resource images above, each 1536 bytes with one opaque four-byte resource payload. These are supported static-parser fixtures; Windows loader acceptance is not claimed. ### Interface CLI and MCP ### Earliest failure stage Native analysis or reconstruction, at stable-file byte admission before PE parsing. ### Relevant output or evidence The 64-byte CLI results have these meaningful fields: ```json { "code": "resource_constraint", "category": "resource_constraint", "remediation": { "action": "Review the reported worker memory limits and available host memory. Retry with sufficient memory or a smaller artifact; REA retains tighter inherited limits." }, "details": { "operation": "inspect_pe_resources", "resource": "memory", "reported_limits": { "max_file_bytes": 64, "max_entries": 16384 } } } ``` The reason begins `Selected artifact exceeds 64 bytes:` and ends with the selected path. With budget 1536, each format reports `{"status":"complete","examined_entries":3,"resources":1}`. Checks confirmed all three resource IDs, payload offset 1112, length 4, SHA-256 of `DATA`, and unchanged input digests after every call. The MCP replay confirmed the same refusal/success pairs. `npm run test:focused -- tests/acceptance/applications/peResources.test.ts` passed its one existing compiled CLI/real stdio MCP acceptance test. It checks budget refusal but does not assert this incorrect resource kind. ### Typed outcome or recovery expected Keep the 64-byte refusal without partial output, but report `details.resource: "file-size"` and guidance naming the caller-selected input byte budget. Keep complete inventory and unchanged identity at budget 1536 for both formats. Preserve separate entry-limit refusal and genuine metadata allocation-limit semantics. Selecting the generic file-size fallback alone would still give advice about worker output writes rather than this input allowance. ### Analysis or suspected cause Source inspection shows the [stable reader](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/artifacts/readStableArtifact.ts#L55-L72) throws `ArtifactReaderFailure("limit", ...)` before opening an oversized regular file. The [PE provider](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/native/pe/PeResourcesProvider.ts#L30-L87) maps every shared `limit` cause to `AnalysisResourceConstraintError(..., "memory", ...)`. The [error projection](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/domain/analysisErrorProjection.ts#L255-L261) exposes that kind unchanged, and [recovery presentation](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/domain/analysisErrorPresentation.ts#L53-L65) chooses the observed memory advice. Current public main at `c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb` has unchanged PE provider/parser/layout, stable reader, schema, service, CLI/MCP contract, composition, error mapping, documentation and acceptance test. Applicability to that revision was inspected, not runtime-reproduced. The [parser metadata accounting](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/src/native/pe/PeResourceParser.ts#L20-L74) has an independent 8 MiB allocation budget and uses the same failure reason; this report does not request relabeling every `limit` cause as file size. ### Additional context Related reports have different triggers and closure signals: [#1702](https://github.com/morluto/rea/issues/1702) concerns the former default entry count, [#3970](https://github.com/morluto/rea/issues/3970) concerns zero-length resource mapping, and [#3538](https://github.com/morluto/rea/issues/3538) concerns JavaScript parent-result recovery. NOT_ENOUGH_INFO: an actual metadata allocation-limit refusal was not executed; that separate limit is source-inspected only. Windows/Linux host behavior, Windows loader acceptance, and installed-package behavior were not verified. No last-known-good or first-bad revision was established.

bsakaguchi commented

2026-10-16 21:08:22 +00:00

Author

Owner

@Abdooo2235 commented on GitHub (Oct 15, 2026):

I am working on this diagnostic classification. I will distinguish input-byte admission from parser allocation/entry limits, retain the existing cleanup behavior, and verify the resource kind and recovery guidance through CLI and MCP.

@Abdooo2235 commented on GitHub (Oct 15, 2026): I am working on this diagnostic classification. I will distinguish input-byte admission from parser allocation/entry limits, retain the existing cleanup behavior, and verify the resource kind and recovery guidance through CLI and MCP.

bsakaguchi commented

2026-10-16 21:08:22 +00:00

Author

Owner

@Abdooo2235 commented on GitHub (Oct 15, 2026):

@coygeek the tested fix is pushed to Abdooo2235:fix/pe-input-byte-budget (2bd92bb). The baseline regression failed as expected; 29 focused tests, static checks and docs checks passed. GitHub rejected PR creation with an insufficient-permissions error, so no PR exists yet. Compare: https://github.com/morluto/rea/compare/main...Abdooo2235:rea:fix/pe-input-byte-budget

@Abdooo2235 commented on GitHub (Oct 15, 2026): @coygeek the tested fix is pushed to Abdooo2235:fix/pe-input-byte-budget (2bd92bb). The baseline regression failed as expected; 29 focused tests, static checks and docs checks passed. GitHub rejected PR creation with an insufficient-permissions error, so no PR exists yet. Compare: https://github.com/morluto/rea/compare/main...Abdooo2235:rea:fix/pe-input-byte-budget

bsakaguchi commented

2026-10-16 21:08:23 +00:00

Author

Owner

@Henglie commented on GitHub (Oct 16, 2026):

Opened a draft PR: https://github.com/morluto/rea/pull/4290

PeResourcesProvider translated every ArtifactReaderFailure with reason: "limit" into a memory resource constraint, which covered two different limits: the caller's input byte budget from readStableArtifact, and the parser's own 8 MiB metadata and max_entries budgets. The PR records the constrained bound on the failure — the way ElectronFuseReader already does — so the input byte budget is reported as file-size and the entry budget as capacity, with the metadata budget unchanged. Readers that do not declare a boundary are unaffected.

@Henglie commented on GitHub (Oct 16, 2026): Opened a draft PR: https://github.com/morluto/rea/pull/4290 `PeResourcesProvider` translated every `ArtifactReaderFailure` with `reason: "limit"` into a `memory` resource constraint, which covered two different limits: the caller's input byte budget from `readStableArtifact`, and the parser's own 8 MiB metadata and `max_entries` budgets. The PR records the constrained bound on the failure — the way `ElectronFuseReader` already does — so the input byte budget is reported as `file-size` and the entry budget as `capacity`, with the metadata budget unchanged. Readers that do not declare a boundary are unaffected.

bsakaguchi closed this issue

2026-10-17 21:02:37 +00:00

bsakaguchi referenced this issue

2026-10-31 00:54:19 +00:00

[PR #1643] [CLOSED] test(hopper): recognize signal-terminated diagnostic fixtures #4720