morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8623] [Bug] Valid Zstandard ZIP window-limit refusal is reported as corrupt input #7498

New issue

Open

opened 2026-11-03 00:30:13 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 00:30:13 +00:00

Owner

Originally created by @xu91102 on GitHub (Nov 2, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8623

Before submitting

I searched open and closed issues and related pull requests.
This report contains only authored fixture bytes, with no credentials or private target data.

Area

Evidence, artifacts, or reconstruction.

Observed behavior

Both public inspect-artifact and extract-artifact report a valid method-93 ZIP as corrupt when its Zstandard frame needs a decoder window above REA's fixed 128 MiB allowance:

{"reason":"format","detail":"ZIP archive is malformed: Invalid compressed data"}

The user-facing action is “Artifact is invalid or has changed. Get a fresh copy and try again.” The source is unchanged and independently decompresses to the exact expected bytes. A fresh copy does not address this resource refusal.

Reproduction

On source 8250bf2320ea57aab4e664db4b4d44b8abd2004e, build with npm run build:cached. Store the following valid raw-block Zstandard frame as member.txt in a method-93 ZIP, with matching CRC32 and size (30):

28b52ffd0090f1000052454120696e646570656e64656e74206672616d6520636f6e74726f6c0a

The window descriptor 90 declares 256 MiB, independently of the small raw block. Run:

node scripts/rea.mjs inspect-artifact large-window.zip --json
node scripts/rea.mjs extract-artifact large-window.zip --json

Each exits 1 with the format error above. Independently, Zstandard CLI 1.5.7:

zstd -d --memory=256MB --stdout large-window.zst

returns exactly REA independent frame control\n. With zstd's default 128 MiB limit, the same frame is explicitly refused for its decoder window and the CLI identifies the memory allowance.

Executed controls on the same REA source:

Change only the window descriptor to 50 (1 MiB): inspection and extraction both succeed, and independent zstd yields identical content.
Corrupt the frame magic: independent zstd and both REA operations reject it; REA's format outcome is appropriate there.
All six REA invocations preserve each input ZIP's SHA-256.

Expected behavior

Keep the current decoder-window ceiling. Distinguish this known resource-limit refusal from malformed compressed bytes, and expose actionable recovery that identifies the fixed 128 MiB window allowance. Do not suggest that downloading the unchanged archive again repairs it, or imply a runtime override exists.

Impact and scope

A valid archive is misdiagnosed as corrupt and receives ineffective recovery advice. This is error classification/recovery, not a request to admit larger windows, add codecs, raise resource budgets, or make limits configurable (#6176). Reproduced with one authored frame and two public operations; real-world prevalence and last known good version are NOT_ENOUGH_INFO.

REA version

6.8.0; 8250bf2320ea57aab4e664db4b4d44b8abd2004e.

Runtime and platform

Node 24.22.1, macOS arm64. Independent Zstandard CLI 1.5.7.

Target and artifact context

Authored stored raw-block Zstandard frame, wrapped in a matching method-93 ZIP. No target program execution or external provider.

Interface

Public compiled CLI; stdio MCP, Windows/Linux and published-package checks NOT_RUN in this report.

Earliest failure stage

Artifact decompression error classification and recovery projection.

Analysis or suspected cause

Source-derived: ZipArtifactReader.typedZipFailure includes native ZSTD_error_frameParameter_windowTooLarge in its malformed-data set, while the codec explicitly sets ZSTD_d_windowLogMax: 27. The same native error can represent REA's deliberate decoder constraint rather than damaged input.

Actual production diffs of #7065/#7780/#7985 (codec support) and #8431 (producer EOF settlement) were read. They do not provide this classification/recovery correction; the existing codec regression explicitly expects a format fault for an excessive-window frame. Related ELF/GPU decoder call sites were inspected as siblings, but their public behavior was not executed and is not claimed as another reproduced defect. No foreign head was run.

2026-11-02 contribution status

During final pre-publication duplicate checking I read the full actual two-file diff of #8625 (SahinurDEV, head 2412f890d726ff3cde0c8cfaacfdeed075a77422). Its shared ZIP native-error classification fixes the same root cause, so I have stopped my independent local experiment and will not submit a duplicate PR. My source remains uncommitted and read-only. My local tests and installed CLI/MCP checks apply only to that local experiment; I have not run the foreign head, and do not claim it adopted my code or that its remaining validation is complete.

Originally created by @xu91102 on GitHub (Nov 2, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8623 ### Before submitting - [x] I searched open and closed issues and related pull requests. - [x] This report contains only authored fixture bytes, with no credentials or private target data. ### Area Evidence, artifacts, or reconstruction. ### Observed behavior Both public `inspect-artifact` and `extract-artifact` report a valid method-93 ZIP as corrupt when its Zstandard frame needs a decoder window above REA's fixed 128 MiB allowance: ```json {"reason":"format","detail":"ZIP archive is malformed: Invalid compressed data"} ``` The user-facing action is “Artifact is invalid or has changed. Get a fresh copy and try again.” The source is unchanged and independently decompresses to the exact expected bytes. A fresh copy does not address this resource refusal. ### Reproduction On source `8250bf2320ea57aab4e664db4b4d44b8abd2004e`, build with `npm run build:cached`. Store the following valid raw-block Zstandard frame as `member.txt` in a method-93 ZIP, with matching CRC32 and size (30): ``` 28b52ffd0090f1000052454120696e646570656e64656e74206672616d6520636f6e74726f6c0a ``` The window descriptor `90` declares 256 MiB, independently of the small raw block. Run: ```sh node scripts/rea.mjs inspect-artifact large-window.zip --json node scripts/rea.mjs extract-artifact large-window.zip --json ``` Each exits 1 with the format error above. Independently, Zstandard CLI 1.5.7: ```sh zstd -d --memory=256MB --stdout large-window.zst ``` returns exactly `REA independent frame control\n`. With zstd's default 128 MiB limit, the same frame is explicitly refused for its decoder window and the CLI identifies the memory allowance. Executed controls on the same REA source: - Change only the window descriptor to `50` (1 MiB): inspection and extraction both succeed, and independent zstd yields identical content. - Corrupt the frame magic: independent zstd and both REA operations reject it; REA's `format` outcome is appropriate there. - All six REA invocations preserve each input ZIP's SHA-256. ### Expected behavior Keep the current decoder-window ceiling. Distinguish this known resource-limit refusal from malformed compressed bytes, and expose actionable recovery that identifies the fixed 128 MiB window allowance. Do not suggest that downloading the unchanged archive again repairs it, or imply a runtime override exists. ### Impact and scope A valid archive is misdiagnosed as corrupt and receives ineffective recovery advice. This is error classification/recovery, not a request to admit larger windows, add codecs, raise resource budgets, or make limits configurable (#6176). Reproduced with one authored frame and two public operations; real-world prevalence and last known good version are NOT_ENOUGH_INFO. ### REA version 6.8.0; `8250bf2320ea57aab4e664db4b4d44b8abd2004e`. ### Runtime and platform Node 24.22.1, macOS arm64. Independent Zstandard CLI 1.5.7. ### Target and artifact context Authored stored raw-block Zstandard frame, wrapped in a matching method-93 ZIP. No target program execution or external provider. ### Interface Public compiled CLI; stdio MCP, Windows/Linux and published-package checks NOT_RUN in this report. ### Earliest failure stage Artifact decompression error classification and recovery projection. ### Analysis or suspected cause Source-derived: `ZipArtifactReader.typedZipFailure` includes native `ZSTD_error_frameParameter_windowTooLarge` in its malformed-data set, while the codec explicitly sets `ZSTD_d_windowLogMax: 27`. The same native error can represent REA's deliberate decoder constraint rather than damaged input. Actual production diffs of #7065/#7780/#7985 (codec support) and #8431 (producer EOF settlement) were read. They do not provide this classification/recovery correction; the existing codec regression explicitly expects a format fault for an excessive-window frame. Related ELF/GPU decoder call sites were inspected as siblings, but their public behavior was not executed and is not claimed as another reproduced defect. No foreign head was run. ### 2026-11-02 contribution status During final pre-publication duplicate checking I read the full actual two-file diff of #8625 (SahinurDEV, head `2412f890d726ff3cde0c8cfaacfdeed075a77422`). Its shared ZIP native-error classification fixes the same root cause, so I have stopped my independent local experiment and will not submit a duplicate PR. My source remains uncommitted and read-only. My local tests and installed CLI/MCP checks apply only to that local experiment; I have not run the foreign head, and do not claim it adopted my code or that its remaining validation is complete.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7498

No description provided.

Rows
Columns