morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7234] [MERGED] feat(artifacts): add U-Boot legacy uImage reader (part of #7129) #7734

New issue

Closed

opened 2026-11-03 13:39:59 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-03 13:39:59 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7234
Author: @iamcipherdev
Created: 10/26/2026
Status: ✅ Merged
Merged: 10/28/2026
Merged by: @gensokyo-dev

Base: main ← Head: fix/rea-7129-uimage-2026-10-26

📝 Commits (7)

18f04d8 feat(artifacts): add U-Boot legacy uImage reader (part of #7129)
c6a741b Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup
6461c62 fix(uimage): preserve aligned component ranges and exact header facts
ef03c82 Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup
3838fb2 fix(uimage): retain buffered members through consumption and cancellation
a67f4f0 Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup
ae6aad3 fix(uimage): preserve active read and validation failure precedence

📊 Changes

25 files changed (+1744 additions, -17 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+7 -0)
📝 docs/cli.md (+10 -0)
📝 docs/firmware-analysis.md (+14 -0)
📝 docs/index.md (+14 -14)
➕ docs/uimage.md (+72 -0)
📝 src/application/BinaryTargetResolver.ts (+12 -3)
📝 src/artifacts/ArtifactReader.ts (+1 -0)
➕ src/artifacts/UImageArtifactReader.boundaries.test.ts (+363 -0)
➕ src/artifacts/UImageArtifactReader.test.ts (+313 -0)
➕ src/artifacts/UImageArtifactReader.ts (+358 -0)
📝 src/artifacts/createArtifactReader.ts (+4 -0)
📝 src/artifacts/extraction/ArtifactExtraction.ts (+1 -0)
📝 src/artifacts/inventory/ArtifactGraphConstruction.ts (+4 -0)
📝 src/artifacts/inventory/classify.ts (+1 -0)
📝 src/artifacts/inventory/reader.ts (+6 -0)
📝 src/domain/analysisSnapshot.ts (+1 -0)
📝 src/domain/artifactFormatMetadata.ts (+39 -0)
📝 src/domain/artifactGraph.ts (+2 -0)
📝 src/domain/binaryTargetTypes.ts (+1 -0)
📝 src/domain/evidenceSubject.ts (+1 -0)

...and 5 more files

📄 Description

Problem and outcome

Legacy uImages now open as archives through the existing CLI/MCP inspection and extraction workflows. The prototype ignored nonfinal component padding: a real mkimage image containing a 267-byte member made the next extracted component start one byte early despite a valid CRC. The shared parser now preserves aligned physical ranges, correct numeric CPU/OS meanings, unknowns and raw name bytes.

Related to #7129 (uImage only); HEX/S-record merged in #7136. FIT remains separate.

Approach

A pure Result-based header/table parser owns format facts and typed format/capacity errors. Shared reader dispatch serves CLI/MCP; obsolete dispatch, fixture paths and custom bytewise CRC code are removed.
Native incremental CRC32 uses bounded chunks. The 4096-component capacity is admitted before scanning the remaining payload; the admitted prefix is reused in that CRC pass. Whole-carrier identity hashing has separate cost.
Invalid UTF-8 names remain null with their original 32 bytes retained. Header architecture/OS/type/address declarations do not establish executable architecture or boot compatibility. Compressed content stays compressed.
Stable entry identity and physical ranges support existing selected-field, paged and batch-ID views with retained complete Evidence. Output ownership and cancellation extend through consumption, final validation and descriptor cleanup retry.

Executed validation

32/32 focused tests pass: reader boundaries, compiled CLI, full/compact MCP and ZIP/ASAR coexistence. Two fully buffered consumption regressions fail before the lifecycle fix and pass after it. A further regression proves cancellation does not mask a pending native final-validation failure; active producer errors retain precedence. Gate tests acknowledge the actual read/final-validation boundary and release/join launched work. check:fast, docs:check, formatting and bounded single-thread typed lint passed; documentation checks preceded the final lifecycle-only fix.

Independent mkimage/dumpimage 2023.01 plus streamed Python CRC/SHA checks verify five inputs: official OpenWrt 24.10.0 GL-AR150 firmware, a multi-component real ELF/system-file image, stored gzip, a truncated UTF-8 name, and a 126,879,696-byte Node ELF payload. CLI and actual MCP inspection/extraction match every expected stored member hash. Source hashes, cleanup receipts and removal of owned output/workspaces verify. OpenWrt SHA-256: 027c60a7061462d8145b11efa23ab02b334acb9a23a215f9095860b632ace4ff; format definitions pinned to U-Boot ece349ade2973e220f524ce59e59711cc919263f.

Completed task costs

Final ae6aad366 versus prototype 103986af1, both on main df7119aee: 12 alternating inspect/extract/SHA/cleanup tasks, three fresh-process pairs per input, no concurrent local builds/tests. First/repeat runs do not imply dropped OS caches.

Input	Elapsed before → after	CPU before → after	Sampled family RSS before → after
OpenWrt, 6.24 MB	5.62–6.32 → 5.16–5.46 s	7.57–8.13 → 7.10–7.51 s	321–328 → 309–327 MiB
Stored ELF, 126.9 MB	13.49–13.57 → 10.36–10.90 s	21.19–21.59 → 17.69–18.48 s	395–400 → 358–368 MiB

CPU includes wait4 plus verifier CPU; family RSS sampled every 10 ms. Shared-host results do not establish a general speedup. A completed five-file MCP session including SDK client/server, catalog, selected views, extraction, hashing and cleanup measured 10.55 s elapsed, 19.64 s CPU, 595.5 MiB family RSS. Node 24.22.1 heaps capped at 2 GiB; tests use two workers.

CLI request+stdout tokenx estimates: OpenWrt 3068→3086, Node 2850→2869; the added raw name fact explains the small increase. Whole five-file MCP task: 97,378 estimated tokens including 50,942 catalog tokens and text/structured duplication. These are estimates, not billing counts; fixture generation and offline token estimation costs were not profiled.

No target was booted. FIT, decompression, runtime/provider behavior and other hosts remain unverified.

Author readiness

Human owner: @iamcipherdev

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7234 **Author:** [@iamcipherdev](https://github.com/iamcipherdev) **Created:** 10/26/2026 **Status:** ✅ Merged **Merged:** 10/28/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `fix/rea-7129-uimage-2026-10-26` --- ### 📝 Commits (7) - [`18f04d8`](https://github.com/morluto/rea/commit/18f04d8edbb9976e5913d04b13b2020ce8eccc83) feat(artifacts): add U-Boot legacy uImage reader (part of #7129) - [`c6a741b`](https://github.com/morluto/rea/commit/c6a741b60bd0fabab0e52cc9bb8ff74164abb017) Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup - [`6461c62`](https://github.com/morluto/rea/commit/6461c621e6bcd4f2369edd12f87172225433ce8a) fix(uimage): preserve aligned component ranges and exact header facts - [`ef03c82`](https://github.com/morluto/rea/commit/ef03c82a0b07bd3a7c58237b8f9fbb3af9bdb945) Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup - [`3838fb2`](https://github.com/morluto/rea/commit/3838fb214a24e2fbe35984cd47b1b6bbffb9d4a0) fix(uimage): retain buffered members through consumption and cancellation - [`a67f4f0`](https://github.com/morluto/rea/commit/a67f4f0d62281f77c9c1949fd8d2947b5003029b) Merge remote-tracking branch 'origin/main' into fix/7129-uimage-followup - [`ae6aad3`](https://github.com/morluto/rea/commit/ae6aad366073ffd99559918eae825e675414d484) fix(uimage): preserve active read and validation failure precedence ### 📊 Changes **25 files changed** (+1744 additions, -17 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+7 -0) 📝 `docs/cli.md` (+10 -0) 📝 `docs/firmware-analysis.md` (+14 -0) 📝 `docs/index.md` (+14 -14) ➕ `docs/uimage.md` (+72 -0) 📝 `src/application/BinaryTargetResolver.ts` (+12 -3) 📝 `src/artifacts/ArtifactReader.ts` (+1 -0) ➕ `src/artifacts/UImageArtifactReader.boundaries.test.ts` (+363 -0) ➕ `src/artifacts/UImageArtifactReader.test.ts` (+313 -0) ➕ `src/artifacts/UImageArtifactReader.ts` (+358 -0) 📝 `src/artifacts/createArtifactReader.ts` (+4 -0) 📝 `src/artifacts/extraction/ArtifactExtraction.ts` (+1 -0) 📝 `src/artifacts/inventory/ArtifactGraphConstruction.ts` (+4 -0) 📝 `src/artifacts/inventory/classify.ts` (+1 -0) 📝 `src/artifacts/inventory/reader.ts` (+6 -0) 📝 `src/domain/analysisSnapshot.ts` (+1 -0) 📝 `src/domain/artifactFormatMetadata.ts` (+39 -0) 📝 `src/domain/artifactGraph.ts` (+2 -0) 📝 `src/domain/binaryTargetTypes.ts` (+1 -0) 📝 `src/domain/evidenceSubject.ts` (+1 -0) _...and 5 more files_ </details> ### 📄 Description ## Problem and outcome Legacy uImages now open as archives through the existing CLI/MCP inspection and extraction workflows. The prototype ignored nonfinal component padding: a real mkimage image containing a 267-byte member made the next extracted component start one byte early despite a valid CRC. The shared parser now preserves aligned physical ranges, correct numeric CPU/OS meanings, unknowns and raw name bytes. Related to #7129 (uImage only); HEX/S-record merged in #7136. FIT remains separate. ## Approach - A pure Result-based header/table parser owns format facts and typed format/capacity errors. Shared reader dispatch serves CLI/MCP; obsolete dispatch, fixture paths and custom bytewise CRC code are removed. - Native incremental CRC32 uses bounded chunks. The 4096-component capacity is admitted before scanning the remaining payload; the admitted prefix is reused in that CRC pass. Whole-carrier identity hashing has separate cost. - Invalid UTF-8 names remain null with their original 32 bytes retained. Header architecture/OS/type/address declarations do not establish executable architecture or boot compatibility. Compressed content stays compressed. - Stable entry identity and physical ranges support existing selected-field, paged and batch-ID views with retained complete Evidence. Output ownership and cancellation extend through consumption, final validation and descriptor cleanup retry. ## Executed validation 32/32 focused tests pass: reader boundaries, compiled CLI, full/compact MCP and ZIP/ASAR coexistence. Two fully buffered consumption regressions fail before the lifecycle fix and pass after it. A further regression proves cancellation does not mask a pending native final-validation failure; active producer errors retain precedence. Gate tests acknowledge the actual read/final-validation boundary and release/join launched work. `check:fast`, `docs:check`, formatting and bounded single-thread typed lint passed; documentation checks preceded the final lifecycle-only fix. Independent mkimage/dumpimage 2023.01 plus streamed Python CRC/SHA checks verify five inputs: official OpenWrt 24.10.0 GL-AR150 firmware, a multi-component real ELF/system-file image, stored gzip, a truncated UTF-8 name, and a 126,879,696-byte Node ELF payload. CLI and actual MCP inspection/extraction match every expected stored member hash. Source hashes, cleanup receipts and removal of owned output/workspaces verify. OpenWrt SHA-256: `027c60a7061462d8145b11efa23ab02b334acb9a23a215f9095860b632ace4ff`; format definitions pinned to U-Boot `ece349ade2973e220f524ce59e59711cc919263f`. ## Completed task costs Final `ae6aad366` versus prototype `103986af1`, both on main `df7119aee`: 12 alternating inspect/extract/SHA/cleanup tasks, three fresh-process pairs per input, no concurrent local builds/tests. First/repeat runs do not imply dropped OS caches. | Input | Elapsed before → after | CPU before → after | Sampled family RSS before → after | |---|---|---|---| | OpenWrt, 6.24 MB | 5.62–6.32 → 5.16–5.46 s | 7.57–8.13 → 7.10–7.51 s | 321–328 → 309–327 MiB | | Stored ELF, 126.9 MB | 13.49–13.57 → 10.36–10.90 s | 21.19–21.59 → 17.69–18.48 s | 395–400 → 358–368 MiB | CPU includes wait4 plus verifier CPU; family RSS sampled every 10 ms. Shared-host results do not establish a general speedup. A completed five-file MCP session including SDK client/server, catalog, selected views, extraction, hashing and cleanup measured 10.55 s elapsed, 19.64 s CPU, 595.5 MiB family RSS. Node 24.22.1 heaps capped at 2 GiB; tests use two workers. CLI request+stdout tokenx estimates: OpenWrt 3068→3086, Node 2850→2869; the added raw name fact explains the small increase. Whole five-file MCP task: 97,378 estimated tokens including 50,942 catalog tokens and text/structured duplication. These are estimates, not billing counts; fixture generation and offline token estimation costs were not profiled. No target was booted. FIT, decompression, runtime/provider behavior and other hosts remain unverified. ## Author readiness Human owner: @iamcipherdev - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-03 13:39:59 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7734

No description provided.

Rows
Columns