[GH-ISSUE #7129] [Feature] Static readers for firmware containers: Intel HEX/S-record regions, U-Boot uImage headers, and FIT images

bsakaguchi commented

2026-10-31 00:48:27 +00:00

Owner

Originally created by @555aaditya on GitHub (Oct 25, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7129

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

An analyst with an MCU programming file (.hex, .ihx, .s19, .s28, .s37, .mot) or a U-Boot boot image wants to know which bytes load at which addresses, what the declared entry point is, and which payloads the file contains. REA cannot answer that today:

open_binary refuses Intel HEX and S-record text as an unsupported format, so inspect_artifact and extract_artifact are never reached.
inspect_firmware_regions passes Binwalk description text through, and it requires Binwalk on Linux.
inspect_devicetree (#7009) reads standalone DTBs, but it does not inventory FIT sub-images, and nothing parses a legacy uImage header.

The analyst has to convert files with srec_cat, objcopy or mkimage outside REA, which loses address provenance.

Use cases

A Cortex-M, AVR or MSP430 vendor update shipped as Intel HEX, with several sparse regions and a start linear address record.
S-record output from automotive or ECU toolchains: S3 records plus S7, with an S0 header naming the build.
Router or single-board-computer firmware with a legacy uImage (a 64-byte header with architecture, compression, load and entry addresses, and CRCs), or a FIT image carrying kernel, ramdisk and DTB sub-images with per-image hashes.

Desired outcome and success criteria

Three separate, purely static PRs through the existing inspect_artifact and extract_artifact contracts. No new tools, no external dependencies, no execution.

Intel HEX (I8/I16/I32) and Motorola S-record (S19/S28/S37).
- Address-contiguous regions become extractable entries, named by address range.
- Root facts record the variant, the start address (records 03/05 and S7/S8/S9), the S0 header and record counts.
- Checksum errors, unknown record types, conflicting overlaps and count mismatches fail closed and name the line.
- A missing EOF or termination record leaves coverage unknown. Gaps are never zero-filled.
U-Boot legacy uImage.
- Header facts: magic, ih_arch/ih_os/ih_type/ih_comp (raw and named), ih_load, ih_ep, name and time, with the header and data CRC32 verified.
- The payload can be extracted as stored bytes, including multi-file images.
- Compressed payloads stay as stored, with ih_comp reported.
- These would be the single owner of the header facts that #4245's CPU-candidate operation consumes, rather than a second parser.
FIT images (FDT-based).
- Reuse DevicetreeParser from #7009 to inventory the /images/* sub-images and /configurations.
- Extract embedded data (and external data-offset/data-size).
- Record declared hashes, and check them when Node's standard library supports the algorithm.
- Report signatures without verifying them.

Each PR states its limits and why they exist, reports declared addresses as facts rather than proven entry points, and includes CLI and stdio-MCP tests with synthetic fixtures.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

These are format inspection and ordinary extraction, which the contribution scope lists as supported. They add no target execution, injection, mutation, new provider or protection recovery, so I don't think a separate scope decision is needed. Please say if you disagree.

Non-goals:

CPU inference (#4245).
Changes to raw-binary loading (#4246, #5744).
Decompressing LZMA, LZ4 or zstd payloads.
Signature verification.

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

Synthetic fixtures with computed checksums for each record type and failure mode. Optional producer goldens from GNU objcopy or srec_cat (HEX/SREC) and mkimage (uImage/FIT), recorded with versions. I won't make claims about real devices.

Alternatives considered

inspect_firmware_regions and extract_firmware (Binwalk/Unblob) are Linux-only external tools that report descriptions, not structured addresses, and they don't handle HEX/SREC text.
inspect_devicetree covers DTBs, but not FIT payload inventory or extraction.
Converting files outside REA loses provenance.

Possible direction

Follow NodeSeaArtifactReader (#7000): recognise formats by content in BinaryTargetResolver.detectArtifactFormat and classifyArtifactBytes, and keep facts in format_metadata.
Extracted regions or payloads can then be opened with open_binary using format=raw-binary and the region start address as entry_point, under today's contract where base and entry are the same.
A better hand-off depends on #4246.

I'll start with the Intel HEX / S-record PR.

References

Related: #3615, #4245, #4246, #5744, #7009, #7000, #3634; #718 (closed roadmap).
Docs: docs/firmware-analysis.md, docs/raw-binary.md.
Formats: Intel Hexadecimal Object File Format Specification Rev A; Motorola S-record (M68000 PRM appendix C and srec_motorola(5)); U-Boot include/image.h and FIT doc/usage/fit.

Originally created by @555aaditya on GitHub (Oct 25, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7129 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation An analyst with an MCU programming file (`.hex`, `.ihx`, `.s19`, `.s28`, `.s37`, `.mot`) or a U-Boot boot image wants to know which bytes load at which addresses, what the declared entry point is, and which payloads the file contains. REA cannot answer that today: - `open_binary` refuses Intel HEX and S-record text as an unsupported format, so `inspect_artifact` and `extract_artifact` are never reached. - `inspect_firmware_regions` passes Binwalk description text through, and it requires Binwalk on Linux. - `inspect_devicetree` (#7009) reads standalone DTBs, but it does not inventory FIT sub-images, and nothing parses a legacy uImage header. The analyst has to convert files with srec_cat, objcopy or mkimage outside REA, which loses address provenance. ### Use cases - A Cortex-M, AVR or MSP430 vendor update shipped as Intel HEX, with several sparse regions and a start linear address record. - S-record output from automotive or ECU toolchains: S3 records plus S7, with an S0 header naming the build. - Router or single-board-computer firmware with a legacy uImage (a 64-byte header with architecture, compression, load and entry addresses, and CRCs), or a FIT image carrying kernel, ramdisk and DTB sub-images with per-image hashes. ### Desired outcome and success criteria Three separate, purely static PRs through the existing `inspect_artifact` and `extract_artifact` contracts. No new tools, no external dependencies, no execution. 1. **Intel HEX (I8/I16/I32) and Motorola S-record (S19/S28/S37).** - Address-contiguous regions become extractable entries, named by address range. - Root facts record the variant, the start address (records 03/05 and S7/S8/S9), the S0 header and record counts. - Checksum errors, unknown record types, conflicting overlaps and count mismatches fail closed and name the line. - A missing EOF or termination record leaves coverage unknown. Gaps are never zero-filled. 2. **U-Boot legacy uImage.** - Header facts: magic, `ih_arch`/`ih_os`/`ih_type`/`ih_comp` (raw and named), `ih_load`, `ih_ep`, name and time, with the header and data CRC32 verified. - The payload can be extracted as stored bytes, including multi-file images. - Compressed payloads stay as stored, with `ih_comp` reported. - These would be the single owner of the header facts that #4245's CPU-candidate operation consumes, rather than a second parser. 3. **FIT images (FDT-based).** - Reuse `DevicetreeParser` from #7009 to inventory the `/images/*` sub-images and `/configurations`. - Extract embedded `data` (and external `data-offset`/`data-size`). - Record declared hashes, and check them when Node's standard library supports the algorithm. - Report signatures without verifying them. Each PR states its limits and why they exist, reports declared addresses as facts rather than proven entry points, and includes CLI and stdio-MCP tests with synthetic fixtures. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) These are format inspection and ordinary extraction, which the contribution scope lists as supported. They add no target execution, injection, mutation, new provider or protection recovery, so I don't think a separate scope decision is needed. Please say if you disagree. Non-goals: - CPU inference (#4245). - Changes to raw-binary loading (#4246, #5744). - Decompressing LZMA, LZ4 or zstd payloads. - Signature verification. ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) Synthetic fixtures with computed checksums for each record type and failure mode. Optional producer goldens from GNU objcopy or srec_cat (HEX/SREC) and `mkimage` (uImage/FIT), recorded with versions. I won't make claims about real devices. ### Alternatives considered - `inspect_firmware_regions` and `extract_firmware` (Binwalk/Unblob) are Linux-only external tools that report descriptions, not structured addresses, and they don't handle HEX/SREC text. - `inspect_devicetree` covers DTBs, but not FIT payload inventory or extraction. - Converting files outside REA loses provenance. ### Possible direction - Follow `NodeSeaArtifactReader` (#7000): recognise formats by content in `BinaryTargetResolver.detectArtifactFormat` and `classifyArtifactBytes`, and keep facts in `format_metadata`. - Extracted regions or payloads can then be opened with `open_binary` using `format=raw-binary` and the region start address as `entry_point`, under today's contract where base and entry are the same. - A better hand-off depends on #4246. I'll start with the Intel HEX / S-record PR. ### References - Related: #3615, #4245, #4246, #5744, #7009, #7000, #3634; #718 (closed roadmap). - Docs: `docs/firmware-analysis.md`, `docs/raw-binary.md`. - Formats: Intel Hexadecimal Object File Format Specification Rev A; Motorola S-record (M68000 PRM appendix C and `srec_motorola(5)`); U-Boot `include/image.h` and FIT `doc/usage/fit`.

bsakaguchi closed this issue

2026-10-31 00:48:27 +00:00

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 25, 2026):

This would be a solid addition to the firmware lane - Intel HEX and S-record readers are frequently needed when recovering raw firmware from bootloaders, and the uImage/FIT parsers pair naturally with the existing flattened devicetree blob inspection. One suggestion: for HEX/S-record, also surface the start/end linear address so downstream Ghidra import can pick the right load base automatically.

@XinwangTao commented on GitHub (Oct 25, 2026): This would be a solid addition to the firmware lane - Intel HEX and S-record readers are frequently needed when recovering raw firmware from bootloaders, and the uImage/FIT parsers pair naturally with the existing flattened devicetree blob inspection. One suggestion: for HEX/S-record, also surface the start/end linear address so downstream Ghidra import can pick the right load base automatically.

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 25, 2026):

Useful capability — HEX/S-record parsing is a natural static-reader fit since it gives byte-at-address load maps, and uImage/FIT would close the gap for boot-image triage. Suggestions: (1) keep the output shape consistent with existing evidence readers (address ranges -> payload bytes + declared entry point); (2) treat checksum/record-type validation as part of the evidence output rather than silently failing, since malformed records are themselves a finding; (3) for FIT (which is a flattened DTB), consider reusing any existing DTB parsing instead of a second parser.

@XinwangTao commented on GitHub (Oct 25, 2026): Useful capability — HEX/S-record parsing is a natural static-reader fit since it gives byte-at-address load maps, and uImage/FIT would close the gap for boot-image triage. Suggestions: (1) keep the output shape consistent with existing evidence readers (address ranges -> payload bytes + declared entry point); (2) treat checksum/record-type validation as part of the evidence output rather than silently failing, since malformed records are themselves a finding; (3) for FIT (which is a flattened DTB), consider reusing any existing DTB parsing instead of a second parser.

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@555aaditya commented on GitHub (Oct 25, 2026):

Thanks @XinwangTao. Taking both comments together:

Address bounds: the Intel HEX/S-record part is in #7136. Each region is an inventory entry with start_address/end_address, named for its range, for example 0x08000000-0x0800001f.bin. The root metadata adds lowest_address/highest_address and the declared start: start_address from 05/S7–S9, or CS:IP from 03. Choosing the Ghidra load base automatically is out of scope here. For now, a region opens with open_binary format=raw-binary, using its start address as entry_point. Raw-binary base selection is tracked in #4246 and #5744.
(1) Output shape: yes. Regions are ordinary artifact entries that extract as raw bytes, and the declared start is a reported fact, not a proven entry point.
(2) Malformed records: failing closed is deliberate, and the failure isn't silent. A bad checksum, an unknown record type or conflicting overlapping data means the address-to-byte map can't be trusted, so returning partial regions as evidence would mislead. The format error names the line and the reason, for example declared versus computed checksum. The one recoverable case, a missing termination record, still returns regions, with coverage unknown and a limitation. If the maintainers want a diagnostic mode that lists every bad record, that could be a follow-up.
(3) FIT: agreed. The FIT reader should reuse the existing parseDevicetree (src/firmware/devicetree/DevicetreeParser.ts) rather than add a second FDT parser. I'll plan the FIT part that way.

@555aaditya commented on GitHub (Oct 25, 2026): Thanks @XinwangTao. Taking both comments together: - **Address bounds:** the Intel HEX/S-record part is in #7136. Each region is an inventory entry with `start_address`/`end_address`, named for its range, for example `0x08000000-0x0800001f.bin`. The root metadata adds `lowest_address`/`highest_address` and the declared start: `start_address` from `05`/`S7`–`S9`, or CS:IP from `03`. Choosing the Ghidra load base automatically is out of scope here. For now, a region opens with `open_binary format=raw-binary`, using its start address as `entry_point`. Raw-binary base selection is tracked in #4246 and #5744. - **(1) Output shape:** yes. Regions are ordinary artifact entries that extract as raw bytes, and the declared start is a reported fact, not a proven entry point. - **(2) Malformed records:** failing closed is deliberate, and the failure isn't silent. A bad checksum, an unknown record type or conflicting overlapping data means the address-to-byte map can't be trusted, so returning partial regions as evidence would mislead. The `format` error names the line and the reason, for example declared versus computed checksum. The one recoverable case, a missing termination record, still returns regions, with coverage `unknown` and a limitation. If the maintainers want a diagnostic mode that lists every bad record, that could be a follow-up. - **(3) FIT:** agreed. The FIT reader should reuse the existing `parseDevicetree` (`src/firmware/devicetree/DevicetreeParser.ts`) rather than add a second FDT parser. I'll plan the FIT part that way.

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@555aaditya commented on GitHub (Oct 28, 2026):

Intel HEX/S-record landed in #7136, thanks @gensokyo-dev. uImage is in progress in #7234 (@iamcipherdev). I'd like to take the remaining FIT part. Plan:

Recognize a FIT as an FDT v16/v17 blob with an /images node. A plain DTB keeps its current inspect_devicetree route.
Parse the structure with the existing parseDevicetree, without a second FDT parser. It already reports each property's file offset and length.
Make each /images/<name> an inventory entry, extracted as its stored bytes. That covers embedded data as well as external data-offset (relative to the end of the FDT) and data-position (absolute) with data-size.
Report the declared type, arch, os, compression, load and entry as facts. Verify hash-N nodes for crc32, sha1, sha256 and md5, and report any other algorithm as unverified. Report /configurations (default plus kernel/fdt/ramdisk references) in root metadata.
Signatures are reported, not verified. Nothing is decompressed or executed.
Fail closed on data outside the file, overlapping or duplicate image data, a missing data, and hash mismatches.

I'll open a draft PR.

@555aaditya commented on GitHub (Oct 28, 2026): Intel HEX/S-record landed in #7136, thanks @gensokyo-dev. uImage is in progress in #7234 (@iamcipherdev). I'd like to take the remaining **FIT** part. Plan: - Recognize a FIT as an FDT v16/v17 blob with an `/images` node. A plain DTB keeps its current `inspect_devicetree` route. - Parse the structure with the existing `parseDevicetree`, without a second FDT parser. It already reports each property's file offset and length. - Make each `/images/<name>` an inventory entry, extracted as its stored bytes. That covers embedded `data` as well as external `data-offset` (relative to the end of the FDT) and `data-position` (absolute) with `data-size`. - Report the declared `type`, `arch`, `os`, `compression`, `load` and `entry` as facts. Verify `hash-N` nodes for `crc32`, `sha1`, `sha256` and `md5`, and report any other algorithm as unverified. Report `/configurations` (default plus kernel/fdt/ramdisk references) in root metadata. - Signatures are reported, not verified. Nothing is decompressed or executed. - Fail closed on data outside the file, overlapping or duplicate image data, a missing `data`, and hash mismatches. I'll open a draft PR.

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 29, 2026):

Thanks for the detailed walkthrough @555aaditya. Agreed on the fail-closed checksum/record validation - returning partial regions from an untrusted address-to-byte map would mislead downstream analysis, and naming the line makes the failure actionable. Reusing DevicetreeParser for the FIT reader is the right call too, keeping a single parser for FDT-based formats. Looking forward to reviewing the PRs.

@XinwangTao commented on GitHub (Oct 29, 2026): Thanks for the detailed walkthrough @555aaditya. Agreed on the fail-closed checksum/record validation - returning partial regions from an untrusted address-to-byte map would mislead downstream analysis, and naming the line makes the failure actionable. Reusing DevicetreeParser for the FIT reader is the right call too, keeping a single parser for FDT-based formats. Looking forward to reviewing the PRs.

bsakaguchi commented

2026-10-31 00:48:28 +00:00

Author

Owner

@555aaditya commented on GitHub (Oct 29, 2026):

All three parts of this issue have now landed on main:

Intel HEX / S-record regions: #7136
U-Boot legacy uImage: #7234 (thanks @iamcipherdev)
FIT images: #7771, which reuses parseDevicetree as planned above

Thanks @gensokyo-dev for the follow-up hardening and the real mkimage/OpenWrt validation on #7771, and @XinwangTao for the review notes. I think this can be closed as completed.

@555aaditya commented on GitHub (Oct 29, 2026): All three parts of this issue have now landed on main: - Intel HEX / S-record regions: #7136 - U-Boot legacy uImage: #7234 (thanks @iamcipherdev) - FIT images: #7771, which reuses `parseDevicetree` as planned above Thanks @gensokyo-dev for the follow-up hardening and the real mkimage/OpenWrt validation on #7771, and @XinwangTao for the review notes. I think this can be closed as completed.

bsakaguchi commented

2026-10-31 00:48:29 +00:00

Author

Owner

@555aaditya commented on GitHub (Oct 29, 2026):

Closing as completed. All three parts landed through #7136 (Intel HEX / S-record), #7234 (U-Boot legacy uImage) and #7771 (FIT images).

@555aaditya commented on GitHub (Oct 29, 2026): Closing as completed. All three parts landed through #7136 (Intel HEX / S-record), #7234 (U-Boot legacy uImage) and #7771 (FIT images).

bsakaguchi commented

2026-10-31 00:48:29 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 29, 2026):

Nice — all three readers are in now, and the FIT part landing on parseDevicetree matches the single-parser plan from the discussion. The fail-closed record validation plus the coverage-unknown handling for a missing termination record gives analysts a trustworthy address-to-byte map without new dependencies or execution. Closing as completed is fair; the remaining Ghidra load-base work stays tracked in #4246 / #5744.

@XinwangTao commented on GitHub (Oct 29, 2026): Nice — all three readers are in now, and the FIT part landing on parseDevicetree matches the single-parser plan from the discussion. The fail-closed record validation plus the coverage-unknown handling for a missing termination record gives analysts a trustworthy address-to-byte map without new dependencies or execution. Closing as completed is fair; the remaining Ghidra load-base work stays tracked in #4246 / #5744.

bsakaguchi referenced this issue

2026-10-31 00:50:17 +00:00

[GH-ISSUE #7555] [Bug] inspect-signature rejects valid thin arm64e Mach-O files #4496

bsakaguchi referenced this issue

2026-10-31 01:12:20 +00:00

[PR #4291] [CLOSED] fix(artifacts): accept bare arm64e as lipo reported name for arm64e slice #6162

bsakaguchi referenced this issue

2026-10-31 01:12:35 +00:00

[PR #4352] [MERGED] fix(artifacts): reconcile ARM32 Mach-O slice names by subtype #6194