morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #7368] [CLOSED] feat(artifacts): inspect and extract Intel HEX and S-record firmware #7837

New issue

Closed

opened 2026-11-05 01:48:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-05 01:48:02 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/7368
Author: @Ayinkx
Created: 10/27/2026
Status: ❌ Closed

Base: main ← Head: fix/7129-intel-hex-srecord-readers

📝 Commits (1)

2c0527a feat(artifacts): inspect and extract Intel HEX and S-record firmware

📊 Changes

25 files changed (+1918 additions, -15 deletions)

View changed files

📝 docs/firmware-analysis.md (+24 -0)
📝 src/application/BinaryTargetResolver.ts (+3 -0)
📝 src/artifacts/ArtifactReader.ts (+2 -0)
➕ src/artifacts/FirmwareTextArtifactReader.ts (+254 -0)
📝 src/artifacts/createArtifactReader.ts (+6 -0)
📝 src/artifacts/extraction/ArtifactExtraction.ts (+5 -1)
📝 src/artifacts/inventory/ArtifactGraphConstruction.ts (+23 -6)
📝 src/artifacts/inventory/classify.ts (+3 -1)
📝 src/artifacts/inventory/reader.ts (+8 -1)
📝 src/artifacts/inventory/scanCanonical.ts (+1 -0)
📝 src/artifacts/inventory/scanReader.ts (+5 -1)
📝 src/contracts/artifactToolContracts.ts (+2 -2)
📝 src/domain/analysisSnapshot.ts (+2 -0)
📝 src/domain/analysisView/artifactInventoryView.ts (+3 -1)
📝 src/domain/artifactFormatMetadata.ts (+64 -0)
📝 src/domain/artifactGraph.ts (+4 -0)
📝 src/domain/binaryTargetTypes.ts (+6 -1)
📝 src/domain/evidenceSubject.ts (+2 -0)
➕ src/domain/firmwareText.test.ts (+280 -0)
➕ src/domain/firmwareText.ts (+728 -0)

...and 5 more files

📄 Description

Summary

Refs #7129. This implements the first of the three requested firmware container readers: Intel HEX (I8HEX/I16HEX/I32HEX) and Motorola S-record (S19/S28/S37).

Before this change, open_binary rejected these text formats, so inspect_artifact and extract_artifact could not be reached. They now inventory and extract address-contiguous decoded regions without external tools, conversion, or execution.

Note: #7136 proposes the same first slice. This branch was developed independently before that PR was found; maintainers can choose the implementation they prefer.

Problem and expected behavior

An analyst with a vendor .hex, .ihx, .s19, .s28, .s37, .mot, or .srec file needs to know which bytes load at which addresses and which start address the file declares.

Expected and implemented:

Address-contiguous bytes become extractable entries named by inclusive range, e.g. 0x08000000-0x08000003.bin.
Gaps are preserved as gaps and are never zero-filled.
Root format_metadata records the variant, start address and its source record, record counts, EOF/termination state, and the S0 header for S-records.
Declared start addresses are reported as source facts, not proven entry points.
Malformed files fail closed with the source line.

Situation	Outcome
Sparse or out-of-order records	Sorted and merged into address-contiguous regions
Identical overlapping bytes	Accepted
Conflicting overlapping bytes	`format` failure naming both lines and the address
Bad checksum, non-hex content, count mismatch, unknown record type, reserved `S4`	`format` failure naming the line
Intel `02` and `04` address modes mixed	`format` failure
Conflicting start-address records	`format` failure
Missing Intel `01` or S-record `S7`/`S8`/`S9`	Regions returned; coverage `unknown`
Data after EOF/termination, duplicate `S0`, duplicate count record	`format` failure

Change and scope

Pure parsers in src/domain/firmwareText.ts: strict line/record parsing, checksums, address-mode validation, overlap comparison, region merging, and bounded expansion.
FirmwareTextArtifactReader provides bounded, cancellable inventory and extraction with a carrier-identity check on open().
Wiring follows the existing artifact reader factory:
- content/suffix recognition in BinaryTargetResolver and classifyArtifactBytes;
- container role and reader selection for inventory and extraction;
- target, Evidence, snapshot, and live-target format enums;
- additive format_metadata schemas.
Limits: decoded bytes use the existing artifact expansion budget, regions are capped at 100,000, and source lines are capped at 4,096 bytes.
Docs: docs/firmware-analysis.md.

Not included, and tracked separately in #7129:

U-Boot legacy uImage and FIT readers.
CPU inference (#4245).
Raw-binary base/entry changes (#4246, #5744).
Decompression and signature verification.

Contribution scope: format inspection and ordinary extraction. No execution, injection, mutation, new provider, or protection recovery.

Contract and boundary impact

Semantic owner and earliest changed stage: src/domain/firmwareText.ts owns record interpretation; the reader translates typed parser failures into ArtifactReaderFailure.
Contribution scope fit and linked maintainer decision: existing format-inspection boundary; not applicable.
CLI and MCP/tool-catalog contract: no new tools. Additive target/artifact format values intel-hex and srec, and additive format_metadata variants. Tool descriptions mention the new formats.
Provider, bridge, target-format, or platform compatibility: pure TypeScript; no host-specific dependency.
Evidence, artifact, provenance, or reconstruction contract: regions carry byteOffset: null because decoded bytes have no single source range. Declared addresses and record counts remain source facts. Failures name the line.
Target selection, authority, upstream behavior, and cleanup details: content-based recognition at the target boundary; no target execution. Reader cleanup follows the existing artifact reader ownership model.
Generated metadata, package, or installation impact: none committed.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added:

src/domain/firmwareText.test.ts: 12 tests covering I8/I16/I32, S19/S28/S37, sparse regions, identical and conflicting overlaps, checksum/count/unknown-type failures, conflicting address modes and start addresses, and missing termination coverage.
tests/boundary/filesystem/firmwareTextArtifact.test.ts: 5 tests covering target classification, inventory facts and hashes, byte-exact extraction, S0 facts, unknown coverage, fail-closed line reporting, and an ELF named .hex staying on the executable route.
tests/acceptance/firmwareTextCli.test.ts: 2 compiled-CLI tests covering inspection, extraction, and malformed S-record line reporting.
tests/boundary/mcp/firmwareTextMcp.test.ts: 2 stdio-MCP tests (full and compact profiles) covering open_binary, summary inspection, extraction, and malformed input.

Representative output from the compiled CLI for an I32 fixture with two regions:

{
  "root_format": "intel-hex",
  "coverage": { "scope": "container-entries", "status": "complete" },
  "root_metadata": {
    "kind": "intel-hex",
    "variant": "i32",
    "start_address": 134217728,
    "start_address_source": "record-05",
    "eof_terminated": true
  },
  "regions": [
    "0x08000000-0x08000003.bin",
    "0x08000010-0x08000011.bin"
  ]
}

Remaining proof gaps:

Synthetic fixtures with computed checksums only; no GNU objcopy, srec_cat, or real vendor files were tested.
No real device programming files were used.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

Host: Windows x64, Node 24.22.1.

npm run compile — exit 0.
vitest run on the four new test files — 21 passed.
Related existing artifact/binary-target tests — 44 passed and 76 passed (10 skipped).
TypeScript typecheck against tsconfig.json — exit 0. The repository's native compiler hit host memory limits, so the same config was run with TypeScript 5.9.3 and a 7 GiB heap.
npm run format:check — all matched files formatted.
npm run verify:test-discovery — all 2,639 test files discovered exactly once.
node scripts/verify-module-boundaries.mjs — clean across 3,213 source files.

Compatibility, safety, and release

Breaking changes or migration steps: none. Previously refused text files are recognized only when the first record is structurally valid or the suffix is known and the prefix is textual.
Real Hopper/Ghidra, browser, or OS coverage: not applicable.
Package or release metadata impact: none.
Security, privacy, process, or containment review: no execution or mutation. Decoded bytes are bounded by the existing artifact expansion budget, region count is capped, generated paths cannot traverse, and the reader is cancellable.

Author readiness

Human owner:

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/7368 **Author:** [@Ayinkx](https://github.com/Ayinkx) **Created:** 10/27/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/7129-intel-hex-srecord-readers` --- ### 📝 Commits (1) - [`2c0527a`](https://github.com/morluto/rea/commit/2c0527ad875d1d3da9556c57aae43f6dded8101c) feat(artifacts): inspect and extract Intel HEX and S-record firmware ### 📊 Changes **25 files changed** (+1918 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `docs/firmware-analysis.md` (+24 -0) 📝 `src/application/BinaryTargetResolver.ts` (+3 -0) 📝 `src/artifacts/ArtifactReader.ts` (+2 -0) ➕ `src/artifacts/FirmwareTextArtifactReader.ts` (+254 -0) 📝 `src/artifacts/createArtifactReader.ts` (+6 -0) 📝 `src/artifacts/extraction/ArtifactExtraction.ts` (+5 -1) 📝 `src/artifacts/inventory/ArtifactGraphConstruction.ts` (+23 -6) 📝 `src/artifacts/inventory/classify.ts` (+3 -1) 📝 `src/artifacts/inventory/reader.ts` (+8 -1) 📝 `src/artifacts/inventory/scanCanonical.ts` (+1 -0) 📝 `src/artifacts/inventory/scanReader.ts` (+5 -1) 📝 `src/contracts/artifactToolContracts.ts` (+2 -2) 📝 `src/domain/analysisSnapshot.ts` (+2 -0) 📝 `src/domain/analysisView/artifactInventoryView.ts` (+3 -1) 📝 `src/domain/artifactFormatMetadata.ts` (+64 -0) 📝 `src/domain/artifactGraph.ts` (+4 -0) 📝 `src/domain/binaryTargetTypes.ts` (+6 -1) 📝 `src/domain/evidenceSubject.ts` (+2 -0) ➕ `src/domain/firmwareText.test.ts` (+280 -0) ➕ `src/domain/firmwareText.ts` (+728 -0) _...and 5 more files_ </details> ### 📄 Description ## Summary Refs #7129. This implements the first of the three requested firmware container readers: Intel HEX (I8HEX/I16HEX/I32HEX) and Motorola S-record (S19/S28/S37). Before this change, `open_binary` rejected these text formats, so `inspect_artifact` and `extract_artifact` could not be reached. They now inventory and extract address-contiguous decoded regions without external tools, conversion, or execution. Note: #7136 proposes the same first slice. This branch was developed independently before that PR was found; maintainers can choose the implementation they prefer. ## Problem and expected behavior An analyst with a vendor `.hex`, `.ihx`, `.s19`, `.s28`, `.s37`, `.mot`, or `.srec` file needs to know which bytes load at which addresses and which start address the file declares. Expected and implemented: - Address-contiguous bytes become extractable entries named by inclusive range, e.g. `0x08000000-0x08000003.bin`. - Gaps are preserved as gaps and are never zero-filled. - Root `format_metadata` records the variant, start address and its source record, record counts, EOF/termination state, and the S0 header for S-records. - Declared start addresses are reported as source facts, not proven entry points. - Malformed files fail closed with the source line. | Situation | Outcome | | --- | --- | | Sparse or out-of-order records | Sorted and merged into address-contiguous regions | | Identical overlapping bytes | Accepted | | Conflicting overlapping bytes | `format` failure naming both lines and the address | | Bad checksum, non-hex content, count mismatch, unknown record type, reserved `S4` | `format` failure naming the line | | Intel `02` and `04` address modes mixed | `format` failure | | Conflicting start-address records | `format` failure | | Missing Intel `01` or S-record `S7`/`S8`/`S9` | Regions returned; coverage `unknown` | | Data after EOF/termination, duplicate `S0`, duplicate count record | `format` failure | ## Change and scope - Pure parsers in `src/domain/firmwareText.ts`: strict line/record parsing, checksums, address-mode validation, overlap comparison, region merging, and bounded expansion. - `FirmwareTextArtifactReader` provides bounded, cancellable inventory and extraction with a carrier-identity check on `open()`. - Wiring follows the existing artifact reader factory: - content/suffix recognition in `BinaryTargetResolver` and `classifyArtifactBytes`; - container role and reader selection for inventory and extraction; - target, Evidence, snapshot, and live-target format enums; - additive `format_metadata` schemas. - Limits: decoded bytes use the existing artifact expansion budget, regions are capped at 100,000, and source lines are capped at 4,096 bytes. - Docs: `docs/firmware-analysis.md`. Not included, and tracked separately in #7129: - U-Boot legacy uImage and FIT readers. - CPU inference (#4245). - Raw-binary base/entry changes (#4246, #5744). - Decompression and signature verification. Contribution scope: format inspection and ordinary extraction. No execution, injection, mutation, new provider, or protection recovery. ## Contract and boundary impact - **Semantic owner and earliest changed stage:** `src/domain/firmwareText.ts` owns record interpretation; the reader translates typed parser failures into `ArtifactReaderFailure`. - **Contribution scope fit and linked maintainer decision:** existing format-inspection boundary; not applicable. - **CLI and MCP/tool-catalog contract:** no new tools. Additive target/artifact format values `intel-hex` and `srec`, and additive `format_metadata` variants. Tool descriptions mention the new formats. - **Provider, bridge, target-format, or platform compatibility:** pure TypeScript; no host-specific dependency. - **Evidence, artifact, provenance, or reconstruction contract:** regions carry `byteOffset: null` because decoded bytes have no single source range. Declared addresses and record counts remain source facts. Failures name the line. - **Target selection, authority, upstream behavior, and cleanup details:** content-based recognition at the target boundary; no target execution. Reader cleanup follows the existing artifact reader ownership model. - **Generated metadata, package, or installation impact:** none committed. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage Tests added: - `src/domain/firmwareText.test.ts`: 12 tests covering I8/I16/I32, S19/S28/S37, sparse regions, identical and conflicting overlaps, checksum/count/unknown-type failures, conflicting address modes and start addresses, and missing termination coverage. - `tests/boundary/filesystem/firmwareTextArtifact.test.ts`: 5 tests covering target classification, inventory facts and hashes, byte-exact extraction, S0 facts, unknown coverage, fail-closed line reporting, and an ELF named `.hex` staying on the executable route. - `tests/acceptance/firmwareTextCli.test.ts`: 2 compiled-CLI tests covering inspection, extraction, and malformed S-record line reporting. - `tests/boundary/mcp/firmwareTextMcp.test.ts`: 2 stdio-MCP tests (full and compact profiles) covering `open_binary`, summary inspection, extraction, and malformed input. Representative output from the compiled CLI for an I32 fixture with two regions: ```json { "root_format": "intel-hex", "coverage": { "scope": "container-entries", "status": "complete" }, "root_metadata": { "kind": "intel-hex", "variant": "i32", "start_address": 134217728, "start_address_source": "record-05", "eof_terminated": true }, "regions": [ "0x08000000-0x08000003.bin", "0x08000010-0x08000011.bin" ] } ``` Remaining proof gaps: - Synthetic fixtures with computed checksums only; no GNU objcopy, srec_cat, or real vendor files were tested. - No real device programming files were used. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed Host: Windows x64, Node 24.22.1. - `npm run compile` — exit 0. - `vitest run` on the four new test files — 21 passed. - Related existing artifact/binary-target tests — 44 passed and 76 passed (10 skipped). - TypeScript typecheck against `tsconfig.json` — exit 0. The repository's native compiler hit host memory limits, so the same config was run with TypeScript 5.9.3 and a 7 GiB heap. - `npm run format:check` — all matched files formatted. - `npm run verify:test-discovery` — all 2,639 test files discovered exactly once. - `node scripts/verify-module-boundaries.mjs` — clean across 3,213 source files. ## Compatibility, safety, and release - **Breaking changes or migration steps:** none. Previously refused text files are recognized only when the first record is structurally valid or the suffix is known and the prefix is textual. - **Real Hopper/Ghidra, browser, or OS coverage:** not applicable. - **Package or release metadata impact:** none. - **Security, privacy, process, or containment review:** no execution or mutation. Decoded bytes are bounded by the existing artifact expansion budget, region count is capped, generated paths cannot traverse, and the reader is cancellable. ## Author readiness  Human owner: - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-05 01:48:02 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7837

No description provided.

Rows
Columns