morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7457] [Feature] Evidence-backed firmware update comparison with component-level change attribution #4452

New issue

Closed

opened 2026-10-31 00:49:54 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-31 00:49:54 +00:00

Owner

Originally created by @krishanth7 on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7457

Before submitting

I searched related open and closed issues and pull requests for overlapping proposals; the related work and remaining distinctions are described below.
I reviewed the contribution scope and understand that scope and architecture must be reviewed before integration; open design questions are welcome.
This feature request contains no credentials, private samples, or sensitive target data.

Problem

REA supports firmware-oriented inspection and evidence workflows, but analysts comparing two locally supplied firmware releases need a bounded, component-level answer to: what changed, where is the supporting evidence, and what remains unknown? A raw whole-package byte difference is often dominated by packaging, compression, timestamps, or component relocation. This proposal asks whether REA can provide a reusable comparison view without implying semantic equivalence or automatically executing firmware.

Desired outcome

Introduce an evidence-backed firmware update comparison workflow, accessible through equivalent CLI and MCP operations, that compares two explicitly selected local firmware artifacts and returns a structured inventory of added, removed, unchanged, and modified components, with traceable provenance and honest coverage.

Proposed initial scope

Accept two caller-selected local firmware packages or previously retained, compatible firmware inventories. Keep input admission, size/depth limits, and filesystem boundaries explicit.
Reuse existing supported inspection/extraction capabilities; do not silently install external tools, unpack unsupported formats, or bypass encryption, signatures, or other protections.
Record package and component hashes, recognized formats, offsets/paths where available, and exact source artifact identities. Match components deterministically using supported identities; distinguish strong matches from tentative associations.
Report additions, removals, byte-level modifications, and unchanged components. Identify architecture/format metadata changes where existing analyzers can substantiate them; avoid claiming behavioral changes from hashes alone.
Emit machine-readable Evidence through CLI/MCP, including unknown/unreadable components, unsupported formats, truncated analysis, resource limits, and explicit confidence/coverage.
Support bounded, reproducible output with component filters or pagination for large packages; retain complete evidence independently of a caller-selected summary view.

Illustrative result (proposed schema, not existing REA output)

{
  "comparison": {"baseline": "sha256:...", "candidate": "sha256:..."},
  "components": [
    {
      "identity": "rootfs/usr/bin/example",
      "status": "modified",
      "basis": "matched component path and recognized format",
      "baseline_sha256": "...",
      "candidate_sha256": "...",
      "evidence_refs": ["..."]
    }
  ],
  "coverage": {"status": "partial", "unknowns": ["unsupported nested payload"]}
}

The schema and command names are illustrative; the maintainers should choose contracts consistent with existing Evidence and comparison APIs.

Acceptance criteria

A fixture with one added, one removed, one unchanged, and one changed component produces correct statuses and links each finding to its original artifact.
A repackaged fixture with changed container bytes but identical supported extracted component bytes does not mislabel those components as modified.
Unsupported or encrypted members remain explicit unknowns; no inferred semantic equivalence is presented as verified.
CLI and MCP results agree on identities, statuses, coverage, and evidence references.
Bounded resource usage, cancellation, cleanup, and malformed/nested archive handling are verified using self-owned synthetic fixtures.
Documentation explains matching rules, false-positive/false-negative limits, supported formats, and how to reproduce a comparison.

Non-goals and safety boundaries

No firmware execution, device flashing, exploitation, vulnerability scoring, credential recovery, protection bypass, network access, or automatic mutation of input artifacts. Any future runtime/device workflow would require a separate authority and lifecycle decision.

Relationship to existing work

#718 tracks broader IoT firmware-analysis tools and roadmap research. This request is narrower: comparison of two firmware releases with component-level attribution and Evidence provenance, ideally implemented by composing supported capabilities rather than adding a new extraction backend.
#4110 discusses Unblob/FACT/Binwalk-NG backend scope; this request does not presume their adoption.
#5851 concerns firmware analysis budgets, which this proposal should respect.
The repository roadmap prioritizes evidence preservation, shared CLI/MCP workflows, and explicit authority boundaries.

Open design questions

Which currently supported firmware inventory or Evidence representation is best suited to stable cross-version component identity?
Should the first iteration compare only recognized, already-extracted components?
How should ambiguous renames, duplicate components, and nested-package coverage be represented?

Status: Design proposal only. No implementation, performance measurements, or runtime verification are claimed.

Originally created by @krishanth7 on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7457 ## Before submitting - [x] I searched related open and closed issues and pull requests for overlapping proposals; the related work and remaining distinctions are described below. - [x] I reviewed the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) and understand that scope and architecture must be reviewed before integration; open design questions are welcome. - [x] This feature request contains no credentials, private samples, or sensitive target data. ## Problem REA supports firmware-oriented inspection and evidence workflows, but analysts comparing two locally supplied firmware releases need a bounded, component-level answer to: **what changed, where is the supporting evidence, and what remains unknown?** A raw whole-package byte difference is often dominated by packaging, compression, timestamps, or component relocation. This proposal asks whether REA can provide a reusable comparison view without implying semantic equivalence or automatically executing firmware. ## Desired outcome Introduce an **evidence-backed firmware update comparison** workflow, accessible through equivalent CLI and MCP operations, that compares two explicitly selected local firmware artifacts and returns a structured inventory of added, removed, unchanged, and modified components, with traceable provenance and honest coverage. ## Proposed initial scope - Accept two caller-selected local firmware packages or previously retained, compatible firmware inventories. Keep input admission, size/depth limits, and filesystem boundaries explicit. - Reuse existing supported inspection/extraction capabilities; do not silently install external tools, unpack unsupported formats, or bypass encryption, signatures, or other protections. - Record package and component hashes, recognized formats, offsets/paths where available, and exact source artifact identities. Match components deterministically using supported identities; distinguish strong matches from tentative associations. - Report additions, removals, byte-level modifications, and unchanged components. Identify architecture/format metadata changes where existing analyzers can substantiate them; avoid claiming behavioral changes from hashes alone. - Emit machine-readable Evidence through CLI/MCP, including unknown/unreadable components, unsupported formats, truncated analysis, resource limits, and explicit confidence/coverage. - Support bounded, reproducible output with component filters or pagination for large packages; retain complete evidence independently of a caller-selected summary view. ## Illustrative result (proposed schema, not existing REA output) ```json { "comparison": {"baseline": "sha256:...", "candidate": "sha256:..."}, "components": [ { "identity": "rootfs/usr/bin/example", "status": "modified", "basis": "matched component path and recognized format", "baseline_sha256": "...", "candidate_sha256": "...", "evidence_refs": ["..."] } ], "coverage": {"status": "partial", "unknowns": ["unsupported nested payload"]} } ``` The schema and command names are illustrative; the maintainers should choose contracts consistent with existing Evidence and comparison APIs. ## Acceptance criteria 1. A fixture with one added, one removed, one unchanged, and one changed component produces correct statuses and links each finding to its original artifact. 2. A repackaged fixture with changed container bytes but identical supported extracted component bytes does not mislabel those components as modified. 3. Unsupported or encrypted members remain explicit unknowns; no inferred semantic equivalence is presented as verified. 4. CLI and MCP results agree on identities, statuses, coverage, and evidence references. 5. Bounded resource usage, cancellation, cleanup, and malformed/nested archive handling are verified using self-owned synthetic fixtures. 6. Documentation explains matching rules, false-positive/false-negative limits, supported formats, and how to reproduce a comparison. ## Non-goals and safety boundaries No firmware execution, device flashing, exploitation, vulnerability scoring, credential recovery, protection bypass, network access, or automatic mutation of input artifacts. Any future runtime/device workflow would require a separate authority and lifecycle decision. ## Relationship to existing work - #718 tracks broader IoT firmware-analysis tools and roadmap research. This request is narrower: **comparison of two firmware releases with component-level attribution and Evidence provenance**, ideally implemented by composing supported capabilities rather than adding a new extraction backend. - #4110 discusses Unblob/FACT/Binwalk-NG backend scope; this request does not presume their adoption. - #5851 concerns firmware analysis budgets, which this proposal should respect. - The repository roadmap prioritizes evidence preservation, shared CLI/MCP workflows, and explicit authority boundaries. ## Open design questions - Which currently supported firmware inventory or Evidence representation is best suited to stable cross-version component identity? - Should the first iteration compare only recognized, already-extracted components? - How should ambiguous renames, duplicate components, and nested-package coverage be represented? **Status:** Design proposal only. No implementation, performance measurements, or runtime verification are claimed.

bsakaguchi closed this issue

2026-10-31 00:49:54 +00:00

bsakaguchi commented

2026-10-31 00:49:55 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 27, 2026):

This framing is right: a bounded, component-level comparison view is much more actionable than raw whole-package byte diffs, which are dominated by packaging noise. Two things I'd keep front and center in the initial scope: (1) make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged; (2) keep the "no automatic firmware execution, no semantic equivalence claims" boundary explicit in the output, since that is what keeps the evidence honest. Equivalent CLI/MCP parity from day one also avoids the usual drift between the two surfaces.

@XinwangTao commented on GitHub (Oct 27, 2026): This framing is right: a bounded, component-level comparison view is much more actionable than raw whole-package byte diffs, which are dominated by packaging noise. Two things I'd keep front and center in the initial scope: (1) make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged; (2) keep the "no automatic firmware execution, no semantic equivalence claims" boundary explicit in the output, since that is what keeps the evidence honest. Equivalent CLI/MCP parity from day one also avoids the usual drift between the two surfaces.

bsakaguchi commented

2026-10-31 00:49:55 +00:00

Author

Owner

@krishanth7 commented on GitHub (Oct 27, 2026):

Make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged.»

Thanks for the feedback, @XinwangTao. I think that's an important distinction, especially when two firmware packages have different container hashes but their internal components haven't actually changed.

I agree that component identity needs to be clearly defined from the beginning. Using normalized paths, file sizes, and hashes seems like a good starting point, although we'd also need to handle cases where files are renamed, relocated, or have duplicate paths without making incorrect matches.

I also agree with keeping the safety boundaries explicit. The intention here is strictly static comparison, not firmware execution or behavioral analysis. Any conclusions should be backed by the available evidence, with unsupported or ambiguous results clearly identified.

Your point about CLI/MCP parity is equally important. I'd prefer both interfaces to share the same underlying comparison logic and Evidence representation rather than maintaining separate implementations.

For the initial scope, I'm leaning toward comparing components that REA can already recognize or extract, then expanding coverage once the matching rules and verification tests are established.

Thanks again for the suggestions. They help narrow this down into something more practical and maintainable.

@krishanth7 commented on GitHub (Oct 27, 2026): > Make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged.» Thanks for the feedback, @XinwangTao. I think that's an important distinction, especially when two firmware packages have different container hashes but their internal components haven't actually changed. I agree that component identity needs to be clearly defined from the beginning. Using normalized paths, file sizes, and hashes seems like a good starting point, although we'd also need to handle cases where files are renamed, relocated, or have duplicate paths without making incorrect matches. I also agree with keeping the safety boundaries explicit. The intention here is strictly static comparison, not firmware execution or behavioral analysis. Any conclusions should be backed by the available evidence, with unsupported or ambiguous results clearly identified. Your point about CLI/MCP parity is equally important. I'd prefer both interfaces to share the same underlying comparison logic and Evidence representation rather than maintaining separate implementations. For the initial scope, I'm leaning toward comparing components that REA can already recognize or extract, then expanding coverage once the matching rules and verification tests are established. Thanks again for the suggestions. They help narrow this down into something more practical and maintainable.

bsakaguchi commented

2026-10-31 00:49:55 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 28, 2026):

Re the rename/relocate/duplicate-path concern: a good way to keep the initial scope bounded is to tier the matching instead of trying to solve similarity up front — strong match (normalized path + size + hash all agree), tentative association (hash agrees but path differs, or path agrees but hash differs), and unmatched (listed with full evidence so an analyst can eyeball it). That maps cleanly onto the "distinguish strong matches from tentative associations" line already in the proposal, and avoids fuzzy-matching heuristics that would be hard to verify. The renamed-but-identical case then surfaces as a small, inspectable tentative set rather than a correctness problem. Happy to help spec the matching rules if you open that as a follow-up.

@XinwangTao commented on GitHub (Oct 28, 2026): Re the rename/relocate/duplicate-path concern: a good way to keep the initial scope bounded is to tier the matching instead of trying to solve similarity up front — strong match (normalized path + size + hash all agree), tentative association (hash agrees but path differs, or path agrees but hash differs), and unmatched (listed with full evidence so an analyst can eyeball it). That maps cleanly onto the "distinguish strong matches from tentative associations" line already in the proposal, and avoids fuzzy-matching heuristics that would be hard to verify. The renamed-but-identical case then surfaces as a small, inspectable tentative set rather than a correctness problem. Happy to help spec the matching rules if you open that as a follow-up.

bsakaguchi commented

2026-10-31 00:49:55 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 30, 2026):

赞同收敛到"REA 已能识别/提取的组件"作为初始范围，与分层匹配（strong/tentative/unmatched）的思路一致，范围可控且可验证。补充一点落地建议：验收标准里那个"repackaged fixture"用例可以同时断言 tentative 集合非空且可被人工检查（比如 renamed-but-identical 组件落入 tentative 而不是被误报为 modified），这样分层匹配的边界就有明确的回归保障。如果后续要开匹配规则的 spec，我可以一起参与定义判定条件。

@XinwangTao commented on GitHub (Oct 30, 2026): 赞同收敛到"REA 已能识别/提取的组件"作为初始范围，与分层匹配（strong/tentative/unmatched）的思路一致，范围可控且可验证。补充一点落地建议：验收标准里那个"repackaged fixture"用例可以同时断言 tentative 集合非空且可被人工检查（比如 renamed-but-identical 组件落入 tentative 而不是被误报为 modified），这样分层匹配的边界就有明确的回归保障。如果后续要开匹配规则的 spec，我可以一起参与定义判定条件。

bsakaguchi commented

2026-10-31 00:49:55 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 30, 2026):

@XinwangTao commented on GitHub (Oct 30, 2026): 赞同收敛到"REA 已能识别/提取的组件"作为初始范围，与分层匹配（strong/tentative/unmatched）的思路一致，范围可控且可验证。补充一点落地建议：验收标准里那个"repackaged fixture"用例可以同时断言 tentative 集合非空且可被人工检查（比如 renamed-but-identical 组件落入 tentative 而不是被误报为 modified），这样分层匹配的边界就有明确的回归保障。如果后续要开匹配规则的 spec，我可以一起参与定义判定条件。

bsakaguchi referenced this issue

2026-10-31 01:13:03 +00:00

[PR #4452] [MERGED] fix(evidence): classify ledger duplicate and missing review Evidence as caller input #6252

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4452

No description provided.

Rows
Columns