[GH-ISSUE #7461] [Feature] Evidence-backed firmware update comparison with component-level change attribution #4454

Open
opened 2026-10-31 00:49:56 +00:00 by bsakaguchi · 10 comments
Owner

Originally created by @krishanth7 on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7461

Before submitting

  • I searched the open and closed issues and pull requests for a related proposal.
  • I understand the contribution scope must be reviewed before integration; open questions are welcome here.
  • This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

Current Limitation

Reverse-engineering analysts often need to compare two versions of a firmware package to determine which internal components have been added, removed, modified, or left unchanged.

Although REA provides firmware-oriented inspection and evidence workflows, there is no clearly established, unified workflow for evidence-backed, component-level comparison between two firmware releases.

Today, analysts may need to inspect each package separately, manually correlate extracted components, compare file hashes, and reconstruct the differences using external scripts or tools.

This creates several challenges:

  • Fragmented analysis: Component identification and comparison require multiple manual steps.
  • Limited change attribution: Whole-package hash differences cannot identify which internal components actually changed.
  • Evidence traceability: Findings must be manually linked to their original firmware artifacts.
  • Reproducibility: Independent analysts may use different matching rules and produce inconsistent results.
  • Incomplete coverage: Encrypted, unsupported, or partially extracted components need explicit reporting rather than assumptions.
  • Automation overhead: CLI and MCP consumers lack a single structured comparison result for downstream workflows.

Example Scenario

An analyst receives two authorized firmware packages:

  • firmware-v1.0.bin — Baseline release
  • firmware-v1.1.bin — Updated release

The analyst wants to identify:

  1. Components added or removed between releases.
  2. Executables, libraries, and configuration files that changed.
  3. Components whose extracted bytes remain identical despite package-level differences.
  4. Cryptographic hashes and source locations supporting each finding.
  5. Components that could not be reliably inspected or matched.

Desired Improvement

A bounded, deterministic firmware comparison workflow that reuses REA's existing analysis capabilities and produces structured, verifiable Evidence with component-level change attribution, explicit coverage limitations, and consistent CLI/MCP results.

The proposed workflow should not execute firmware, modify source artifacts, or bypass security protections.

Note: This is a proposed capability, not a claim that every underlying operation is currently unsupported or that a particular failure has been reproduced.

Use cases

Use Cases

1. Firmware Version Comparison

Analysts comparing two authorized firmware releases need to identify added, removed, modified, and unchanged internal components without relying exclusively on whole-package binary differences.

Practical impact: Reduces manual comparison work and improves the traceability of release-to-release changes.

2. Embedded Systems and IoT Research

Researchers investigating embedded Linux devices, network appliances, and IoT firmware need to compare extracted executables, libraries, configuration files, and other recognized artifacts across firmware versions.

Practical impact: Helps researchers identify relevant component changes and prioritize further investigation.

3. Software Supply Chain Verification

Engineering and security teams need to verify whether specific firmware components have changed between approved baseline and candidate releases.

Practical impact: Provides reproducible component hashes, change classifications, and source-artifact references for technical review.

4. Automated CLI and MCP Workflows

Developers and AI-assisted reverse-engineering workflows need structured comparison results that can be consumed programmatically.

Practical impact: Enables consistent downstream reporting, automated change summaries, and evidence-based investigation without manually correlating independent inspection outputs.

5. Evidence Preservation and Auditability

Analysts need to retain the relationship between each reported change and the original firmware package, including component identity, source location, cryptographic hashes, and analysis limitations.

Practical impact: Improves reproducibility, auditability, and confidence in technical findings.

Representative Workflow

  1. Select two locally available firmware packages.
  2. Inspect supported package structures and components.
  3. Match recognized components using deterministic criteria.
  4. Classify verified additions, removals, modifications, and unchanged components.
  5. Report unsupported or ambiguous components as unknown.
  6. Generate structured comparison Evidence for CLI/MCP consumers.

Frequency and Measurements

These are representative proposed use cases, not claims of observed production usage.

No benchmark results, measured time savings, user-frequency statistics, or production-scale performance measurements are available for this proposal.

Initial validation should use synthetic, independently verifiable firmware fixtures and explicitly report supported formats, resource consumption, comparison accuracy, and coverage limitations.

Desired outcome and success criteria

Desired Outcome

Introduce a reliable, deterministic, evidence-backed firmware comparison capability within REA that enables analysts to compare two authorized, locally supplied firmware releases and identify component-level changes through a consistent, reproducible workflow.

The feature should integrate with REA's existing inspection, artifact, and Evidence capabilities while preserving compatibility with CLI and MCP interfaces.

1. Expected Functional Behavior

The proposed capability should:

  • Accept two explicitly selected firmware artifacts representing baseline and candidate releases.
  • Inspect supported firmware containers and their recognized internal components.
  • Identify added, removed, modified, and unchanged components.
  • Distinguish package-level differences from verified component-level changes.
  • Match components using deterministic, documented identification rules.
  • Preserve cryptographic hashes and source-artifact references.
  • Report ambiguous matches, unsupported formats, encrypted components, and incomplete inspections explicitly.
  • Produce consistent, machine-readable results through CLI and MCP interfaces.
  • Operate within documented resource limits without modifying or executing the supplied firmware.

2. Expected Output

A successful comparison should produce a structured report containing:

Field Expected Result
Baseline artifact Identity and SHA-256 hash
Candidate artifact Identity and SHA-256 hash
Component identity Recognized path or stable identifier
Change classification Added, removed, modified, or unchanged
Comparison basis Documented matching criteria
Component hashes Baseline and candidate hashes where applicable
Evidence references Traceable source-artifact locations
Coverage Complete, partial, or unsupported
Unknowns Explicit unresolved or uninspected components
Diagnostics Actionable errors and recovery guidance

Illustrative JSON Output

The following represents a proposed output structure, not an existing REA API contract.

{
  "operation": "firmware_comparison",
  "baseline": {
    "artifact": "firmware-v1.0.bin",
    "sha256": "<baseline-sha256>"
  },
  "candidate": {
    "artifact": "firmware-v1.1.bin",
    "sha256": "<candidate-sha256>"
  },
  "summary": {
    "added": 1,
    "removed": 1,
    "modified": 1,
    "unchanged": 1
  },
  "components": [
    {
      "identity": "rootfs/usr/bin/example",
      "status": "modified",
      "matching_basis": "recognized_path_and_format",
      "baseline_sha256": "<old-sha256>",
      "candidate_sha256": "<new-sha256>",
      "evidence_refs": [
        "<baseline-evidence-ref>",
        "<candidate-evidence-ref>"
      ]
    }
  ],
  "coverage": {
    "status": "partial",
    "unknowns": [
      "unsupported_nested_payload"
    ]
  }
}

The final schema, command names, and Evidence representation should follow REA's existing architectural and compatibility contracts.

Success Criteria

The feature should be considered successful when the following acceptance criteria are satisfied.

  • SC-01 — Input validation: Both firmware inputs are validated before analysis, with clear diagnostics for invalid, inaccessible, or unsupported inputs.

  • SC-02 — Accurate classification: Synthetic test fixtures containing added, removed, modified, and unchanged components produce the expected classifications.

  • SC-03 — Repackaging awareness: Different outer package bytes do not automatically cause identical extracted components to be classified as modified.

  • SC-04 — Deterministic results: Repeated comparisons of identical inputs and settings produce equivalent component identities, classifications, and Evidence references.

  • SC-05 — Evidence traceability: Every reported component-level finding can be traced to the corresponding original artifact and supporting comparison evidence.

  • SC-06 — Honest coverage: Unsupported, encrypted, ambiguous, or partially inspected components are reported explicitly without unsupported conclusions.

  • SC-07 — CLI/MCP consistency: Equivalent CLI and MCP requests produce semantically consistent comparison results.

  • SC-08 — Resource safety: Processing respects documented memory, execution-time, extraction-depth, cancellation, and cleanup boundaries.

  • SC-09 — Input integrity: Original firmware artifacts remain unchanged. No firmware execution, device flashing, network activity, or protection bypass occurs.

  • SC-10 — Error recovery: Malformed packages, unavailable components, and resource-limit failures produce actionable diagnostics while preserving valid findings where supported.

  • SC-11 — Automated verification: Tests cover ordinary comparisons, duplicate component identities, ambiguous matches, nested archives, unsupported formats, and interrupted operations.

  • SC-12 — Documentation: User documentation explains supported formats, comparison rules, limitations, example workflows, and the meaning of coverage and Evidence fields.

Proposed Validation Approach

Use small, self-contained synthetic firmware fixtures with independently known component inventories and expected differences.

Validate:

  1. Correctness of component classification.
  2. Stability of deterministic matching.
  3. Integrity and traceability of Evidence.
  4. Consistency between CLI and MCP outputs.
  5. Safe behavior for malformed and unsupported inputs.
  6. Resource-boundary enforcement and cleanup.

Performance measurements should be collected during implementation rather than assumed in advance.

Scope and Compatibility

The initial implementation should prioritize supported static inspection and comparison capabilities.

It should not introduce automatic firmware execution, credential extraction, security-control bypass, device modification, or uncontrolled external tool execution.

Existing REA Evidence contracts, provider boundaries, input-validation policies, and compatibility requirements should remain authoritative.

Definition of Done

The feature is complete when the agreed initial scope passes its automated verification, produces accurate and reproducible component-level comparison Evidence, preserves input integrity, documents known limitations, and receives the required maintainer review.

Proposal status: Requested capability only. No implementation, successful tests, performance improvements, or maintainer approval are claimed.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Target and Scope Details

Target Platforms

The proposed feature targets authorized, locally available firmware artifacts used in embedded systems and IoT environments.

Representative targets include:

  • Embedded Linux firmware packages.
  • IoT device firmware images.
  • Network appliance firmware.
  • Router and gateway firmware.
  • Industrial embedded-controller firmware.
  • Firmware archives containing recognized executables, libraries, configuration files, and filesystem components.

These are potential use cases, not a claim that REA currently supports every listed format or platform.

Expected Target Interaction

Selected effect: Read existing files or captured data.

The workflow should perform static, read-only analysis of two explicitly supplied firmware artifacts.

Expected operations include:

  1. Reading and validating baseline and candidate firmware files.
  2. Inspecting supported container structures and component metadata.
  3. Comparing recognized components using deterministic matching rules.
  4. Calculating cryptographic hashes where supported.
  5. Producing component-level change classifications.
  6. Preserving source references and reporting incomplete analysis.

The original firmware artifacts must remain unchanged.

Scope Boundaries

The initial implementation should:

  • Reuse existing REA inspection and Evidence capabilities.
  • Operate only on explicitly selected local inputs.
  • Avoid executing firmware or launching target processes.
  • Avoid attaching debuggers or injecting instrumentation.
  • Avoid changing process memory, registers, or target configurations.
  • Avoid contacting network services.
  • Avoid bypassing encryption, signatures, or other security protections.
  • Respect existing resource limits and cleanup requirements.

Output and File Handling

The preferred initial workflow is read-only with respect to source artifacts.

Machine-readable comparison results should be returned through existing CLI/MCP output mechanisms.

Whether optional report export, temporary extraction, or persistent artifact storage requires additional file-write effects should be decided during design review.

Non-Goals

  • Firmware flashing or device modification.
  • Runtime instrumentation or dynamic execution.
  • Exploit development or protection bypass.
  • Credential extraction.
  • Automated vulnerability exploitation.
  • Unsupported claims of semantic or behavioral equivalence.
  • Unrestricted recursive extraction of unknown formats.

Open Scope Questions

  1. Should the first release support only firmware formats already recognized by REA?
  2. Should comparison initially operate on existing extracted component inventories?
  3. How should ambiguous component identities and renamed files be represented?
  4. Should report export be a separate optional capability?
  5. What resource limits should apply to nested firmware containers?

Initial Implementation Preference

Start with a narrowly scoped, deterministic, static comparison workflow using existing REA capabilities and synthetic test fixtures.

Expand format coverage only after the initial Evidence contracts, comparison accuracy, resource safety, and maintainer-approved scope are established.

Status: Proposed design scope; no implementation or platform-specific verification is claimed.

Verification available so far

Idea only; I need help defining checks

Verification details (optional)

Verification Details

Verification Status

Current status: Design proposal only.

No working prototype, executed test suite, benchmark results, or production validation is available at this stage.

The following verification scenarios and expected results are proposed for discussion with REA maintainers.

1. Proposed Test Environment

Use self-generated, synthetic firmware-like archives containing known files and deterministic modifications.

Suggested fixtures:

  • firmware-baseline.bin — Baseline package.
  • firmware-candidate.bin — Updated package.
  • expected-changes.json — Independently defined expected results.

The fixtures should contain no proprietary firmware, credentials, private data, or third-party protected content.

2. Component Classification Test

Construct two supported firmware-like packages with the following differences:

Component Baseline Candidate Expected
bin/app Version A Version B Modified
lib/core.so Present Present, identical bytes Unchanged
etc/legacy.conf Present Absent Removed
etc/new.conf Absent Present Added

Expected result: REA correctly classifies all four components and provides traceable source references.

3. Repackaging Test

Create two packages containing identical supported components but different outer container metadata or compression settings.

Expected result: Package-level hashes may differ, while verified identical extracted components remain classified as unchanged.

4. Evidence Integrity Test

Verify that each reported component change includes:

  • Source artifact identity.
  • Baseline and candidate component hashes where applicable.
  • Deterministic matching basis.
  • Relevant source paths or offsets.
  • Evidence references.
  • Explicit analysis coverage and unknowns.

Expected result: Findings are reproducible and traceable to their original inputs.

5. Unsupported Component Test

Include an unsupported or intentionally unreadable nested payload in a synthetic fixture.

Expected result: REA reports the component as unresolved or unsupported without inventing a change classification or claiming complete coverage.

6. CLI and MCP Consistency Test

Submit equivalent comparison requests through the proposed CLI and MCP interfaces.

Expected result: Both interfaces return semantically equivalent component classifications, evidence references, and coverage information.

7. Resource and Safety Test

Use synthetic fixtures containing malformed archives, excessive nesting, duplicate component names, and intentionally ambiguous matches.

Expected result:

  • Resource limits are enforced.
  • Invalid inputs receive actionable diagnostics.
  • Cancellation and cleanup behave predictably.
  • Original artifacts remain unchanged.
  • No firmware execution or network interaction occurs.
  • Unsupported or incomplete analysis is reported accurately.

8. Illustrative Expected Output

The following is a proposed result, not actual REA output or an approved API schema.

{
  "operation": "firmware_comparison",
  "summary": {
    "added": 1,
    "removed": 1,
    "modified": 1,
    "unchanged": 1
  },
  "coverage": {
    "status": "complete",
    "unknowns": []
  },
  "verification": {
    "fixture_type": "synthetic",
    "expected_classifications": 4,
    "status": "not_executed"
  }
}

9. Proposed Acceptance Checks

  • All four known component changes are correctly classified.
  • Identical components remain unchanged across repackaged containers.
  • Component findings include valid source and Evidence references.
  • Unsupported components produce explicit incomplete coverage.
  • Equivalent CLI and MCP requests return consistent results.
  • Malformed inputs and resource limits are handled safely.
  • Original firmware artifacts remain unmodified.
  • Automated regression tests and documentation are provided.

10. Maintainer Feedback Requested

I would appreciate guidance on:

  1. The appropriate existing REA Evidence representation.
  2. The smallest useful initial implementation scope.
  3. Supported fixture formats for initial verification.
  4. Component identity and matching rules.
  5. Required CLI/MCP contract and regression tests.

Verification declaration: All checks above are proposed and remain unexecuted. No successful results, performance improvements, or implementation readiness are claimed.

Alternatives considered

Alternatives Considered

The following approaches were considered conceptually. They have not been independently benchmarked or tested against the proposed workflow.

1. Existing REA Firmware Inspection

Approach: Use REA's existing firmware-oriented inspection and artifact analysis capabilities to examine baseline and candidate firmware packages separately.

Limitations:

  • Requires additional correlation between independent analysis results.
  • Does not establish a single, verified component-level comparison workflow.
  • May require manual matching of component identities and source locations.
  • Does not inherently provide a unified change inventory with cross-version Evidence references.

Conclusion: Existing capabilities should be reused as the foundation rather than replaced.

2. Existing REA Evidence and Comparison Operations

Approach: Use existing Evidence export, import, and comparison functionality to evaluate retained analysis results.

Limitations:

  • Evidence-level differences do not necessarily establish component-level firmware changes.
  • Package repackaging can complicate interpretation of differences.
  • Component identity matching and ambiguity handling require explicit rules.
  • Firmware-specific coverage and unknowns need a defined comparison contract.

Conclusion: Extend or compose existing Evidence workflows where practical, preserving compatibility with current contracts.

3. Manual Hash-Based Comparison

Approach: Extract recognized firmware components and compare their SHA-256 hashes using scripts or command-line utilities.

Limitations:

  • Requires separate extraction and component-matching steps.
  • Whole-package hashes cannot identify individual changed components.
  • File relocation and duplicate names complicate matching.
  • Evidence provenance and incomplete coverage require additional manual handling.

Conclusion: Useful for basic verification, but insufficient as a unified evidence-backed workflow.

4. External Firmware Analysis Tools

Approach: Use established tools such as Binwalk, Unblob, or FACT for supported firmware extraction and inspection workflows.

Limitations:

  • Capabilities, supported formats, and dependencies differ between tools.
  • Outputs may require normalization before reliable cross-version comparison.
  • Component matching and Evidence provenance must still be coordinated.
  • Introducing additional dependencies may increase maintenance and integration costs.

Conclusion: These tools may provide useful optional capabilities, but their adoption should follow REA's existing provider and contribution-scope decisions.

5. Custom Comparison Scripts

Approach: Build a standalone Python or shell workflow to extract, hash, match, and classify firmware components.

Limitations:

  • Duplicates workflow logic outside REA.
  • Requires independent handling of errors, limits, and ambiguous matches.
  • Creates additional maintenance and testing responsibilities.
  • Does not automatically preserve consistent REA CLI/MCP Evidence contracts.

Conclusion: Suitable as a temporary workaround or reference implementation, but less desirable than a reusable capability integrated with existing REA architecture.

Preferred Approach

Implement the smallest maintainable, evidence-backed firmware comparison workflow by composing existing REA inspection, artifact, and Evidence capabilities.

The initial scope should prioritize:

  1. Deterministic component matching.
  2. Accurate change classification.
  3. Traceable source Evidence.
  4. Explicit incomplete-coverage reporting.
  5. Consistent CLI/MCP results.
  6. Safe, bounded, read-only target interaction.
  7. Compatibility with existing REA contracts.
  • #718 — IoT firmware analysis tools and roadmap research.
  • #4110 — Unblob, FACT Core, and Binwalk-NG scope.
  • #5851 — Firmware analysis budgets.

These discussions may overlap with implementation choices but do not, by themselves, establish the requested end-to-end component-level firmware comparison behavior.

Final Consideration

A standalone implementation should not be introduced if existing REA operations can already provide the desired outcome through a supported composition.

Maintainer feedback is requested to identify reusable functionality, confirm any actual capability gap, and determine whether a new operation, an extension to an existing workflow, or documentation of an existing approach is most appropriate.

Evaluation status: Conceptual alternatives review only. No comparative benchmark, confirmed performance advantage, or production validation is claimed.

Possible direction

Possible Implementation Direction

1. Architecture-First Integration

Consider implementing firmware comparison as a bounded, reusable application workflow built on REA's existing firmware inspection, artifact handling, and Evidence infrastructure.

The preferred direction is to compose existing capabilities rather than introduce a separate analysis engine or duplicate provider functionality.

The architecture should preserve REA's established validation, resource ownership, error handling, and CLI/MCP contracts.

2. Proposed Processing Pipeline

A possible workflow consists of six stages:

Stage 1 — Input Admission

  • Accept baseline and candidate firmware artifacts.
  • Validate input paths, accessibility, and supported formats.
  • Establish explicit resource budgets and processing boundaries.
  • Preserve original artifact integrity.

Stage 2 — Component Inventory

  • Reuse supported static inspection and extraction capabilities.
  • Identify recognized components and available metadata.
  • Record component paths, offsets, formats, sizes, and hashes.
  • Preserve source-artifact provenance.

Stage 3 — Deterministic Matching

  • Match components using documented identity rules.
  • Prioritize reliable component identities and cryptographic hashes.
  • Distinguish exact matches from ambiguous associations.
  • Avoid treating filename similarity as proof of equivalence.

Stage 4 — Change Classification

  • Identify added components.
  • Identify removed components.
  • Identify byte-level modifications.
  • Identify unchanged components.
  • Report unresolved comparisons separately.

Stage 5 — Evidence Construction

  • Generate structured component-level comparison Evidence.
  • Preserve baseline and candidate artifact references.
  • Record matching criteria and classification reasons.
  • Represent unsupported components and incomplete coverage explicitly.

Stage 6 — Result Delivery

  • Return consistent machine-readable results.
  • Expose the workflow through supported CLI and MCP interfaces.
  • Support bounded summaries for large component inventories.
  • Preserve actionable diagnostics and recovery guidance.

3. Suggested Internal Design

The following names are conceptual responsibilities, not required classes, modules, or public API names.

Responsibility Proposed Function
Input admission Validate selected firmware artifacts
Inventory builder Collect recognized components
Identity matcher Establish cross-version matches
Change classifier Determine component differences
Evidence builder Preserve provenance and coverage
Result presenter Deliver consistent CLI/MCP results

Maintainers should determine whether these responsibilities belong in existing application services, domain modules, or a narrowly scoped new component.

4. Suggested Incremental Delivery

Phase 1 — Design and Contracts

  • Confirm the capability gap and contribution scope.
  • Identify reusable REA operations.
  • Define matching rules and Evidence representation.
  • Establish explicit non-goals.

Phase 2 — Minimal Comparison Workflow

  • Support already-recognized component inventories.
  • Implement deterministic hash-based comparisons.
  • Classify added, removed, modified, and unchanged components.
  • Preserve source references.

Phase 3 — Integration

  • Add appropriate CLI and MCP access.
  • Reuse existing output and error contracts.
  • Implement resource limits, cancellation, and cleanup.

Phase 4 — Verification

  • Introduce self-contained synthetic fixtures.
  • Test classification correctness and ambiguity handling.
  • Verify CLI/MCP consistency.
  • Document limitations and recovery behavior.

5. Design Principles

The implementation should prioritize:

  • Correctness: Never report inferred changes as verified facts.
  • Determinism: Equivalent inputs and settings produce equivalent results.
  • Traceability: Findings remain connected to original artifacts.
  • Compatibility: Existing REA contracts remain authoritative.
  • Safety: No target execution, flashing, network access, or protection bypass.
  • Performance: Bounded resource consumption and reuse of existing analysis.
  • Maintainability: Minimal duplication and clear ownership boundaries.
  • Transparency: Unsupported and unresolved cases remain visible.

6. Alternative Integration Strategy

If existing REA operations can already produce suitable component inventories and Evidence, the initial implementation could be a thin comparison layer over those outputs.

This would avoid introducing additional firmware parsers, extraction engines, or external dependencies.

A new public command or MCP tool should be introduced only if existing interfaces cannot express the workflow cleanly.

7. Maintainer Decisions Requested

Before implementation, I would appreciate feedback on:

  1. The most appropriate existing REA integration point.
  2. Whether comparison should initially operate on retained component inventories.
  3. The preferred component identity and matching contract.
  4. Whether a new operation is necessary.
  5. The minimum Evidence and coverage requirements.
  6. The smallest acceptable initial verification scope.

Final Recommendation

Begin with a minimal, static, evidence-backed comparison of already-supported firmware components.

Expand supported formats, matching strategies, and reporting capabilities only when justified by verified use cases and maintainer review.

Implementation status: Conceptual proposal only. No source changes, completed prototype, executed tests, or performance measurements are claimed. The implementation approach remains subject to maintainer discussion and approval.

References

References and Supporting Resources

1. REA Repository and Contribution Guidelines

REA — Official Repository
https://github.com/morluto/rea

Primary reference for REA's architecture, existing capabilities, documentation, and development conventions.

Contribution Guidelines — Contribution Scope
https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope

Defines the contribution boundaries relevant to static inspection, artifact processing, reverse engineering, Evidence preservation, and runtime authority.

REA Tool Design Guide
https://github.com/morluto/rea/blob/main/docs/tool-design.md

Relevant to CLI/MCP consistency, input validation, structured results, and integration contracts.

REA CLI Documentation
https://github.com/morluto/rea/blob/main/docs/cli.md

Reference for existing command-line workflows and output conventions.

#718 — Integrate IoT firmware analysis tools into REA: research and roadmap
https://github.com/morluto/rea/issues/718

Related to broader firmware-analysis capability planning. The current proposal focuses specifically on comparing two firmware releases and preserving component-level change Evidence.

#4110 — Record scope for Unblob, FACT Core, and Binwalk-NG
https://github.com/morluto/rea/issues/4110

Relevant to supported extraction backends, dependency decisions, and firmware-analysis integration boundaries.

#5851 — Expose firmware analysis budgets independently of publication recovery
https://github.com/morluto/rea/issues/5851

Relevant to resource limits, bounded processing, and recovery behavior.

#740 — Roadmap: clarify investigation ownership and modularize REA capability integration
https://github.com/morluto/rea/issues/740

Relevant to architectural ownership and the reuse of existing capabilities.

These references indicate related design work; they do not establish that the requested firmware comparison workflow has already been implemented.

3. External Tools and Prior Art

Binwalk
https://github.com/ReFirmLabs/binwalk

Established firmware inspection and embedded-content identification tool. Relevant as prior art for recognizing firmware structures and embedded artifacts.

Unblob
https://github.com/onekey-sec/unblob

Extraction-oriented framework for supported archive and firmware formats. Relevant to component discovery and extraction boundaries.

FACT — Firmware Analysis and Comparison Tool
https://github.com/fkie-cad/FACT_core

Firmware-analysis framework relevant to firmware inspection, component inventories, and comparison-oriented research.

Ghidra
https://github.com/NationalSecurityAgency/ghidra

Software reverse-engineering framework relevant to deeper static analysis of recognized executable components.

These tools are references for possible workflows and design considerations. This proposal does not require REA to adopt, replace, or automatically execute them.

4. Integrity and Reproducibility References

NIST FIPS 180-4 — Secure Hash Standard
https://csrc.nist.gov/pubs/fips/180-4/upd1/final

Reference for SHA-256 and other standardized cryptographic hash functions.

Component hashes can support byte-level identity verification, but matching hashes or filenames should not be presented as proof of semantic or behavioral equivalence.

5. Proposed Verification Artifacts

The following self-owned synthetic fixtures could be created during implementation:

  • firmware-baseline.bin
  • firmware-candidate.bin
  • expected-changes.json
  • comparison-result.json

They would provide independently known component differences and expected classifications.

No such fixtures are attached to this request, and no test execution is claimed.

6. Evidence and Measurement Status

Currently available:

  • A documented feature proposal.
  • Representative use cases.
  • Proposed component matching and classification behavior.
  • Suggested verification scenarios.
  • Related repository issues and external prior art.

Not currently available:

  • A working implementation or prototype.
  • Executed REA comparison tests.
  • Measured processing-time improvements.
  • Memory or CPU benchmarks.
  • Production-scale validation.
  • Verified compatibility across firmware formats.

7. Expected Benefits — Not Yet Measured

Potential benefits include:

  1. Reduced manual component-correlation work.
  2. More reproducible firmware comparisons.
  3. Improved Evidence traceability.
  4. Consistent structured results for CLI/MCP consumers.
  5. Clearer reporting of unsupported and ambiguous components.
  6. Reuse of existing REA inspection and Evidence capabilities.

These are expected outcomes, not observed performance or accuracy improvements.

8. Maintainer Review

The references are provided to support technical discussion and help identify reusable capabilities.

I welcome maintainer guidance on existing REA functionality, architectural fit, supported initial formats, verification requirements, and whether the proposed capability should be implemented as a new operation or an extension of an existing workflow.

Reference status: Supporting documentation and conceptual prior art only. No benchmark, implementation, or experimental result is represented as completed.

Originally created by @krishanth7 on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7461 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation ### Current Limitation Reverse-engineering analysts often need to compare two versions of a firmware package to determine which internal components have been added, removed, modified, or left unchanged. Although REA provides firmware-oriented inspection and evidence workflows, there is no clearly established, unified workflow for evidence-backed, component-level comparison between two firmware releases. Today, analysts may need to inspect each package separately, manually correlate extracted components, compare file hashes, and reconstruct the differences using external scripts or tools. This creates several challenges: - **Fragmented analysis:** Component identification and comparison require multiple manual steps. - **Limited change attribution:** Whole-package hash differences cannot identify which internal components actually changed. - **Evidence traceability:** Findings must be manually linked to their original firmware artifacts. - **Reproducibility:** Independent analysts may use different matching rules and produce inconsistent results. - **Incomplete coverage:** Encrypted, unsupported, or partially extracted components need explicit reporting rather than assumptions. - **Automation overhead:** CLI and MCP consumers lack a single structured comparison result for downstream workflows. ### Example Scenario An analyst receives two authorized firmware packages: - `firmware-v1.0.bin` — Baseline release - `firmware-v1.1.bin` — Updated release The analyst wants to identify: 1. Components added or removed between releases. 2. Executables, libraries, and configuration files that changed. 3. Components whose extracted bytes remain identical despite package-level differences. 4. Cryptographic hashes and source locations supporting each finding. 5. Components that could not be reliably inspected or matched. ### Desired Improvement A bounded, deterministic firmware comparison workflow that reuses REA's existing analysis capabilities and produces structured, verifiable Evidence with component-level change attribution, explicit coverage limitations, and consistent CLI/MCP results. The proposed workflow should not execute firmware, modify source artifacts, or bypass security protections. **Note:** This is a proposed capability, not a claim that every underlying operation is currently unsupported or that a particular failure has been reproduced. ### Use cases ### Use Cases #### 1. Firmware Version Comparison Analysts comparing two authorized firmware releases need to identify added, removed, modified, and unchanged internal components without relying exclusively on whole-package binary differences. **Practical impact:** Reduces manual comparison work and improves the traceability of release-to-release changes. #### 2. Embedded Systems and IoT Research Researchers investigating embedded Linux devices, network appliances, and IoT firmware need to compare extracted executables, libraries, configuration files, and other recognized artifacts across firmware versions. **Practical impact:** Helps researchers identify relevant component changes and prioritize further investigation. #### 3. Software Supply Chain Verification Engineering and security teams need to verify whether specific firmware components have changed between approved baseline and candidate releases. **Practical impact:** Provides reproducible component hashes, change classifications, and source-artifact references for technical review. #### 4. Automated CLI and MCP Workflows Developers and AI-assisted reverse-engineering workflows need structured comparison results that can be consumed programmatically. **Practical impact:** Enables consistent downstream reporting, automated change summaries, and evidence-based investigation without manually correlating independent inspection outputs. #### 5. Evidence Preservation and Auditability Analysts need to retain the relationship between each reported change and the original firmware package, including component identity, source location, cryptographic hashes, and analysis limitations. **Practical impact:** Improves reproducibility, auditability, and confidence in technical findings. ### Representative Workflow 1. Select two locally available firmware packages. 2. Inspect supported package structures and components. 3. Match recognized components using deterministic criteria. 4. Classify verified additions, removals, modifications, and unchanged components. 5. Report unsupported or ambiguous components as unknown. 6. Generate structured comparison Evidence for CLI/MCP consumers. ### Frequency and Measurements These are representative proposed use cases, not claims of observed production usage. No benchmark results, measured time savings, user-frequency statistics, or production-scale performance measurements are available for this proposal. Initial validation should use synthetic, independently verifiable firmware fixtures and explicitly report supported formats, resource consumption, comparison accuracy, and coverage limitations. ### Desired outcome and success criteria ## Desired Outcome Introduce a reliable, deterministic, evidence-backed firmware comparison capability within REA that enables analysts to compare two authorized, locally supplied firmware releases and identify component-level changes through a consistent, reproducible workflow. The feature should integrate with REA's existing inspection, artifact, and Evidence capabilities while preserving compatibility with CLI and MCP interfaces. ### 1. Expected Functional Behavior The proposed capability should: - Accept two explicitly selected firmware artifacts representing baseline and candidate releases. - Inspect supported firmware containers and their recognized internal components. - Identify added, removed, modified, and unchanged components. - Distinguish package-level differences from verified component-level changes. - Match components using deterministic, documented identification rules. - Preserve cryptographic hashes and source-artifact references. - Report ambiguous matches, unsupported formats, encrypted components, and incomplete inspections explicitly. - Produce consistent, machine-readable results through CLI and MCP interfaces. - Operate within documented resource limits without modifying or executing the supplied firmware. ### 2. Expected Output A successful comparison should produce a structured report containing: | Field | Expected Result | |---|---| | Baseline artifact | Identity and SHA-256 hash | | Candidate artifact | Identity and SHA-256 hash | | Component identity | Recognized path or stable identifier | | Change classification | Added, removed, modified, or unchanged | | Comparison basis | Documented matching criteria | | Component hashes | Baseline and candidate hashes where applicable | | Evidence references | Traceable source-artifact locations | | Coverage | Complete, partial, or unsupported | | Unknowns | Explicit unresolved or uninspected components | | Diagnostics | Actionable errors and recovery guidance | #### Illustrative JSON Output The following represents a proposed output structure, not an existing REA API contract. ```json { "operation": "firmware_comparison", "baseline": { "artifact": "firmware-v1.0.bin", "sha256": "<baseline-sha256>" }, "candidate": { "artifact": "firmware-v1.1.bin", "sha256": "<candidate-sha256>" }, "summary": { "added": 1, "removed": 1, "modified": 1, "unchanged": 1 }, "components": [ { "identity": "rootfs/usr/bin/example", "status": "modified", "matching_basis": "recognized_path_and_format", "baseline_sha256": "<old-sha256>", "candidate_sha256": "<new-sha256>", "evidence_refs": [ "<baseline-evidence-ref>", "<candidate-evidence-ref>" ] } ], "coverage": { "status": "partial", "unknowns": [ "unsupported_nested_payload" ] } } ``` The final schema, command names, and Evidence representation should follow REA's existing architectural and compatibility contracts. ## Success Criteria The feature should be considered successful when the following acceptance criteria are satisfied. - [ ] **SC-01 — Input validation:** Both firmware inputs are validated before analysis, with clear diagnostics for invalid, inaccessible, or unsupported inputs. - [ ] **SC-02 — Accurate classification:** Synthetic test fixtures containing added, removed, modified, and unchanged components produce the expected classifications. - [ ] **SC-03 — Repackaging awareness:** Different outer package bytes do not automatically cause identical extracted components to be classified as modified. - [ ] **SC-04 — Deterministic results:** Repeated comparisons of identical inputs and settings produce equivalent component identities, classifications, and Evidence references. - [ ] **SC-05 — Evidence traceability:** Every reported component-level finding can be traced to the corresponding original artifact and supporting comparison evidence. - [ ] **SC-06 — Honest coverage:** Unsupported, encrypted, ambiguous, or partially inspected components are reported explicitly without unsupported conclusions. - [ ] **SC-07 — CLI/MCP consistency:** Equivalent CLI and MCP requests produce semantically consistent comparison results. - [ ] **SC-08 — Resource safety:** Processing respects documented memory, execution-time, extraction-depth, cancellation, and cleanup boundaries. - [ ] **SC-09 — Input integrity:** Original firmware artifacts remain unchanged. No firmware execution, device flashing, network activity, or protection bypass occurs. - [ ] **SC-10 — Error recovery:** Malformed packages, unavailable components, and resource-limit failures produce actionable diagnostics while preserving valid findings where supported. - [ ] **SC-11 — Automated verification:** Tests cover ordinary comparisons, duplicate component identities, ambiguous matches, nested archives, unsupported formats, and interrupted operations. - [ ] **SC-12 — Documentation:** User documentation explains supported formats, comparison rules, limitations, example workflows, and the meaning of coverage and Evidence fields. ## Proposed Validation Approach Use small, self-contained synthetic firmware fixtures with independently known component inventories and expected differences. Validate: 1. Correctness of component classification. 2. Stability of deterministic matching. 3. Integrity and traceability of Evidence. 4. Consistency between CLI and MCP outputs. 5. Safe behavior for malformed and unsupported inputs. 6. Resource-boundary enforcement and cleanup. Performance measurements should be collected during implementation rather than assumed in advance. ## Scope and Compatibility The initial implementation should prioritize supported static inspection and comparison capabilities. It should not introduce automatic firmware execution, credential extraction, security-control bypass, device modification, or uncontrolled external tool execution. Existing REA Evidence contracts, provider boundaries, input-validation policies, and compatibility requirements should remain authoritative. ## Definition of Done The feature is complete when the agreed initial scope passes its automated verification, produces accurate and reproducible component-level comparison Evidence, preserves input integrity, documents known limitations, and receives the required maintainer review. **Proposal status:** Requested capability only. No implementation, successful tests, performance improvements, or maintainer approval are claimed. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) ## Target and Scope Details ### Target Platforms The proposed feature targets authorized, locally available firmware artifacts used in embedded systems and IoT environments. Representative targets include: - Embedded Linux firmware packages. - IoT device firmware images. - Network appliance firmware. - Router and gateway firmware. - Industrial embedded-controller firmware. - Firmware archives containing recognized executables, libraries, configuration files, and filesystem components. These are potential use cases, not a claim that REA currently supports every listed format or platform. ### Expected Target Interaction **Selected effect:** Read existing files or captured data. The workflow should perform static, read-only analysis of two explicitly supplied firmware artifacts. Expected operations include: 1. Reading and validating baseline and candidate firmware files. 2. Inspecting supported container structures and component metadata. 3. Comparing recognized components using deterministic matching rules. 4. Calculating cryptographic hashes where supported. 5. Producing component-level change classifications. 6. Preserving source references and reporting incomplete analysis. The original firmware artifacts must remain unchanged. ### Scope Boundaries The initial implementation should: - Reuse existing REA inspection and Evidence capabilities. - Operate only on explicitly selected local inputs. - Avoid executing firmware or launching target processes. - Avoid attaching debuggers or injecting instrumentation. - Avoid changing process memory, registers, or target configurations. - Avoid contacting network services. - Avoid bypassing encryption, signatures, or other security protections. - Respect existing resource limits and cleanup requirements. ### Output and File Handling The preferred initial workflow is read-only with respect to source artifacts. Machine-readable comparison results should be returned through existing CLI/MCP output mechanisms. Whether optional report export, temporary extraction, or persistent artifact storage requires additional file-write effects should be decided during design review. ### Non-Goals - Firmware flashing or device modification. - Runtime instrumentation or dynamic execution. - Exploit development or protection bypass. - Credential extraction. - Automated vulnerability exploitation. - Unsupported claims of semantic or behavioral equivalence. - Unrestricted recursive extraction of unknown formats. ### Open Scope Questions 1. Should the first release support only firmware formats already recognized by REA? 2. Should comparison initially operate on existing extracted component inventories? 3. How should ambiguous component identities and renamed files be represented? 4. Should report export be a separate optional capability? 5. What resource limits should apply to nested firmware containers? ### Initial Implementation Preference Start with a narrowly scoped, deterministic, static comparison workflow using existing REA capabilities and synthetic test fixtures. Expand format coverage only after the initial Evidence contracts, comparison accuracy, resource safety, and maintainer-approved scope are established. **Status:** Proposed design scope; no implementation or platform-specific verification is claimed. ### Verification available so far Idea only; I need help defining checks ### Verification details (optional) ## Verification Details ### Verification Status **Current status:** Design proposal only. No working prototype, executed test suite, benchmark results, or production validation is available at this stage. The following verification scenarios and expected results are proposed for discussion with REA maintainers. ### 1. Proposed Test Environment Use self-generated, synthetic firmware-like archives containing known files and deterministic modifications. Suggested fixtures: - `firmware-baseline.bin` — Baseline package. - `firmware-candidate.bin` — Updated package. - `expected-changes.json` — Independently defined expected results. The fixtures should contain no proprietary firmware, credentials, private data, or third-party protected content. ### 2. Component Classification Test Construct two supported firmware-like packages with the following differences: | Component | Baseline | Candidate | Expected | |---|---|---|---| | `bin/app` | Version A | Version B | Modified | | `lib/core.so` | Present | Present, identical bytes | Unchanged | | `etc/legacy.conf` | Present | Absent | Removed | | `etc/new.conf` | Absent | Present | Added | **Expected result:** REA correctly classifies all four components and provides traceable source references. ### 3. Repackaging Test Create two packages containing identical supported components but different outer container metadata or compression settings. **Expected result:** Package-level hashes may differ, while verified identical extracted components remain classified as unchanged. ### 4. Evidence Integrity Test Verify that each reported component change includes: - Source artifact identity. - Baseline and candidate component hashes where applicable. - Deterministic matching basis. - Relevant source paths or offsets. - Evidence references. - Explicit analysis coverage and unknowns. **Expected result:** Findings are reproducible and traceable to their original inputs. ### 5. Unsupported Component Test Include an unsupported or intentionally unreadable nested payload in a synthetic fixture. **Expected result:** REA reports the component as unresolved or unsupported without inventing a change classification or claiming complete coverage. ### 6. CLI and MCP Consistency Test Submit equivalent comparison requests through the proposed CLI and MCP interfaces. **Expected result:** Both interfaces return semantically equivalent component classifications, evidence references, and coverage information. ### 7. Resource and Safety Test Use synthetic fixtures containing malformed archives, excessive nesting, duplicate component names, and intentionally ambiguous matches. **Expected result:** - Resource limits are enforced. - Invalid inputs receive actionable diagnostics. - Cancellation and cleanup behave predictably. - Original artifacts remain unchanged. - No firmware execution or network interaction occurs. - Unsupported or incomplete analysis is reported accurately. ### 8. Illustrative Expected Output The following is a proposed result, not actual REA output or an approved API schema. ```json { "operation": "firmware_comparison", "summary": { "added": 1, "removed": 1, "modified": 1, "unchanged": 1 }, "coverage": { "status": "complete", "unknowns": [] }, "verification": { "fixture_type": "synthetic", "expected_classifications": 4, "status": "not_executed" } } ``` ### 9. Proposed Acceptance Checks - [ ] All four known component changes are correctly classified. - [ ] Identical components remain unchanged across repackaged containers. - [ ] Component findings include valid source and Evidence references. - [ ] Unsupported components produce explicit incomplete coverage. - [ ] Equivalent CLI and MCP requests return consistent results. - [ ] Malformed inputs and resource limits are handled safely. - [ ] Original firmware artifacts remain unmodified. - [ ] Automated regression tests and documentation are provided. ### 10. Maintainer Feedback Requested I would appreciate guidance on: 1. The appropriate existing REA Evidence representation. 2. The smallest useful initial implementation scope. 3. Supported fixture formats for initial verification. 4. Component identity and matching rules. 5. Required CLI/MCP contract and regression tests. **Verification declaration:** All checks above are proposed and remain unexecuted. No successful results, performance improvements, or implementation readiness are claimed. ### Alternatives considered ## Alternatives Considered The following approaches were considered conceptually. They have not been independently benchmarked or tested against the proposed workflow. ### 1. Existing REA Firmware Inspection **Approach:** Use REA's existing firmware-oriented inspection and artifact analysis capabilities to examine baseline and candidate firmware packages separately. **Limitations:** - Requires additional correlation between independent analysis results. - Does not establish a single, verified component-level comparison workflow. - May require manual matching of component identities and source locations. - Does not inherently provide a unified change inventory with cross-version Evidence references. **Conclusion:** Existing capabilities should be reused as the foundation rather than replaced. ### 2. Existing REA Evidence and Comparison Operations **Approach:** Use existing Evidence export, import, and comparison functionality to evaluate retained analysis results. **Limitations:** - Evidence-level differences do not necessarily establish component-level firmware changes. - Package repackaging can complicate interpretation of differences. - Component identity matching and ambiguity handling require explicit rules. - Firmware-specific coverage and unknowns need a defined comparison contract. **Conclusion:** Extend or compose existing Evidence workflows where practical, preserving compatibility with current contracts. ### 3. Manual Hash-Based Comparison **Approach:** Extract recognized firmware components and compare their SHA-256 hashes using scripts or command-line utilities. **Limitations:** - Requires separate extraction and component-matching steps. - Whole-package hashes cannot identify individual changed components. - File relocation and duplicate names complicate matching. - Evidence provenance and incomplete coverage require additional manual handling. **Conclusion:** Useful for basic verification, but insufficient as a unified evidence-backed workflow. ### 4. External Firmware Analysis Tools **Approach:** Use established tools such as Binwalk, Unblob, or FACT for supported firmware extraction and inspection workflows. **Limitations:** - Capabilities, supported formats, and dependencies differ between tools. - Outputs may require normalization before reliable cross-version comparison. - Component matching and Evidence provenance must still be coordinated. - Introducing additional dependencies may increase maintenance and integration costs. **Conclusion:** These tools may provide useful optional capabilities, but their adoption should follow REA's existing provider and contribution-scope decisions. ### 5. Custom Comparison Scripts **Approach:** Build a standalone Python or shell workflow to extract, hash, match, and classify firmware components. **Limitations:** - Duplicates workflow logic outside REA. - Requires independent handling of errors, limits, and ambiguous matches. - Creates additional maintenance and testing responsibilities. - Does not automatically preserve consistent REA CLI/MCP Evidence contracts. **Conclusion:** Suitable as a temporary workaround or reference implementation, but less desirable than a reusable capability integrated with existing REA architecture. ## Preferred Approach Implement the smallest maintainable, evidence-backed firmware comparison workflow by composing existing REA inspection, artifact, and Evidence capabilities. The initial scope should prioritize: 1. Deterministic component matching. 2. Accurate change classification. 3. Traceable source Evidence. 4. Explicit incomplete-coverage reporting. 5. Consistent CLI/MCP results. 6. Safe, bounded, read-only target interaction. 7. Compatibility with existing REA contracts. ## Related Proposals - #718 — IoT firmware analysis tools and roadmap research. - #4110 — Unblob, FACT Core, and Binwalk-NG scope. - #5851 — Firmware analysis budgets. These discussions may overlap with implementation choices but do not, by themselves, establish the requested end-to-end component-level firmware comparison behavior. ## Final Consideration A standalone implementation should not be introduced if existing REA operations can already provide the desired outcome through a supported composition. Maintainer feedback is requested to identify reusable functionality, confirm any actual capability gap, and determine whether a new operation, an extension to an existing workflow, or documentation of an existing approach is most appropriate. **Evaluation status:** Conceptual alternatives review only. No comparative benchmark, confirmed performance advantage, or production validation is claimed. ### Possible direction ## Possible Implementation Direction ### 1. Architecture-First Integration Consider implementing firmware comparison as a bounded, reusable application workflow built on REA's existing firmware inspection, artifact handling, and Evidence infrastructure. The preferred direction is to compose existing capabilities rather than introduce a separate analysis engine or duplicate provider functionality. The architecture should preserve REA's established validation, resource ownership, error handling, and CLI/MCP contracts. ### 2. Proposed Processing Pipeline A possible workflow consists of six stages: **Stage 1 — Input Admission** - Accept baseline and candidate firmware artifacts. - Validate input paths, accessibility, and supported formats. - Establish explicit resource budgets and processing boundaries. - Preserve original artifact integrity. **Stage 2 — Component Inventory** - Reuse supported static inspection and extraction capabilities. - Identify recognized components and available metadata. - Record component paths, offsets, formats, sizes, and hashes. - Preserve source-artifact provenance. **Stage 3 — Deterministic Matching** - Match components using documented identity rules. - Prioritize reliable component identities and cryptographic hashes. - Distinguish exact matches from ambiguous associations. - Avoid treating filename similarity as proof of equivalence. **Stage 4 — Change Classification** - Identify added components. - Identify removed components. - Identify byte-level modifications. - Identify unchanged components. - Report unresolved comparisons separately. **Stage 5 — Evidence Construction** - Generate structured component-level comparison Evidence. - Preserve baseline and candidate artifact references. - Record matching criteria and classification reasons. - Represent unsupported components and incomplete coverage explicitly. **Stage 6 — Result Delivery** - Return consistent machine-readable results. - Expose the workflow through supported CLI and MCP interfaces. - Support bounded summaries for large component inventories. - Preserve actionable diagnostics and recovery guidance. ### 3. Suggested Internal Design The following names are conceptual responsibilities, not required classes, modules, or public API names. | Responsibility | Proposed Function | |---|---| | Input admission | Validate selected firmware artifacts | | Inventory builder | Collect recognized components | | Identity matcher | Establish cross-version matches | | Change classifier | Determine component differences | | Evidence builder | Preserve provenance and coverage | | Result presenter | Deliver consistent CLI/MCP results | Maintainers should determine whether these responsibilities belong in existing application services, domain modules, or a narrowly scoped new component. ### 4. Suggested Incremental Delivery **Phase 1 — Design and Contracts** - Confirm the capability gap and contribution scope. - Identify reusable REA operations. - Define matching rules and Evidence representation. - Establish explicit non-goals. **Phase 2 — Minimal Comparison Workflow** - Support already-recognized component inventories. - Implement deterministic hash-based comparisons. - Classify added, removed, modified, and unchanged components. - Preserve source references. **Phase 3 — Integration** - Add appropriate CLI and MCP access. - Reuse existing output and error contracts. - Implement resource limits, cancellation, and cleanup. **Phase 4 — Verification** - Introduce self-contained synthetic fixtures. - Test classification correctness and ambiguity handling. - Verify CLI/MCP consistency. - Document limitations and recovery behavior. ### 5. Design Principles The implementation should prioritize: - **Correctness:** Never report inferred changes as verified facts. - **Determinism:** Equivalent inputs and settings produce equivalent results. - **Traceability:** Findings remain connected to original artifacts. - **Compatibility:** Existing REA contracts remain authoritative. - **Safety:** No target execution, flashing, network access, or protection bypass. - **Performance:** Bounded resource consumption and reuse of existing analysis. - **Maintainability:** Minimal duplication and clear ownership boundaries. - **Transparency:** Unsupported and unresolved cases remain visible. ### 6. Alternative Integration Strategy If existing REA operations can already produce suitable component inventories and Evidence, the initial implementation could be a thin comparison layer over those outputs. This would avoid introducing additional firmware parsers, extraction engines, or external dependencies. A new public command or MCP tool should be introduced only if existing interfaces cannot express the workflow cleanly. ### 7. Maintainer Decisions Requested Before implementation, I would appreciate feedback on: 1. The most appropriate existing REA integration point. 2. Whether comparison should initially operate on retained component inventories. 3. The preferred component identity and matching contract. 4. Whether a new operation is necessary. 5. The minimum Evidence and coverage requirements. 6. The smallest acceptable initial verification scope. ### Final Recommendation Begin with a minimal, static, evidence-backed comparison of already-supported firmware components. Expand supported formats, matching strategies, and reporting capabilities only when justified by verified use cases and maintainer review. **Implementation status:** Conceptual proposal only. No source changes, completed prototype, executed tests, or performance measurements are claimed. The implementation approach remains subject to maintainer discussion and approval. ### References ## References and Supporting Resources ### 1. REA Repository and Contribution Guidelines **REA — Official Repository** https://github.com/morluto/rea Primary reference for REA's architecture, existing capabilities, documentation, and development conventions. **Contribution Guidelines — Contribution Scope** https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope Defines the contribution boundaries relevant to static inspection, artifact processing, reverse engineering, Evidence preservation, and runtime authority. **REA Tool Design Guide** https://github.com/morluto/rea/blob/main/docs/tool-design.md Relevant to CLI/MCP consistency, input validation, structured results, and integration contracts. **REA CLI Documentation** https://github.com/morluto/rea/blob/main/docs/cli.md Reference for existing command-line workflows and output conventions. ### 2. Related REA Issues and Design Discussions **#718 — Integrate IoT firmware analysis tools into REA: research and roadmap** https://github.com/morluto/rea/issues/718 Related to broader firmware-analysis capability planning. The current proposal focuses specifically on comparing two firmware releases and preserving component-level change Evidence. **#4110 — Record scope for Unblob, FACT Core, and Binwalk-NG** https://github.com/morluto/rea/issues/4110 Relevant to supported extraction backends, dependency decisions, and firmware-analysis integration boundaries. **#5851 — Expose firmware analysis budgets independently of publication recovery** https://github.com/morluto/rea/issues/5851 Relevant to resource limits, bounded processing, and recovery behavior. **#740 — Roadmap: clarify investigation ownership and modularize REA capability integration** https://github.com/morluto/rea/issues/740 Relevant to architectural ownership and the reuse of existing capabilities. These references indicate related design work; they do not establish that the requested firmware comparison workflow has already been implemented. ### 3. External Tools and Prior Art **Binwalk** https://github.com/ReFirmLabs/binwalk Established firmware inspection and embedded-content identification tool. Relevant as prior art for recognizing firmware structures and embedded artifacts. **Unblob** https://github.com/onekey-sec/unblob Extraction-oriented framework for supported archive and firmware formats. Relevant to component discovery and extraction boundaries. **FACT — Firmware Analysis and Comparison Tool** https://github.com/fkie-cad/FACT_core Firmware-analysis framework relevant to firmware inspection, component inventories, and comparison-oriented research. **Ghidra** https://github.com/NationalSecurityAgency/ghidra Software reverse-engineering framework relevant to deeper static analysis of recognized executable components. These tools are references for possible workflows and design considerations. This proposal does not require REA to adopt, replace, or automatically execute them. ### 4. Integrity and Reproducibility References **NIST FIPS 180-4 — Secure Hash Standard** https://csrc.nist.gov/pubs/fips/180-4/upd1/final Reference for SHA-256 and other standardized cryptographic hash functions. Component hashes can support byte-level identity verification, but matching hashes or filenames should not be presented as proof of semantic or behavioral equivalence. ### 5. Proposed Verification Artifacts The following self-owned synthetic fixtures could be created during implementation: - `firmware-baseline.bin` - `firmware-candidate.bin` - `expected-changes.json` - `comparison-result.json` They would provide independently known component differences and expected classifications. No such fixtures are attached to this request, and no test execution is claimed. ### 6. Evidence and Measurement Status **Currently available:** - A documented feature proposal. - Representative use cases. - Proposed component matching and classification behavior. - Suggested verification scenarios. - Related repository issues and external prior art. **Not currently available:** - A working implementation or prototype. - Executed REA comparison tests. - Measured processing-time improvements. - Memory or CPU benchmarks. - Production-scale validation. - Verified compatibility across firmware formats. ### 7. Expected Benefits — Not Yet Measured Potential benefits include: 1. Reduced manual component-correlation work. 2. More reproducible firmware comparisons. 3. Improved Evidence traceability. 4. Consistent structured results for CLI/MCP consumers. 5. Clearer reporting of unsupported and ambiguous components. 6. Reuse of existing REA inspection and Evidence capabilities. These are expected outcomes, not observed performance or accuracy improvements. ### 8. Maintainer Review The references are provided to support technical discussion and help identify reusable capabilities. I welcome maintainer guidance on existing REA functionality, architectural fit, supported initial formats, verification requirements, and whether the proposed capability should be implemented as a new operation or an extension of an existing workflow. **Reference status:** Supporting documentation and conceptual prior art only. No benchmark, implementation, or experimental result is represented as completed.
Author
Owner

@bungakarunanidhi commented on GitHub (Oct 27, 2026):

I think this would be a useful addition to REA, especially for developers and security researchers who need to compare firmware versions and understand exactly what has changed between releases.

One approach could be to build on the existing inspection and Evidence workflows rather than introducing a separate analysis pipeline. Comparing component hashes, identifying added or removed files, and clearly reporting components that could not be analyzed would make the results more reliable and easier to verify.

I also think it would be worth starting with a small implementation using synthetic firmware samples with known changes. This would help validate the comparison logic and ensure that repackaging a firmware image does not incorrectly classify unchanged components as modified.

Before going further, it would be helpful to understand which existing REA capabilities can already support this workflow and where additional functionality is actually needed.

I'd be interested to hear what other contributors think about this approach and how it could fit into the existing architecture.

<!-- gh-comment-id:6435535083 --> @bungakarunanidhi commented on GitHub (Oct 27, 2026): I think this would be a useful addition to **REA**, especially for developers and security researchers who need to compare firmware versions and understand exactly what has changed between releases. One approach could be to build on the existing inspection and Evidence workflows rather than introducing a separate analysis pipeline. Comparing component hashes, identifying added or removed files, and clearly reporting components that could not be analyzed would make the results more reliable and easier to verify. I also think it would be worth starting with a small implementation using synthetic firmware samples with known changes. This would help validate the comparison logic and ensure that repackaging a firmware image does not incorrectly classify unchanged components as modified. Before going further, it would be helpful to understand which existing REA capabilities can already support this workflow and where additional functionality is actually needed. I'd be interested to hear what other contributors think about this approach and how it could fit into the existing architecture.
Author
Owner

@krishanth7 commented on GitHub (Oct 27, 2026):

One approach could be to build on the existing inspection and Evidence workflows rather than introducing a separate analysis pipeline.

Thanks for sharing your thoughts, @bungakarunanidhi. I agree that building on REA's existing capabilities makes more sense than introducing an entirely separate analysis pipeline.

My main goal with this proposal is to make firmware version comparisons easier to reproduce and verify, particularly when the outer firmware package changes but some of its internal components remain identical.

I also like your suggestion of starting with a small set of synthetic firmware samples. That would give us a practical way to test component matching, hash comparisons, and how unsupported or ambiguous results should be reported before considering a broader implementation.

I think the most useful next step would be to identify which parts of this workflow REA already supports and where the actual gaps are. That should help keep the initial scope manageable and avoid duplicating existing functionality.

Thanks again for the feedback. I'd be interested in hearing the maintainers' thoughts on the best integration point.

<!-- gh-comment-id:6435604094 --> @krishanth7 commented on GitHub (Oct 27, 2026): > One approach could be to build on the existing inspection and Evidence workflows rather than introducing a separate analysis pipeline. Thanks for sharing your thoughts, @bungakarunanidhi. I agree that building on REA's existing capabilities makes more sense than introducing an entirely separate analysis pipeline. My main goal with this proposal is to make firmware version comparisons easier to reproduce and verify, particularly when the outer firmware package changes but some of its internal components remain identical. I also like your suggestion of starting with a small set of synthetic firmware samples. That would give us a practical way to test component matching, hash comparisons, and how unsupported or ambiguous results should be reported before considering a broader implementation. I think the most useful next step would be to identify which parts of this workflow REA already supports and where the actual gaps are. That should help keep the initial scope manageable and avoid duplicating existing functionality. Thanks again for the feedback. I'd be interested in hearing the maintainers' thoughts on the best integration point.
Author
Owner

@XinwangTao commented on GitHub (Oct 27, 2026):

This proposal looks valuable, and I agree with building on the existing inspection/Evidence workflows instead of a separate pipeline. One concrete suggestion: define component identity explicitly (e.g. normalized path + size + hash) so that repackaged-but-identical components are reported as unchanged rather than falsely flagged. The SC-03 repackaging awareness checklist item already hints at this; making the matching rule part of the Evidence output would also help reproducibility across analysts.

<!-- gh-comment-id:6436757262 --> @XinwangTao commented on GitHub (Oct 27, 2026): This proposal looks valuable, and I agree with building on the existing inspection/Evidence workflows instead of a separate pipeline. One concrete suggestion: define component identity explicitly (e.g. normalized path + size + hash) so that repackaged-but-identical components are reported as unchanged rather than falsely flagged. The SC-03 repackaging awareness checklist item already hints at this; making the matching rule part of the Evidence output would also help reproducibility across analysts.
Author
Owner

@krishanth7 commented on GitHub (Oct 28, 2026):

Make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged.

Thanks for the feedback, @XinwangTao. I think that's an important distinction, especially when two firmware packages have different container hashes but their internal components haven't actually changed.

I agree that component identity needs to be clearly defined from the beginning. Using normalized paths, file sizes, and hashes seems like a good starting point, although we'd also need to handle cases where files are renamed, relocated, or have duplicate paths without making incorrect matches.

I also agree with keeping the safety boundaries explicit. The intention here is strictly static comparison, not firmware execution or behavioral analysis. Any conclusions should be backed by the available evidence, with unsupported or ambiguous results clearly identified.

Your point about CLI/MCP parity is equally important. I'd prefer both interfaces to share the same underlying comparison logic and Evidence representation rather than maintaining separate implementations.

For the initial scope, I'm leaning toward comparing components that REA can already recognize or extract, then expanding coverage once the matching rules and verification tests are established.

Thanks again for the suggestions. They help narrow this down into something more practical and maintainable.

<!-- gh-comment-id:6464647720 --> @krishanth7 commented on GitHub (Oct 28, 2026): > Make component identity explicit (normalized path + size + hash) so repackaged-but-identical components are reported as unchanged. Thanks for the feedback, @XinwangTao. I think that's an important distinction, especially when two firmware packages have different container hashes but their internal components haven't actually changed. I agree that component identity needs to be clearly defined from the beginning. Using normalized paths, file sizes, and hashes seems like a good starting point, although we'd also need to handle cases where files are renamed, relocated, or have duplicate paths without making incorrect matches. I also agree with keeping the safety boundaries explicit. The intention here is strictly static comparison, not firmware execution or behavioral analysis. Any conclusions should be backed by the available evidence, with unsupported or ambiguous results clearly identified. Your point about CLI/MCP parity is equally important. I'd prefer both interfaces to share the same underlying comparison logic and Evidence representation rather than maintaining separate implementations. For the initial scope, I'm leaning toward comparing components that REA can already recognize or extract, then expanding coverage once the matching rules and verification tests are established. Thanks again for the suggestions. They help narrow this down into something more practical and maintainable.
Author
Owner

@XinwangTao commented on GitHub (Oct 29, 2026):

关于可复现性补充一点:建议把匹配规则本身作为 Evidence 的一部分输出——normalized path 的规约规则(大小写、路径分隔符、相对/绝对路径)、hash 算法、以及同名校验策略都应记录,否则不同分析师执行同一对比可能得到不同的 added/removed 判定。另外,无匹配组件的归类建议参考 #7038 的 reader 分类思路,明确区分「确实新增/删除」与「无法可靠匹配(ambiguous)」,避免把 rename/relocate 或解析失败的组件静默并进 modified。范围控制上,可以先只做 1:1 同名组件匹配,把 rename/relocate 检测放到第二阶段。

<!-- gh-comment-id:6485323747 --> @XinwangTao commented on GitHub (Oct 29, 2026): 关于可复现性补充一点:建议把匹配规则本身作为 Evidence 的一部分输出——normalized path 的规约规则(大小写、路径分隔符、相对/绝对路径)、hash 算法、以及同名校验策略都应记录,否则不同分析师执行同一对比可能得到不同的 added/removed 判定。另外,无匹配组件的归类建议参考 #7038 的 reader 分类思路,明确区分「确实新增/删除」与「无法可靠匹配(ambiguous)」,避免把 rename/relocate 或解析失败的组件静默并进 modified。范围控制上,可以先只做 1:1 同名组件匹配,把 rename/relocate 检测放到第二阶段。
Author
Owner

@krishanth7 commented on GitHub (Oct 29, 2026):

感谢补充,@XinwangTao!你提到的这个问题确实很关键,尤其是匹配规则的可复现性。

我之前更多关注的是组件之间的差异,但仔细想想,如果没有记录路径规范化方式、哈希算法和同名组件的处理规则,即使比较结果看起来正确,其他人也很难完整复现整个过程。所以我很赞同把这些规则一起纳入 Evidence,而不只是保存最终的比较结果。

另外,关于 added、removed 和 ambiguous 的区分,我也认同你的建议。特别是遇到组件重命名、路径变化或者解析失败时,直接归类为新增、删除或修改,确实容易产生误导。对于无法确认的情况,明确标记出来会更可靠。

第一阶段先支持基于规范化路径的严格 1:1 匹配,我觉得是个很合理的起点。这样可以先把基础逻辑、Evidence 结构和测试做好,再逐步考虑 rename/relocate 检测,避免一开始就把范围扩大得太多。

我也会再看看 #7038 中的 reader 分类思路,看看有哪些设计可以借鉴。

谢谢你这么详细的建议!这些反馈对明确第一阶段的实现范围很有帮助,也让整个方案更扎实了。期待继续交流!

<!-- gh-comment-id:6486578505 --> @krishanth7 commented on GitHub (Oct 29, 2026): 感谢补充,@XinwangTao!你提到的这个问题确实很关键,尤其是匹配规则的可复现性。 我之前更多关注的是组件之间的差异,但仔细想想,如果没有记录路径规范化方式、哈希算法和同名组件的处理规则,即使比较结果看起来正确,其他人也很难完整复现整个过程。所以我很赞同把这些规则一起纳入 Evidence,而不只是保存最终的比较结果。 另外,关于 `added`、`removed` 和 `ambiguous` 的区分,我也认同你的建议。特别是遇到组件重命名、路径变化或者解析失败时,直接归类为新增、删除或修改,确实容易产生误导。对于无法确认的情况,明确标记出来会更可靠。 第一阶段先支持基于规范化路径的严格 1:1 匹配,我觉得是个很合理的起点。这样可以先把基础逻辑、Evidence 结构和测试做好,再逐步考虑 rename/relocate 检测,避免一开始就把范围扩大得太多。 我也会再看看 #7038 中的 reader 分类思路,看看有哪些设计可以借鉴。 谢谢你这么详细的建议!这些反馈对明确第一阶段的实现范围很有帮助,也让整个方案更扎实了。期待继续交流!
Author
Owner

@XinwangTao commented on GitHub (Oct 29, 2026):

补充两点供第一阶段落地参考:1) Evidence 建议同时记录外层包级 hash 与组件级 hash,这样 SC-03 重打包场景即可明确区分"外层容器变化"与"组件未变",判定链路更可审计;2) ambiguous 归类建议给一个最小判定标准(如路径归一化后仍冲突、或 hash 不可比时标记 unresolved),避免不同分析师对边界样本得出不一致的结论。第一阶段收敛为 1:1 同名组件匹配并对齐 #7038 的 reader 分类思路,我认为是合理的起点。

<!-- gh-comment-id:6487756791 --> @XinwangTao commented on GitHub (Oct 29, 2026): 补充两点供第一阶段落地参考:1) Evidence 建议同时记录外层包级 hash 与组件级 hash,这样 SC-03 重打包场景即可明确区分"外层容器变化"与"组件未变",判定链路更可审计;2) ambiguous 归类建议给一个最小判定标准(如路径归一化后仍冲突、或 hash 不可比时标记 unresolved),避免不同分析师对边界样本得出不一致的结论。第一阶段收敛为 1:1 同名组件匹配并对齐 #7038 的 reader 分类思路,我认为是合理的起点。
Author
Owner

@krishanth7 commented on GitHub (Oct 29, 2026):

感谢继续补充,@XinwangTao!这两点都很有价值,特别是外层包和内部组件分别记录 hash 的建议。

我觉得这样处理会更清楚。比如同一个固件重新打包后,整个文件的 hash 可能已经变了,但里面的组件其实完全相同。如果 Evidence 能同时保留这两个层级的 hash,就可以直接看出变化发生在哪里,也更容易追溯判断依据。这个思路也很适合用来验证 SC-03 的场景。

关于 ambiguous 和 unresolved,我也赞同应该在第一阶段就明确最基本的判定规则,而不是等到后面再补。路径归一化之后仍然存在冲突,或者缺少可靠的 hash 信息时,宁可明确标记为无法确认,也不应该直接推断组件发生了变化。

我还想到一点:除了记录最终的分类结果,最好也能保留触发该分类的具体原因。这样其他人查看 Evidence 时,不需要重新猜测当时为什么做出这个判断。

整体上,我也倾向于先把 1:1 同名组件匹配做好,并参考 #7038 的分类方式处理边界情况。等基础逻辑和测试稳定后,再考虑重命名、路径迁移等更复杂的场景。

再次感谢你的建议!这些细节对后续明确实现范围和验收标准都很有帮助。🙂

<!-- gh-comment-id:6496302384 --> @krishanth7 commented on GitHub (Oct 29, 2026): 感谢继续补充,@XinwangTao!这两点都很有价值,特别是外层包和内部组件分别记录 hash 的建议。 我觉得这样处理会更清楚。比如同一个固件重新打包后,整个文件的 hash 可能已经变了,但里面的组件其实完全相同。如果 Evidence 能同时保留这两个层级的 hash,就可以直接看出变化发生在哪里,也更容易追溯判断依据。这个思路也很适合用来验证 SC-03 的场景。 关于 `ambiguous` 和 `unresolved`,我也赞同应该在第一阶段就明确最基本的判定规则,而不是等到后面再补。路径归一化之后仍然存在冲突,或者缺少可靠的 hash 信息时,宁可明确标记为无法确认,也不应该直接推断组件发生了变化。 我还想到一点:除了记录最终的分类结果,最好也能保留触发该分类的具体原因。这样其他人查看 Evidence 时,不需要重新猜测当时为什么做出这个判断。 整体上,我也倾向于先把 1:1 同名组件匹配做好,并参考 #7038 的分类方式处理边界情况。等基础逻辑和测试稳定后,再考虑重命名、路径迁移等更复杂的场景。 再次感谢你的建议!这些细节对后续明确实现范围和验收标准都很有帮助。🙂
Author
Owner

@XinwangTao commented on GitHub (Oct 30, 2026):

赞同把触发分类的原因一并写入 Evidence,避免事后靠猜测还原判定链路。落地建议:为 classification_reason 定义固定枚举(如 matched_by_path_and_hash / repackaged_unchanged / path_collision / hash_unavailable),并在文档中逐项说明判定条件,这样 SC-04 可复现性与 SC-06 诚实覆盖都能直接落到字段层。第一阶段收敛为 1:1 同名匹配并带上该字段,我认为已经足够,rename/relocate 检测留到第二阶段也合适。

<!-- gh-comment-id:6512346906 --> @XinwangTao commented on GitHub (Oct 30, 2026): 赞同把触发分类的原因一并写入 Evidence,避免事后靠猜测还原判定链路。落地建议:为 classification_reason 定义固定枚举(如 matched_by_path_and_hash / repackaged_unchanged / path_collision / hash_unavailable),并在文档中逐项说明判定条件,这样 SC-04 可复现性与 SC-06 诚实覆盖都能直接落到字段层。第一阶段收敛为 1:1 同名匹配并带上该字段,我认为已经足够,rename/relocate 检测留到第二阶段也合适。
Author
Owner

@XinwangTao commented on GitHub (Oct 30, 2026):

赞同把触发分类的原因一并写入 Evidence,避免事后靠猜测还原判定链路。落地建议:为 classification_reason 定义固定枚举(如 matched_by_path_and_hash / repackaged_unchanged / path_collision / hash_unavailable),并在文档中逐项说明判定条件,这样 SC-04 可复现性与 SC-06 诚实覆盖都能直接落到字段层。第一阶段收敛为 1:1 同名匹配并带上该字段,我认为已经足够,rename/relocate 检测留到第二阶段也合适。

<!-- gh-comment-id:6512368140 --> @XinwangTao commented on GitHub (Oct 30, 2026): 赞同把触发分类的原因一并写入 Evidence,避免事后靠猜测还原判定链路。落地建议:为 classification_reason 定义固定枚举(如 matched_by_path_and_hash / repackaged_unchanged / path_collision / hash_unavailable),并在文档中逐项说明判定条件,这样 SC-04 可复现性与 SC-06 诚实覆盖都能直接落到字段层。第一阶段收敛为 1:1 同名匹配并带上该字段,我认为已经足够,rename/relocate 检测留到第二阶段也合适。
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#4454
No description provided.