morluto/rea
1
0
Fork 0
mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00
Code Issues 1.3k Projects Packages Wiki Activity

[GH-ISSUE #7955] [Roadmap] Embedded firmware: make REA work on the devices around us #4654

New issue
Open
opened 2026-10-31 00:52:01 +00:00 by bsakaguchi · 4 comments
bsakaguchi commented 2026-10-31 00:52:01 +00:00
Owner
Copy link

Originally created by @Entrpi on GitHub (Oct 29, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7955

Before submitting

  • I searched the open and closed issues and pull requests for a related discussion.
  • I understand the contribution scope must be reviewed before integration; open questions are welcome here.
  • This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Morluto's video shows the REA mech as it fights a TV monster that has microphone tentacles. It is a cartoon, but it points at a real question: what do the devices around us record, and where do they send it? A TV is one device. Many devices around it run on microcontrollers: voice remotes, Bluetooth microphones and earbuds, smart plugs, sensors and wearables. Their firmware usually arrives as a flash dump or a vendor update file, not as an ELF build.

On main, REA cannot take most of that firmware to readable code. Of the 26 embedded platforms in PlatformIO's top 30, four now get verified code analysis on main, all of them only partly. #8229 brought it to their RISC-V ELF builds. The open and queued PRs below bring that number to 19 platforms, five of them only partly. This roadmap is the work it takes for REA to read firmware across that landscape.

Platforms this roadmap covers, and common devices that they power:

Platform Common devices On main With the open and queued PRs
ESP32, S2 and S3 (Xtensa) Wi-Fi smart plugs, switches and bulbs, small cameras, voice assistants, maker boards App images are inspected (#7498), but their code is refused. Flash dumps are read as partitions and apps (#8228). UF2 files are read (#8230). Code is analyzed in a verified Ghidra lane (#7486).
ESP32-C3, C6, H2 and P4 (RISC-V) Newer Wi-Fi, Thread, Zigbee and Matter devices Soft-float ELF builds are analyzed under a written calling-convention policy (#8229), but in C6 and H2 builds Ghidra puts flash code in an overlay, where it loses its DWARF facts (#8461). Flash code is analyzed at its ELF address (#8464), and single-float ELF builds are analyzed too (#8401). Builds that declare B, Zc or Espressif's own extensions stay refused. Every P4 build declares Espressif's extensions (#8463).
ESP8266 Older Wi-Fi smart plugs, switches and bulbs Images are refused by design. Xtensa ELF is refused. Images are read, and call0 code is analyzed.
STM32 and other Cortex-M parts (SAMD, Renesas RA, GD32) Drone flight controllers, 3D-printer boards, e-scooter controllers, Bluetooth earbuds, industrial sensors Raw dumps decode as ARM code, with the entry point as the base (#5744). ELF analysis is unverified. DfuSe files are read (#8230). Raw images open in Thumb state at the base that the caller gives (#7951). ELF goes through a verified lane. Next: base candidates.
Nordic nRF52, nRF53 and nRF54 Fitness bands, smartwatches, item trackers, Bluetooth remotes, wireless keyboards and mice HEX files are inventoried. ELF analysis is unverified. HEX regions open as raw images (#7951). ELF goes through a verified lane. Next: Nordic DFU packages.
RP2040 and RP2350 Raspberry Pi Pico boards, custom keyboards, game controllers UF2 files are read (#8230). ELF analysis is unverified. Raw images open (#7951), and ELF goes through a verified lane. The RISC-V cores of RP2350 wait for Ghidra.
Tuya-style modules (Beken, Realtek, Bouffalo) Budget smart plugs, bulbs and switches sold under many brands LibreTiny's UF2 files are read (#8230). Realtek Cortex-M code opens through #7951. Next: Beken ARMv5TE images and Beken RBL files.
MIPS and Arm Linux Home routers, Wi-Fi access points, IP cameras Binwalk and Unblob extraction, uImage, FIT images (#7771) and the MIPS lanes. Not in this queue. Another contributor's #1922 verifies the 32-bit Arm lane.
AVR (ATmega, ATtiny) Arduino Uno and Mega boards, many hobby builds HEX files are inventoried (#7136). No AVR decoding. Not yet. Next: Harvard address spaces (D6), then an AVR lane.
8051, PIC and MSP430 Remote controls, appliance boards, chargers, meters and sensors No lane. Later: 8051 and PIC after D6, then MSP430.

In the last column, a PR number marks an open PR: #1922 belongs to another contributor, and the others are mine. Items without a number are queued on my fork, and "Next" items are planned but not built.

What I ask. Thank you for merging #7481–#7485 and #8227–#8230. Eight PRs are open, all ready for review, and none has conflicts with main. They are in the order that unblocks the most queued work:

PR Size Change What this unlocks
#7486 +3,301 −62 Xtensa ELF through a pinned Ghidra lane: ESP32 code analysis Decompiled code for ESP32, S2 and S3 ELF programs. These Wi-Fi chips run many smart plugs, cameras and voice gadgets. Next: ESP8266.
#7951 +5,605 −402 Cortex-M raw images open in Thumb state at a load base that the caller gives (D1) Correct code from raw flash dumps of STM32 and other Cortex-M chips (drones, 3D printers, earbuds), at the load base you give. Today they decode in the wrong instruction mode. Next: Cortex-M ELF.
#7487 +3,236 −156 Rizin Evidence records which tool and version answered Every Rizin result names the Rizin build behind it, so anyone can check and repeat it. Next: Rizin inputs for chips without a Ghidra lane.
#8231 +1,505 −389 One table decides how compare_artifacts compares each format_metadata kind Comparing two versions of a package or firmware no longer misses metadata changes in some formats.
#8401 +2,081 −496 RISC-V ilp32f ELF under the same calling-convention policy (D3's follow-up) Decompiled code with correct float arguments for single-float RISC-V programs, such as ESP-IDF 5.5 builds for ESP32-H4. Its H4 evidence predates #8461; it is measured again with #8464.
#8464 +2,880 −116 ESP-IDF flash code moves into Ghidra's default address space before analysis; other code overlaps are refused. Scope questions are in #8461 DWARF signatures, no-return flags and pseudocode that ends at esp_restart() for flash code of ESP32-C6, H2 and H21 builds, and of H4 with #8401. ESP-IDF 5.5.2's C5 and C61 builds, which reserve the code twice, are refused until a policy covers them.
#8462 +12 −7 The host no-return verifier's probe, updated for the current bridge with #4892's lines, crediting its author verify-real-ghidra-noreturn runs all 63 probe checks again; on main it stops after 43.
#8395 +489 −754 One base class for the ESP image, ESP flash, Godot PCK and Node SEA readers Godot PCK and Node SEA files get the fixes that the ESP readers already have for closed and cancelled reads. Later fixes land once, not four times.

If you prefer to absorb a PR yourselves, as with #7498 and #7869, that works for me. If you want a PR split, or the stages in a different order, tell me here.

Context and evidence

How each PR is built:

  • Verified against real tools. A lane is admitted only after real Ghidra 12.1.4 runs, with an LLVM disassembly oracle where one exists. ESP images are verified against esptool 4.12 and 5.6, and the Rizin route against Rizin 0.7.4 and 0.9.1. Acceptance runs used real ESP32 builds and a flash dump from a real ESP32-S3 device.
  • Observed facts only. REA does not guess load addresses, architectures or ABIs. Inferences carry a label, and limitations name what REA did not verify.
  • Protection facts stay apart. Encryption, unknown high-entropy content, a signature's presence, its cryptographic verification and device enforcement are different facts. Encrypted payloads stay opaque.
  • One owner per fact. Each vocabulary and each comparison rule lives in one table. A new format, architecture or vendor quirk is one row, not edits across many files.
  • Narrow, additive, versioned contracts. A public contract change comes with a decision record, and it merges only after you confirm the decision.
  • Licensed inputs only. Corpus images and signature sets record their licenses. Private dumps never enter them.
  • Static analysis only. No target execution, flashing, decryption or key recovery. Device access comes only after a scope decision.
The work, stage by stage

Each stage lists what is done, open or queued, then what comes next. Items run when their prerequisites are met, not in stage order. Sizes are XS to L.

1. Recognize and unpack.

  • Done:
    • Intel HEX and S-records (#7136)
    • ESP32 app images (#7498)
    • uImage (#7234)
    • U-Boot FIT images (#7771)
    • ESP flash layouts (#8228)
    • UF2 and DfuSe (#8230)
    • the Binwalk and Unblob route, verified on Linux x64
  • Open: one shared reader behind the ESP image, ESP flash, Godot PCK and Node SEA readers (#8395).
  • Queued: ESP8266 images.
  • Next:
    • STM32's plain DFU suffix (XS)
    • MCUboot (S)
    • RP2350 IMAGE_DEF blocks (S)
    • i.MX RT boot headers (S)
    • Nordic DFU packages (S–M)
    • ESP data partitions, LittleFS first (M)
    • vendor update formats by reach: Beken RBL, Silicon Labs GBL and TI OAD
  • Design first:
    • a decoded-source handoff for compressed payloads, before more readers produce them
    • delta updates, which need an identified baseline
    • TRX, scoped on its own
    • native filesystem readers, decided only after the corpus shows where extraction fails

2. Locate the code.

  • Open: raw M-profile images (#7951, D1), with a caller-supplied base and variant.
  • Next: Cortex-M base and vector-table candidates (M) that feed D1. They extend #4245's candidate idea to load addresses. They are ranked, labeled inferences that report competing explanations, coverage and abstention, and the caller chooses one.
  • Later: architecture and endianness candidates for unknown blobs (#4245), kept apart from declared header facts.

3. Load.

  • Design issues first:
    • F1, one owner for architecture families, variants and decode selection (M, a minimal first step with a consumer)
    • F2, a load plan that turns raw images, HEX regions and containers into segments with per-field provenance (L), reconciled with D1's two identity levels
    • D6, Harvard address spaces, reviewed with F2 so that F2 does not fix a model that Harvard targets cannot use
  • Then:
    • F2 for the DOL lane, then for raw images, with compatibility checks
    • HEX loading (S)
    • ESP app-image loading (M), after device descriptions and annotations
  • Throughout: related components are recorded as related components, never flattened into one address space. Examples: a bootloader and its application, a radio stack, secure and non-secure images, separate cores.

4. Analyze.

  • Done: RISC-V ilp32 (#8229).
  • Open: Xtensa ELF (#7486), RISC-V ilp32f (#8401), ESP-IDF flash code in Ghidra's default address space (#8464, for #8461) and the host no-return verifier's probe (#8462).
  • Queued:
    • the M-profile ELF lane
    • call0 Xtensa ELF
    • Rizin decoder and base-address inputs
    • the expert Ghidra language override
  • Later:
    • AVR, then 8051 and PIC, on D6
    • MSP430
    • Beken ARMv5TE raw images, which also cover MediaTek MT6261 and MT2503 modules
    • RISC-V residuals
  • Upstream:
    • Ghidra decoding for RISC-V B and Zc, which blocks RP2350's Hazard3 cores, ESP-IDF 6 builds for some ESP32 chips, CH32V and GD32VW553. NationalSecurityAgency/ghidra#8848 decodes ratified B; I tested it and posted four fixes there as a patch.
    • decoding for Espressif's own RISC-V extensions, which every ESP32-P4 build declares. #8463 proposes admitting such images with a report of what Ghidra cannot decode.
    • Armv8.1-M low-overhead loops

5. Enrich.

  • Next: ESP ROM names from caller-supplied linker scripts (S–M). Their labels do not imply that REA analyzed the ROM bytes.
  • Then: peripheral names from caller-supplied SVD files (L).
  • Pilots, design issues first:
    • SDK library identification with caller-supplied Function ID or BSim sets, tested on held-out builds, with ambiguous matches kept
    • static RTOS recognition with task-entry evidence

6. Workflows.

  • A firmware-triage guided workflow: an MCP prompt and an on-demand skill page.
  • Firmware version comparison as an end-to-end goal.
  • Both are judged on completed CLI and MCP tasks, not on documentation size.

Across all stages: a firmware corpus.

  • Contents: recipes, manifests and pinned digests, with no committed binaries. It holds self-built images for each platform group, including builds from the old toolchains that PlatformIO pins, plus licensed public images such as OpenWrt's.
  • Measures, kept apart: correct mappings, recovered known functions, ABI correctness and justified refusals. A count of found functions alone can reward a wrong load.

Next after the queue. These are work items, not single PRs:

  1. The corpus and a baseline failure report.
  2. Cortex-M base candidates, an extension of #4245.
  3. A minimal F1, with a consumer.
  4. F2 for DOL, then for raw images.
  5. HEX loading, then ESP loading.

F1, F2 and D6 start as design issues. Stage 1 items run alongside whenever foundation decisions stall.

Platform details: PlatformIO's top 10, the platforms beyond them, and where reach is highest

PlatformIO's top 10, by its search index on 29 Oct. Below 9th, its package records order some platforms differently.

# Platform Chips (cores) Default output On main Open or queued PRs Next
1 espressif32 ESP32 (Xtensa LX6); S2, S3 (LX7); C3, C6 (RISC-V); pioarduino's fork adds C2, C5, C61, H2 and P4 ESP app image App images inspected for every chip (#7498); flash layouts (#8228); UF2 (#8230); RISC-V ilp32 ELF (#8229), which accepts ESP-IDF 5.5's C-series and H2 builds, with flash code in a Ghidra overlay (#8461); Xtensa ELF refused Xtensa ELF (#7486); RISC-V ilp32f (#8401); flash code at its ELF address (#8464) ESP image loading; data partitions; NVS (#3615); ROM names. P4 builds (Espressif's own extensions, #8463) and ESP-IDF 6 builds that declare B or Zc stay refused
2 atmelavr AVR8: ATmega328P, 32U4, 2560, ATtiny Intel HEX, .eep HEX inventoried (#7136); no AVR family — D6, then an AVR ELF lane, then HEX and raw images
3 espressif8266 ESP8266 (Xtensa LX106, call0 ABI) ESP8266 v1 image Images refused by design; Xtensa ELF refused v1 images; call0 Xtensa ELF SDK split images
4 ststm32 STM32 (Cortex-M0 to M7, M33) Raw .bin ELF unverified; raw dumps decoded in ARM state; HEX inventoried; DfuSe (#8230) Raw images (#7951); M-profile ELF lane Base candidates; plain DFU suffix; MCUboot; SVD names
5 native Host OS Host executables Host binaries supported — Outside this work
6 teensy i.MX RT1062 (M7), Kinetis (M4, M0+), AVR (Teensy 2.0) Intel HEX ELF unverified; HEX inventoried M-profile ELF lane; HEX regions open as raw images (#7951) HEX loading; i.MX RT boot headers; AVR after D6
7 nordicnrf52 nRF52840, 52832, 52833 (M4) Intel HEX, merged with the SoftDevice ELF unverified; HEX inventoried; DFU .zip read as a plain ZIP M-profile ELF lane; HEX regions open as raw images (#7951) HEX loading; Nordic DFU packages; SoftDevice as a related component
8 raspberrypi RP2040 (M0+); maxgerhardt's community platform adds RP2350 (M33, or Hazard3 RISC-V) UF2 ELF unverified; UF2 (#8230), which names the RP2350 families; Hazard3 RISC-V ELF refused M-profile ELF lane; raw .bin (#7951), with the vector table after boot2 on RP2040 RP2350 IMAGE_DEF blocks. Hazard3 RISC-V waits for Ghidra to support Zba, Zbb, Zbs and Zbkb
9 atmelsam SAMD21 (M0+), SAMD51 (M4F), SAM3X (M3) Raw .bin ELF unverified; raw dumps decoded in ARM state Raw images (#7951); M-profile ELF lane Base candidates
10 renesas-ra RA4M1 (M4F, Arduino UNO R4), RA6M5 (M33) Raw .bin ELF unverified; raw dumps decoded in ARM state Raw images (#7951); M-profile ELF lane Base candidates

"ELF unverified" means that Ghidra loads Cortex-M ELF with its automatic ARM:LE:32:v8, and no lane verifies the result. "Raw dumps decoded in ARM state" is #5744: the entry doubles as the base, and Thumb code decodes as ARM.

Beyond the top 10, from PlatformIO's ranks 11–30, from Git-installed platforms and from outside PlatformIO:

  • Cortex-M:

    • PlatformIO ranks 13, 18, 22, 24 and 26–28
    • the GD32 platforms (Cortex-M23 to M33), Artery AT32, nRF54L, Apollo3 and PY32
    • outside PlatformIO, EFR32, nRF53 and nRF54, TI CC13xx and CC26xx, and NXP S32K

    Their code needs only #7951 and the M-profile lane. Their gaps are containers and vendor facts: MCUboot, Nordic DFU packages, Silicon Labs GBL, TI OAD, boot headers, NXP LPC's checksum in vector slot 7, and CubeCell's .cyacd.

  • MIPS and Arm Linux routers: 73% of the 2,235 devices in OpenWrt's hardware table are MIPS. main has the Binwalk and Unblob route, uImage, FIT images (#7771) and the MIPS lanes. Next: the 32-bit Arm Linux lane (#1922).

  • Tuya-style modules: LibreTiny (rank 14) covers Beken ARMv5TE, Realtek Cortex-M and Bouffalo RISC-V. Next: ARMv5TE raw images, Beken RBL inventory and caller-declared flash layouts.

  • D6's Harvard targets: atmelmegaavr (11) and lgt8f (19) beside AVR, intel_mcs51 (12), and Microchip PIC, PIC24 and dsPIC.

  • RISC-V: the ilp32 lane (#8229) takes sifive (15), chipsalliance (21), nuclei (29) and ESP32-C2, C5, C61 and H2, when their builds record Tag_RISCV_arch and use only extensions that Ghidra supports. With #8464, ESP-IDF 5.5.2's C5 and C61 builds, which reserve flash code twice, are refused until a policy covers that layout (#8461). Most CH32V boards, GD32VW553 and kendryte210 (30, RV64 lp64f) stay refused for now.

  • MSP430 (20): header-only admission, then a lane.

  • Out of reach for now:

    • STM8 (17), TI C2000, and Renesas RL78 and RX have no stock Ghidra decoder.
    • AURIX TriCore and RH850 (through Ghidra's V850 module) come later through the expert language override, unverified.

  • Host platforms (16, 23 and 25) are outside this work.

Where reach is highest. These groups come from the signals below:

  • Highest: the ESP32 family, STM32 and the other Cortex-M platforms, MIPS and Arm Linux routers, and Tuya-style Wi-Fi modules (Beken, Realtek, Bouffalo).
  • High: ESP8266, Nordic nRF52 to nRF54, RP2040 and RP2350, and AVR, which PlatformIO ranks second, mostly for hobby builds.
  • Long tail: 8051, PIC and dsPIC, MSP430, other RISC-V boards (SiFive, GD32V, CH32V, Kendryte), EFR32, TI CC13xx and CC26xx, and NXP S32K.
  • Out of reach without a decoder: TI C2000, Renesas RL78 and RX, and STM8.

The signals, read on 29 Oct:

  • PlatformIO's platform ranking, and the Git-installed platforms that it does not rank, by GitHub stars
  • GitHub topic counts
  • NVD CVE keyword matches
  • OpenWrt's hardware table

They measure attention or support, not how often analysts meet a device. They also undercount automotive and industrial parts. Two better measures come next. One classifies REA's own firmware issues by artifact and by the first stage that blocked them. The other is the corpus baseline.

The four queued branches, in the order I plan to open them

#8227–#8230 merged on 31 Oct. On 1 Nov #8395 (the shared positional reader), #8464 (ESP-IDF flash code) and #8462 (the no-return verifier's probe) opened, and they and #8401 (RISC-V ilp32f) became ready for review. Each queued branch waits until the PR that it builds on merges.

  1. ESP8266 images and call0 Xtensa ELF, after #7486
  2. The M-profile Arm ELF lane for STM32 and the other Cortex-M platforms, after #7951
  3. Rizin decoder and base-address inputs, after #7487
  4. The expert Ghidra language override, after the D1 decision in #7951

Alongside these:

My fork's embedded-integration branch merges main with all of them, so conflicts between them show up before any PR opens.

Decisions this work waits on, each with a proposed answer

Each decision already has a proposed answer in its own thread, so a confirmation or a redirect there is enough.

Decision Proposed answer Where
D1: raw M-profile images The decision record in #7951 #4246, #7951
C1: xtensa in three closed output enums Additive, with a release note, as for Hexagon and PowerPC. The same rule covers later families and formats #7486
D3: RISC-V soft- and single-float ABIs Decided on #7477 (31 Oct): a per-ABI policy in the bridge. ilp32 merged in #8229, and ilp32f is #8401 #7477, #8229, #8401
Format owner Two behavior-preserving refactors, format half first; a one-time enum reorder; one reader registry later #7478
D4: NVS scope A general reader with no decryption, key recovery or redaction by key name #3615
.ARM.attributes owner Whichever of #1922 and the M-profile lane lands second adopts the reader of the other #1922
ESP-IDF flash overlay The RISC-V lanes prepare Ghidra's memory map for one NOBITS reservation over code, and refuse other code overlaps; C5 and C61's second reservation is a follow-up #8461, #8464
ESP32-P4 vendor extensions A vendor mode that admits Espressif's three extension names and reports what Ghidra cannot decode; S31 is not admitted on P4's evidence #8463
F1 and F2: architecture owner and load plan A minimal F1 with a consumer; F2 for DOL, then for raw images, with identity reconciled with D1's two levels Design issues, to be opened
D6: Harvard address spaces Space-qualified addresses, sizes in addressable units and named space roles; AVR first, then 8051 and PIC; reviewed with F2 A design issue, to be opened

Expected interaction with targets

Read existing files or captured data

Boundary details (optional)

Static analysis inside existing boundaries. Ghidra and Rizin run as they do today, and no target code runs.

  • No injection, mutation, flashing, decryption or key recovery.
  • No bundled vendor SVD or ATDF files.
  • No guessed bases, architectures or ABIs.
  • Partitions and payloads that declare encryption are reported as encrypted, never decrypted.

Device access (read-only acquisition, remote debugging) starts as design issues, and is built only after a scope decision.

Options and tradeoffs

  • Structure. The stages describe what a task needs. Dependencies, not stage order, set the sequence, and platform reach sets the priorities within a stage. Item IDs (Q, F, D) stay stable.
  • Breadth versus verification. Verified lanes cover the highest-reach platforms. Architectures with no verified lane yet stay reachable through Rizin and the expert language override. Their Evidence names the tool or language that answered, without a claim of a verified lane.
  • Evidence for priorities. The reach groups come from attention and support signals. REA's own issues and the corpus baseline will test them.
  • PR size. One PR per theme keeps each review coherent, but it makes some PRs large: D1 adds about 5,600 lines across 71 files. If you prefer smaller reviews, I can split a theme further.

Verification available so far

I have a prototype or observed results

Success criteria and open questions

  • A flash dump or update file from each highest-reach group reaches readable code, or a justified refusal, with no guessed facts: ESP32, STM32 and other Cortex-M parts, router images and Tuya-style modules.
  • The corpus reports correct mappings, recovered known functions, ABI correctness and justified refusals for each platform group.
  • Rizin Evidence names the tool and decoder that answered.
  • A new platform takes rows in tables: a family, a format or a vendor fact.
  • Open: the decisions above, and your order for the stages and platforms.

References

#32, #1922, #3483, #3615, #4245, #4246, #4927, #5744, #7136, #7234, #7477, #7478, #7490 (fixed by #7823), #7678 (done in #7869), #7679, #7680, #8461, #8462, #8463, #8464, NationalSecurityAgency/ghidra#8848

Originally created by @Entrpi on GitHub (Oct 29, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7955 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Morluto's [video](https://x.com/morluto/status/2114792311628730654) shows the REA mech as it fights a TV monster that has microphone tentacles. It is a cartoon, but it points at a real question: what do the devices around us record, and where do they send it? A TV is one device. Many devices around it run on microcontrollers: voice remotes, Bluetooth microphones and earbuds, smart plugs, sensors and wearables. Their firmware usually arrives as a flash dump or a vendor update file, not as an ELF build. On `main`, REA cannot take most of that firmware to readable code. Of the 26 embedded platforms in PlatformIO's top 30, four now get verified code analysis on `main`, all of them only partly. #8229 brought it to their RISC-V ELF builds. The open and queued PRs below bring that number to 19 platforms, five of them only partly. This roadmap is the work it takes for REA to read firmware across that landscape. **Platforms this roadmap covers, and common devices that they power:** | Platform | Common devices | On `main` | With the open and queued PRs | | --- | --- | --- | --- | | ESP32, S2 and S3 (Xtensa) | Wi-Fi smart plugs, switches and bulbs, small cameras, voice assistants, maker boards | App images are inspected (#7498), but their code is refused. Flash dumps are read as partitions and apps (#8228). UF2 files are read (#8230). | Code is analyzed in a verified Ghidra lane (#7486). | | ESP32-C3, C6, H2 and P4 (RISC-V) | Newer Wi-Fi, Thread, Zigbee and Matter devices | Soft-float ELF builds are analyzed under a written calling-convention policy (#8229), but in C6 and H2 builds Ghidra puts flash code in an overlay, where it loses its DWARF facts (#8461). | Flash code is analyzed at its ELF address (#8464), and single-float ELF builds are analyzed too (#8401). Builds that declare B, Zc or Espressif's own extensions stay refused. Every P4 build declares Espressif's extensions (#8463). | | ESP8266 | Older Wi-Fi smart plugs, switches and bulbs | Images are refused by design. Xtensa ELF is refused. | Images are read, and call0 code is analyzed. | | STM32 and other Cortex-M parts (SAMD, Renesas RA, GD32) | Drone flight controllers, 3D-printer boards, e-scooter controllers, Bluetooth earbuds, industrial sensors | Raw dumps decode as ARM code, with the entry point as the base (#5744). ELF analysis is unverified. DfuSe files are read (#8230). | Raw images open in Thumb state at the base that the caller gives (#7951). ELF goes through a verified lane. Next: base candidates. | | Nordic nRF52, nRF53 and nRF54 | Fitness bands, smartwatches, item trackers, Bluetooth remotes, wireless keyboards and mice | HEX files are inventoried. ELF analysis is unverified. | HEX regions open as raw images (#7951). ELF goes through a verified lane. Next: Nordic DFU packages. | | RP2040 and RP2350 | Raspberry Pi Pico boards, custom keyboards, game controllers | UF2 files are read (#8230). ELF analysis is unverified. | Raw images open (#7951), and ELF goes through a verified lane. The RISC-V cores of RP2350 wait for Ghidra. | | Tuya-style modules (Beken, Realtek, Bouffalo) | Budget smart plugs, bulbs and switches sold under many brands | LibreTiny's UF2 files are read (#8230). | Realtek Cortex-M code opens through #7951. Next: Beken ARMv5TE images and Beken RBL files. | | MIPS and Arm Linux | Home routers, Wi-Fi access points, IP cameras | Binwalk and Unblob extraction, uImage, FIT images (#7771) and the MIPS lanes. | Not in this queue. Another contributor's #1922 verifies the 32-bit Arm lane. | | AVR (ATmega, ATtiny) | Arduino Uno and Mega boards, many hobby builds | HEX files are inventoried (#7136). No AVR decoding. | Not yet. Next: Harvard address spaces (D6), then an AVR lane. | | 8051, PIC and MSP430 | Remote controls, appliance boards, chargers, meters and sensors | No lane. | Later: 8051 and PIC after D6, then MSP430. | In the last column, a PR number marks an open PR: #1922 belongs to another contributor, and the others are mine. Items without a number are queued on my fork, and "Next" items are planned but not built. **What I ask.** Thank you for merging #7481–#7485 and #8227–#8230. Eight PRs are open, all ready for review, and none has conflicts with `main`. They are in the order that unblocks the most queued work: | PR | Size | Change | What this unlocks | | --- | ---: | --- | --- | | #7486 | +3,301 −62 | Xtensa ELF through a pinned Ghidra lane: ESP32 code analysis | Decompiled code for ESP32, S2 and S3 ELF programs. These Wi-Fi chips run many smart plugs, cameras and voice gadgets. Next: ESP8266. | | #7951 | +5,605 −402 | Cortex-M raw images open in Thumb state at a load base that the caller gives (D1) | Correct code from raw flash dumps of STM32 and other Cortex-M chips (drones, 3D printers, earbuds), at the load base you give. Today they decode in the wrong instruction mode. Next: Cortex-M ELF. | | #7487 | +3,236 −156 | Rizin Evidence records which tool and version answered | Every Rizin result names the Rizin build behind it, so anyone can check and repeat it. Next: Rizin inputs for chips without a Ghidra lane. | | #8231 | +1,505 −389 | One table decides how `compare_artifacts` compares each `format_metadata` kind | Comparing two versions of a package or firmware no longer misses metadata changes in some formats. | | #8401 | +2,081 −496 | RISC-V ilp32f ELF under the same calling-convention policy (D3's follow-up) | Decompiled code with correct float arguments for single-float RISC-V programs, such as ESP-IDF 5.5 builds for ESP32-H4. Its H4 evidence predates #8461; it is measured again with #8464. | | #8464 | +2,880 −116 | ESP-IDF flash code moves into Ghidra's default address space before analysis; other code overlaps are refused. Scope questions are in #8461 | DWARF signatures, no-return flags and pseudocode that ends at `esp_restart()` for flash code of ESP32-C6, H2 and H21 builds, and of H4 with #8401. ESP-IDF 5.5.2's C5 and C61 builds, which reserve the code twice, are refused until a policy covers them. | | #8462 | +12 −7 | The host no-return verifier's probe, updated for the current bridge with #4892's lines, crediting its author | `verify-real-ghidra-noreturn` runs all 63 probe checks again; on `main` it stops after 43. | | #8395 | +489 −754 | One base class for the ESP image, ESP flash, Godot PCK and Node SEA readers | Godot PCK and Node SEA files get the fixes that the ESP readers already have for closed and cancelled reads. Later fixes land once, not four times. | If you prefer to absorb a PR yourselves, as with #7498 and #7869, that works for me. If you want a PR split, or the stages in a different order, tell me here. ### Context and evidence **How each PR is built:** - **Verified against real tools.** A lane is admitted only after real Ghidra 12.1.4 runs, with an LLVM disassembly oracle where one exists. ESP images are verified against esptool 4.12 and 5.6, and the Rizin route against Rizin 0.7.4 and 0.9.1. Acceptance runs used real ESP32 builds and a flash dump from a real ESP32-S3 device. - **Observed facts only.** REA does not guess load addresses, architectures or ABIs. Inferences carry a label, and limitations name what REA did not verify. - **Protection facts stay apart.** Encryption, unknown high-entropy content, a signature's presence, its cryptographic verification and device enforcement are different facts. Encrypted payloads stay opaque. - **One owner per fact.** Each vocabulary and each comparison rule lives in one table. A new format, architecture or vendor quirk is one row, not edits across many files. - **Narrow, additive, versioned contracts.** A public contract change comes with a decision record, and it merges only after you confirm the decision. - **Licensed inputs only.** Corpus images and signature sets record their licenses. Private dumps never enter them. - **Static analysis only.** No target execution, flashing, decryption or key recovery. Device access comes only after a scope decision. <details> <summary><b>The work, stage by stage</b></summary> Each stage lists what is done, open or queued, then what comes next. Items run when their prerequisites are met, not in stage order. Sizes are XS to L. **1. Recognize and unpack.** - **Done:** - Intel HEX and S-records (#7136) - ESP32 app images (#7498) - uImage (#7234) - U-Boot FIT images (#7771) - ESP flash layouts (#8228) - UF2 and DfuSe (#8230) - the Binwalk and Unblob route, verified on Linux x64 - **Open:** one shared reader behind the ESP image, ESP flash, Godot PCK and Node SEA readers (#8395). - **Queued:** ESP8266 images. - **Next:** - STM32's plain DFU suffix (XS) - MCUboot (S) - RP2350 IMAGE_DEF blocks (S) - i.MX RT boot headers (S) - Nordic DFU packages (S–M) - ESP data partitions, LittleFS first (M) - vendor update formats by reach: Beken RBL, Silicon Labs GBL and TI OAD - **Design first:** - a decoded-source handoff for compressed payloads, before more readers produce them - delta updates, which need an identified baseline - TRX, scoped on its own - native filesystem readers, decided only after the corpus shows where extraction fails **2. Locate the code.** - **Open:** raw M-profile images (#7951, D1), with a caller-supplied base and variant. - **Next:** Cortex-M base and vector-table candidates (M) that feed D1. They extend #4245's candidate idea to load addresses. They are ranked, labeled inferences that report competing explanations, coverage and abstention, and the caller chooses one. - **Later:** architecture and endianness candidates for unknown blobs (#4245), kept apart from declared header facts. **3. Load.** - **Design issues first:** - F1, one owner for architecture families, variants and decode selection (M, a minimal first step with a consumer) - F2, a load plan that turns raw images, HEX regions and containers into segments with per-field provenance (L), reconciled with D1's two identity levels - D6, Harvard address spaces, reviewed with F2 so that F2 does not fix a model that Harvard targets cannot use - **Then:** - F2 for the DOL lane, then for raw images, with compatibility checks - HEX loading (S) - ESP app-image loading (M), after device descriptions and annotations - **Throughout:** related components are recorded as related components, never flattened into one address space. Examples: a bootloader and its application, a radio stack, secure and non-secure images, separate cores. **4. Analyze.** - **Done:** RISC-V `ilp32` (#8229). - **Open:** Xtensa ELF (#7486), RISC-V `ilp32f` (#8401), ESP-IDF flash code in Ghidra's default address space (#8464, for #8461) and the host no-return verifier's probe (#8462). - **Queued:** - the M-profile ELF lane - call0 Xtensa ELF - Rizin decoder and base-address inputs - the expert Ghidra language override - **Later:** - AVR, then 8051 and PIC, on D6 - MSP430 - Beken ARMv5TE raw images, which also cover MediaTek MT6261 and MT2503 modules - RISC-V residuals - **Upstream:** - Ghidra decoding for RISC-V B and Zc, which blocks RP2350's Hazard3 cores, ESP-IDF 6 builds for some ESP32 chips, CH32V and GD32VW553. NationalSecurityAgency/ghidra#8848 decodes ratified B; I tested it and posted four fixes there as a patch. - decoding for Espressif's own RISC-V extensions, which every ESP32-P4 build declares. #8463 proposes admitting such images with a report of what Ghidra cannot decode. - Armv8.1-M low-overhead loops **5. Enrich.** - **Next:** ESP ROM names from caller-supplied linker scripts (S–M). Their labels do not imply that REA analyzed the ROM bytes. - **Then:** peripheral names from caller-supplied SVD files (L). - **Pilots, design issues first:** - SDK library identification with caller-supplied Function ID or BSim sets, tested on held-out builds, with ambiguous matches kept - static RTOS recognition with task-entry evidence **6. Workflows.** - A firmware-triage guided workflow: an MCP prompt and an on-demand skill page. - Firmware version comparison as an end-to-end goal. - Both are judged on completed CLI and MCP tasks, not on documentation size. **Across all stages: a firmware corpus.** - **Contents:** recipes, manifests and pinned digests, with no committed binaries. It holds self-built images for each platform group, including builds from the old toolchains that PlatformIO pins, plus licensed public images such as OpenWrt's. - **Measures, kept apart:** correct mappings, recovered known functions, ABI correctness and justified refusals. A count of found functions alone can reward a wrong load. **Next after the queue.** These are work items, not single PRs: 1. The corpus and a baseline failure report. 2. Cortex-M base candidates, an extension of #4245. 3. A minimal F1, with a consumer. 4. F2 for DOL, then for raw images. 5. HEX loading, then ESP loading. F1, F2 and D6 start as design issues. Stage 1 items run alongside whenever foundation decisions stall. </details> <details> <summary><b>Platform details</b>: PlatformIO's top 10, the platforms beyond them, and where reach is highest</summary> PlatformIO's top 10, by its search index on 29 Oct. Below 9th, its package records order some platforms differently. | # | Platform | Chips (cores) | Default output | On `main` | Open or queued PRs | Next | | ---: | --- | --- | --- | --- | --- | --- | | 1 | `espressif32` | ESP32 (Xtensa LX6); S2, S3 (LX7); C3, C6 (RISC-V); pioarduino's fork adds C2, C5, C61, H2 and P4 | ESP app image | App images inspected for every chip (#7498); flash layouts (#8228); UF2 (#8230); RISC-V `ilp32` ELF (#8229), which accepts ESP-IDF 5.5's C-series and H2 builds, with flash code in a Ghidra overlay (#8461); Xtensa ELF refused | Xtensa ELF (#7486); RISC-V `ilp32f` (#8401); flash code at its ELF address (#8464) | ESP image loading; data partitions; NVS (#3615); ROM names. P4 builds (Espressif's own extensions, #8463) and ESP-IDF 6 builds that declare B or Zc stay refused | | 2 | `atmelavr` | AVR8: ATmega328P, 32U4, 2560, ATtiny | Intel HEX, `.eep` | HEX inventoried (#7136); no AVR family | — | D6, then an AVR ELF lane, then HEX and raw images | | 3 | `espressif8266` | ESP8266 (Xtensa LX106, call0 ABI) | ESP8266 v1 image | Images refused by design; Xtensa ELF refused | v1 images; call0 Xtensa ELF | SDK split images | | 4 | `ststm32` | STM32 (Cortex-M0 to M7, M33) | Raw `.bin` | ELF unverified; raw dumps decoded in ARM state; HEX inventoried; DfuSe (#8230) | Raw images (#7951); M-profile ELF lane | Base candidates; plain DFU suffix; MCUboot; SVD names | | 5 | `native` | Host OS | Host executables | Host binaries supported | — | Outside this work | | 6 | `teensy` | i.MX RT1062 (M7), Kinetis (M4, M0+), AVR (Teensy 2.0) | Intel HEX | ELF unverified; HEX inventoried | M-profile ELF lane; HEX regions open as raw images (#7951) | HEX loading; i.MX RT boot headers; AVR after D6 | | 7 | `nordicnrf52` | nRF52840, 52832, 52833 (M4) | Intel HEX, merged with the SoftDevice | ELF unverified; HEX inventoried; DFU `.zip` read as a plain ZIP | M-profile ELF lane; HEX regions open as raw images (#7951) | HEX loading; Nordic DFU packages; SoftDevice as a related component | | 8 | `raspberrypi` | RP2040 (M0+); maxgerhardt's community platform adds RP2350 (M33, or Hazard3 RISC-V) | UF2 | ELF unverified; UF2 (#8230), which names the RP2350 families; Hazard3 RISC-V ELF refused | M-profile ELF lane; raw `.bin` (#7951), with the vector table after boot2 on RP2040 | RP2350 IMAGE_DEF blocks. Hazard3 RISC-V waits for Ghidra to support Zba, Zbb, Zbs and Zbkb | | 9 | `atmelsam` | SAMD21 (M0+), SAMD51 (M4F), SAM3X (M3) | Raw `.bin` | ELF unverified; raw dumps decoded in ARM state | Raw images (#7951); M-profile ELF lane | Base candidates | | 10 | `renesas-ra` | RA4M1 (M4F, Arduino UNO R4), RA6M5 (M33) | Raw `.bin` | ELF unverified; raw dumps decoded in ARM state | Raw images (#7951); M-profile ELF lane | Base candidates | "ELF unverified" means that Ghidra loads Cortex-M ELF with its automatic `ARM:LE:32:v8`, and no lane verifies the result. "Raw dumps decoded in ARM state" is #5744: the entry doubles as the base, and Thumb code decodes as ARM. **Beyond the top 10**, from PlatformIO's ranks 11–30, from Git-installed platforms and from outside PlatformIO: - **Cortex-M:** - PlatformIO ranks 13, 18, 22, 24 and 26–28 - the GD32 platforms (Cortex-M23 to M33), Artery AT32, nRF54L, Apollo3 and PY32 - outside PlatformIO, EFR32, nRF53 and nRF54, TI CC13xx and CC26xx, and NXP S32K Their code needs only #7951 and the M-profile lane. Their gaps are containers and vendor facts: MCUboot, Nordic DFU packages, Silicon Labs GBL, TI OAD, boot headers, NXP LPC's checksum in vector slot 7, and CubeCell's `.cyacd`. - **MIPS and Arm Linux routers:** 73% of the 2,235 devices in OpenWrt's hardware table are MIPS. `main` has the Binwalk and Unblob route, uImage, FIT images (#7771) and the MIPS lanes. Next: the 32-bit Arm Linux lane (#1922). - **Tuya-style modules:** LibreTiny (rank 14) covers Beken ARMv5TE, Realtek Cortex-M and Bouffalo RISC-V. Next: ARMv5TE raw images, Beken RBL inventory and caller-declared flash layouts. - **D6's Harvard targets:** `atmelmegaavr` (11) and `lgt8f` (19) beside AVR, `intel_mcs51` (12), and Microchip PIC, PIC24 and dsPIC. - **RISC-V:** the `ilp32` lane (#8229) takes `sifive` (15), `chipsalliance` (21), `nuclei` (29) and ESP32-C2, C5, C61 and H2, when their builds record `Tag_RISCV_arch` and use only extensions that Ghidra supports. With #8464, ESP-IDF 5.5.2's C5 and C61 builds, which reserve flash code twice, are refused until a policy covers that layout (#8461). Most CH32V boards, GD32VW553 and `kendryte210` (30, RV64 `lp64f`) stay refused for now. - **MSP430** (20): header-only admission, then a lane. - **Out of reach for now:** - STM8 (17), TI C2000, and Renesas RL78 and RX have no stock Ghidra decoder. - AURIX TriCore and RH850 (through Ghidra's V850 module) come later through the expert language override, unverified. - **Host platforms** (16, 23 and 25) are outside this work. **Where reach is highest.** These groups come from the signals below: - **Highest:** the ESP32 family, STM32 and the other Cortex-M platforms, MIPS and Arm Linux routers, and Tuya-style Wi-Fi modules (Beken, Realtek, Bouffalo). - **High:** ESP8266, Nordic nRF52 to nRF54, RP2040 and RP2350, and AVR, which PlatformIO ranks second, mostly for hobby builds. - **Long tail:** 8051, PIC and dsPIC, MSP430, other RISC-V boards (SiFive, GD32V, CH32V, Kendryte), EFR32, TI CC13xx and CC26xx, and NXP S32K. - **Out of reach without a decoder:** TI C2000, Renesas RL78 and RX, and STM8. The signals, read on 29 Oct: - PlatformIO's [platform ranking](https://registry.platformio.org/search?t=platform&s=popularity), and the Git-installed platforms that it does not rank, by GitHub stars - GitHub topic counts - NVD CVE keyword matches - OpenWrt's hardware table They measure attention or support, not how often analysts meet a device. They also undercount automotive and industrial parts. Two better measures come next. One classifies REA's own firmware issues by artifact and by the first stage that blocked them. The other is the corpus baseline. </details> <details> <summary><b>The four queued branches</b>, in the order I plan to open them</summary> #8227–#8230 merged on 31 Oct. On 1 Nov #8395 (the shared positional reader), #8464 (ESP-IDF flash code) and #8462 (the no-return verifier's probe) opened, and they and #8401 (RISC-V ilp32f) became ready for review. Each queued branch waits until the PR that it builds on merges. 1. [ESP8266 images](https://github.com/Entrpi/rea/tree/feat-artifacts-esp8266-image) and [call0 Xtensa ELF](https://github.com/Entrpi/rea/tree/feat-ghidra-xtensa-call0), after #7486 2. [The M-profile Arm ELF lane](https://github.com/Entrpi/rea/tree/feat-ghidra-mprofile-elf) for STM32 and the other Cortex-M platforms, after #7951 3. [Rizin decoder and base-address inputs](https://github.com/Entrpi/rea/tree/feat-rizin-decoder-inputs), after #7487 4. [The expert Ghidra language override](https://github.com/Entrpi/rea/tree/feat-ghidra-provider-language), after the D1 decision in #7951 Alongside these: - The [target-format owner prototype](https://github.com/Entrpi/rea/tree/refactor-target-format-owner) for #7478. - #7680, for the 6.8.0 release notes. My fork's [`embedded-integration`](https://github.com/Entrpi/rea/tree/embedded-integration) branch merges `main` with all of them, so conflicts between them show up before any PR opens. </details> <details> <summary><b>Decisions this work waits on</b>, each with a proposed answer</summary> Each decision already has a proposed answer in its own thread, so a confirmation or a redirect there is enough. | Decision | Proposed answer | Where | | --- | --- | --- | | D1: raw M-profile images | The decision record in #7951 | #4246, #7951 | | C1: `xtensa` in three closed output enums | Additive, with a release note, as for Hexagon and PowerPC. The same rule covers later families and formats | #7486 | | D3: RISC-V soft- and single-float ABIs | Decided on #7477 (31 Oct): a per-ABI policy in the bridge. ilp32 merged in #8229, and ilp32f is #8401 | #7477, #8229, #8401 | | Format owner | Two behavior-preserving refactors, format half first; a one-time enum reorder; one reader registry later | #7478 | | D4: NVS scope | A general reader with no decryption, key recovery or redaction by key name | #3615 | | `.ARM.attributes` owner | Whichever of #1922 and the M-profile lane lands second adopts the reader of the other | #1922 | | ESP-IDF flash overlay | The RISC-V lanes prepare Ghidra's memory map for one NOBITS reservation over code, and refuse other code overlaps; C5 and C61's second reservation is a follow-up | #8461, #8464 | | ESP32-P4 vendor extensions | A vendor mode that admits Espressif's three extension names and reports what Ghidra cannot decode; S31 is not admitted on P4's evidence | #8463 | | F1 and F2: architecture owner and load plan | A minimal F1 with a consumer; F2 for DOL, then for raw images, with identity reconciled with D1's two levels | Design issues, to be opened | | D6: Harvard address spaces | Space-qualified addresses, sizes in addressable units and named space roles; AVR first, then 8051 and PIC; reviewed with F2 | A design issue, to be opened | </details> ### Expected interaction with targets Read existing files or captured data ### Boundary details (optional) Static analysis inside existing boundaries. Ghidra and Rizin run as they do today, and no target code runs. - No injection, mutation, flashing, decryption or key recovery. - No bundled vendor SVD or ATDF files. - No guessed bases, architectures or ABIs. - Partitions and payloads that declare encryption are reported as encrypted, never decrypted. Device access (read-only acquisition, remote debugging) starts as design issues, and is built only after a scope decision. ### Options and tradeoffs - **Structure.** The stages describe what a task needs. Dependencies, not stage order, set the sequence, and platform reach sets the priorities within a stage. Item IDs (Q, F, D) stay stable. - **Breadth versus verification.** Verified lanes cover the highest-reach platforms. Architectures with no verified lane yet stay reachable through Rizin and the expert language override. Their Evidence names the tool or language that answered, without a claim of a verified lane. - **Evidence for priorities.** The reach groups come from attention and support signals. REA's own issues and the corpus baseline will test them. - **PR size.** One PR per theme keeps each review coherent, but it makes some PRs large: D1 adds about 5,600 lines across 71 files. If you prefer smaller reviews, I can split a theme further. ### Verification available so far I have a prototype or observed results ### Success criteria and open questions - A flash dump or update file from each highest-reach group reaches readable code, or a justified refusal, with no guessed facts: ESP32, STM32 and other Cortex-M parts, router images and Tuya-style modules. - The corpus reports correct mappings, recovered known functions, ABI correctness and justified refusals for each platform group. - Rizin Evidence names the tool and decoder that answered. - A new platform takes rows in tables: a family, a format or a vendor fact. - Open: the decisions above, and your order for the stages and platforms. ### References #32, #1922, #3483, #3615, #4245, #4246, #4927, #5744, #7136, #7234, #7477, #7478, #7490 (fixed by #7823), #7678 (done in #7869), #7679, #7680, #8461, #8462, #8463, #8464, NationalSecurityAgency/ghidra#8848 - [PlatformIO platform registry](https://registry.platformio.org/search?t=platform&s=popularity) - [Morluto's TV video](https://x.com/morluto/status/2114792311628730654)
bsakaguchi commented 2026-10-31 00:52:02 +00:00
Author
Owner
Copy link

@Entrpi commented on GitHub (Oct 30, 2026):

@morluto @gensokyo-dev

Morluto's TV video shows why this work matters. A TV is one device, and many devices around it run on microcontrollers whose firmware REA cannot read yet. I rewrote the roadmap above so that it starts from those devices.

Eight PRs are open, which is the limit for one contributor, and eleven more wait on my fork. Five of the open PRs are small and independent: #7483 (+36), #7485 (+62), #7482 (+107), #7481 (+250) and #7484 (+262). Can you review these five first? Each merge frees a slot for the next branch. If you prefer to absorb, close or split a PR, tell me which one.

<!-- gh-comment-id:6519465576 --> @Entrpi commented on GitHub (Oct 30, 2026): @morluto @gensokyo-dev Morluto's [TV video](https://x.com/morluto/status/2114792311628730654) shows why this work matters. A TV is one device, and many devices around it run on microcontrollers whose firmware REA cannot read yet. I rewrote the roadmap above so that it starts from those devices. Eight PRs are open, which is the limit for one contributor, and eleven more wait on my fork. Five of the open PRs are small and independent: #7483 (+36), #7485 (+62), #7482 (+107), #7481 (+250) and #7484 (+262). Can you review these five first? Each merge frees a slot for the next branch. If you prefer to absorb, close or split a PR, tell me which one.
bsakaguchi commented 2026-11-01 21:11:13 +00:00
Author
Owner
Copy link

@Entrpi commented on GitHub (Oct 31, 2026):

Roadmap updated. Thank you for merging #7481–#7485. Eight PRs are open again, which is the limit for one contributor, and six more wait on my fork. Each merge frees a slot for the next one. All eight are ready for review. The smallest is a test fix that makes verify:ghidra:mips pass on main again:

PR Size Change What this unlocks
#8227 +207 −71 Carved MIPS copies are refused at target opening, so the MIPS lane passes again The real-Ghidra check for MIPS router firmware passes on main again, so it can catch regressions again.

The feature PRs, in the order that unblocks the most queued work:

PR Size Change What this unlocks
#7486 +3,035 −71 Xtensa ELF through a pinned Ghidra lane: ESP32 code analysis Decompiled code for ESP32, S2 and S3 ELF programs. These Wi-Fi chips run many smart plugs, cameras and voice gadgets. Next: ESP8266.
#7951 +5,585 −424 Cortex-M raw images open in Thumb state at a load base that the caller gives (D1) Correct code from raw flash dumps of STM32 and other Cortex-M chips (drones, 3D printers, earbuds), at the load base you give. Today they decode in the wrong instruction mode. Next: Cortex-M ELF.
#8228 +7,447 −131 ESP32 flash dumps read as a partition table with its apps A full ESP32 flash dump opens as its partition table, apps and data partitions, each one ready to extract.
#8229 +3,674 −263 RISC-V ilp32 ELF (ESP32-C3, C6, H2) under a written calling-convention policy (D3, decided on #7477) Decompiled code with correct function arguments for ESP32-C3, C6 and H2 ELF programs, the chips in newer Wi-Fi, Thread, Zigbee and Matter devices. Next: ESP32-P4.
#7487 +2,094 −134 Rizin Evidence records which tool and version answered Every Rizin result names the Rizin build behind it, so anyone can check and repeat it. Next: Rizin inputs for chips without a Ghidra lane.
#8230 +4,667 −389 UF2 (RP2040, RP2350, LibreTiny, ESP32-S2 and S3) and DfuSe containers Update files for Raspberry Pi Pico boards, LibreTiny smart plugs and STM32 DFU updates open as their code regions, ready to extract.
#8231 +1,464 −340 One table decides how compare_artifacts compares each format_metadata kind Comparing two versions of a package or firmware no longer misses metadata changes in some formats.

If you prefer to absorb a PR yourselves, as with #7498 and #7869, that works for me. If you want a PR split, or the stages in a different order, tell me here.

<!-- gh-comment-id:6536577701 --> @Entrpi commented on GitHub (Oct 31, 2026): **Roadmap updated.** Thank you for merging #7481–#7485. Eight PRs are open again, which is the limit for one contributor, and six more wait on my fork. Each merge frees a slot for the next one. All eight are ready for review. The smallest is a test fix that makes `verify:ghidra:mips` pass on `main` again: | PR | Size | Change | What this unlocks | | --- | ---: | --- | --- | | #8227 | +207 −71 | Carved MIPS copies are refused at target opening, so the MIPS lane passes again | The real-Ghidra check for MIPS router firmware passes on `main` again, so it can catch regressions again. | The feature PRs, in the order that unblocks the most queued work: | PR | Size | Change | What this unlocks | | --- | ---: | --- | --- | | #7486 | +3,035 −71 | Xtensa ELF through a pinned Ghidra lane: ESP32 code analysis | Decompiled code for ESP32, S2 and S3 ELF programs. These Wi-Fi chips run many smart plugs, cameras and voice gadgets. Next: ESP8266. | | #7951 | +5,585 −424 | Cortex-M raw images open in Thumb state at a load base that the caller gives (D1) | Correct code from raw flash dumps of STM32 and other Cortex-M chips (drones, 3D printers, earbuds), at the load base you give. Today they decode in the wrong instruction mode. Next: Cortex-M ELF. | | #8228 | +7,447 −131 | ESP32 flash dumps read as a partition table with its apps | A full ESP32 flash dump opens as its partition table, apps and data partitions, each one ready to extract. | | #8229 | +3,674 −263 | RISC-V ilp32 ELF (ESP32-C3, C6, H2) under a written calling-convention policy (D3, decided on #7477) | Decompiled code with correct function arguments for ESP32-C3, C6 and H2 ELF programs, the chips in newer Wi-Fi, Thread, Zigbee and Matter devices. Next: ESP32-P4. | | #7487 | +2,094 −134 | Rizin Evidence records which tool and version answered | Every Rizin result names the Rizin build behind it, so anyone can check and repeat it. Next: Rizin inputs for chips without a Ghidra lane. | | #8230 | +4,667 −389 | UF2 (RP2040, RP2350, LibreTiny, ESP32-S2 and S3) and DfuSe containers | Update files for Raspberry Pi Pico boards, LibreTiny smart plugs and STM32 DFU updates open as their code regions, ready to extract. | | #8231 | +1,464 −340 | One table decides how `compare_artifacts` compares each `format_metadata` kind | Comparing two versions of a package or firmware no longer misses metadata changes in some formats. | If you prefer to absorb a PR yourselves, as with #7498 and #7869, that works for me. If you want a PR split, or the stages in a different order, tell me here.
bsakaguchi commented 2026-11-01 21:11:13 +00:00
Author
Owner
Copy link

@Entrpi commented on GitHub (Nov 1, 2026):

Roadmap updated. Thank you for merging #8227–#8230. None of the six open PRs has conflicts with main. Four are ready for review: #7486, #7951, #7487 and #8231. Two new drafts follow the merged work: #8401 (RISC-V ilp32f) and #8395 (one shared reader behind the ESP, Godot PCK and Node SEA readers).

<!-- gh-comment-id:6551181869 --> @Entrpi commented on GitHub (Nov 1, 2026): **Roadmap updated.** Thank you for merging #8227–#8230. None of the six open PRs has conflicts with `main`. Four are ready for review: #7486, #7951, #7487 and #8231. Two new drafts follow the merged work: #8401 (RISC-V ilp32f) and #8395 (one shared reader behind the ESP, Godot PCK and Node SEA readers).
bsakaguchi commented 2026-11-03 00:29:39 +00:00
Author
Owner
Copy link

@Entrpi commented on GitHub (Nov 2, 2026):

Roadmap updated. All eight open PRs are ready for review, and none has conflicts with main. Thank you for marking #8401 ready. New since my last comment:

  • ESP-IDF flash code (#8461, fixed by #8464). ESP-IDF reserves the addresses of .flash.text with a NOBITS section, so Ghidra 12.1.4 loads that code into an overlay that its DWARF import does not reach. #8464 moves the code into the default address space before analysis. Imported signatures of DWARF-described flash functions go from 6 to 278 of 278 in an ESP32-H4 build (with #8401) and from 18 to 549 of 549 in a C6 build, and pseudocode ends at esp_restart(). ESP-IDF 5.5.2's C5 and C61 builds reserve the code twice and are refused for now; #8461 has the scope questions.
  • The host no-return verifier (#8462). Its probe is updated for the current bridge with #4892's lines, crediting their author. verify-real-ghidra-noreturn runs all 63 probe checks again; on main it stops after 43.
  • ESP32-P4 (#8463). A design proposal: admit P4 images that declare Espressif's vendor extensions, with a report of what Ghidra cannot decode.
  • Upstream Ghidra. I tested NationalSecurityAgency/ghidra#8848, which decodes ratified B, and posted four fixes there as a patch. Without B and Zc decoding, RP2350's Hazard3 cores, ESP-IDF 6 builds for some ESP32 chips, CH32V and GD32VW553 stay out of reach.
  • Review rounds. #7486 now also refuses Xtensa images whose L32R literals come from LITBASE (USE_ABSOLUTE_LITERALS=1), because Ghidra 12.1.4 decodes every L32R as PC-relative. #7487's title now marks one breaking change: process capture rejects a PATH search that meets an error such as ELOOP or EIO before its match, because a native search can stop there.
  • CI. #7486, #7951 and #8401 merged main for its move of the lane docs to docs/testing-lanes.md, and #7951 and #8464 re-pinned profile digests for #8531 and #8520. Their test shards and static checks fail only where main's do today, which #8592 and #8596 fix. On my fork's integration branch, which merges main with all eight PRs and the queued branches, the full test run on Linux fails only on those same tests.
<!-- gh-comment-id:6566088618 --> @Entrpi commented on GitHub (Nov 2, 2026): **Roadmap updated.** All eight open PRs are ready for review, and none has conflicts with `main`. Thank you for marking #8401 ready. New since my last comment: - **ESP-IDF flash code (#8461, fixed by #8464).** ESP-IDF reserves the addresses of `.flash.text` with a NOBITS section, so Ghidra 12.1.4 loads that code into an overlay that its DWARF import does not reach. #8464 moves the code into the default address space before analysis. Imported signatures of DWARF-described flash functions go from 6 to 278 of 278 in an ESP32-H4 build (with #8401) and from 18 to 549 of 549 in a C6 build, and pseudocode ends at `esp_restart()`. ESP-IDF 5.5.2's C5 and C61 builds reserve the code twice and are refused for now; #8461 has the scope questions. - **The host no-return verifier (#8462).** Its probe is updated for the current bridge with #4892's lines, crediting their author. `verify-real-ghidra-noreturn` runs all 63 probe checks again; on `main` it stops after 43. - **ESP32-P4 (#8463).** A design proposal: admit P4 images that declare Espressif's vendor extensions, with a report of what Ghidra cannot decode. - **Upstream Ghidra.** I tested NationalSecurityAgency/ghidra#8848, which decodes ratified B, and posted four fixes there as a patch. Without B and Zc decoding, RP2350's Hazard3 cores, ESP-IDF 6 builds for some ESP32 chips, CH32V and GD32VW553 stay out of reach. - **Review rounds.** #7486 now also refuses Xtensa images whose L32R literals come from `LITBASE` (`USE_ABSOLUTE_LITERALS=1`), because Ghidra 12.1.4 decodes every L32R as PC-relative. #7487's title now marks one breaking change: process capture rejects a `PATH` search that meets an error such as `ELOOP` or `EIO` before its match, because a native search can stop there. - **CI.** #7486, #7951 and #8401 merged `main` for its move of the lane docs to `docs/testing-lanes.md`, and #7951 and #8464 re-pinned profile digests for #8531 and #8520. Their test shards and static checks fail only where `main`'s do today, which #8592 and #8596 fix. On my fork's integration branch, which merges `main` with all eight PRs and the queued branches, the full test run on Linux fails only on those same tests.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
codex/procedure-reference-stacked
release-please--branches--main--components--rea-agents
release-please--branches--main--components--rea-agents--release-notes
codex/kuna-independent-retry
codex/byte-search-overlay-coverage
codex/open-failure-target-receipt
codex/kuna-xref-evidence
codex/ghidra-ambiguous-procedure-recovery
codex/function-source-bundle
codex/trace-function-integration-prototype
codex/snapshot-source-closure
codex/recorded-evidence-array-pages
codex/evidence-recovery-redesign
codex/trace-feature-address-scope
codex/call-discovery-scope
codex/ghidra-literal-candidate-evidence
codex/decompile-directory-independent-retry
codex/retry-provenance-base
codex/value-flow-capability-experiment
codex/ghidra-containing-instruction
cursor/readme-discord-button-ef9f
dependabot/github_actions/actions/setup-node-7.1.0
dependabot/github_actions/actions/upload-artifact-7.0.2
dependabot/npm_and_yarn/npm-minor-and-patch-d8d97ce6df
codex/kuna-1989
fix/8559-native-ui-fixture-ownership
fix/8555-go-verifier-mcp-ownership
codex/provider-capability-recovery
fix/8642-rizin-target-stdin
codex/ghidra-memory-effect-scope
codex/ghidra-annotation-entry-selector
codex/ghidra-readonly-recovery
codex/ghidra-regex-verifier-recovery
codex/evidence-discovery-view-guidance
fix/8545-bulk-append-overflow
fix/8428-navigation-refusal-alternatives
codex/address-context-recovery
codex/matching-member-limitations
codex/unknown-scope-contract
codex/evidence-view-guidance
feat/8373-native-trace-views
codex/ghidra-snapshot-verifier-binding
codex/elf-export-visibility
codex/evidence-view-recovery
codex/native-reference-cli-help
codex/provider-candidate-observation-scope
codex/kuna-session-verification
codex/ghidra-switch-lifetime-guidance
codex/pseudocode-paging-discovery
codex/clarify-evidence-view-support
codex/kuna-xrefs-selector-recovery
codex/context-capacity-recovery
codex/evidence-closure-large-links
codex/same-function-evidence-identity
codex/ghidra-verifier-inspected-jdk
codex/audit-guided-workflows
codex/crash-prompt-retained-dossiers
codex/analysis-view-error-recovery
codex/kuna-verifier-target-selection
codex/ghidra-selected-project-call-coverage
codex/docs-native-analysis-scope
codex/ghidra-java-socket-boundary
codex/ghidra-retained-snapshot-verifier
codex/ghidra-search-inventory-facts
fix/8284-zero-argument-parameter-bindings
fix/8198-ui-action-handler-offset
fix/8257-elf-entry-label-identity
fix/8304-deferred-output-schema-projection
fix/8277-call-path-computed-target-uncertainty
fix/pr-8089-conflicts
fix/ci-static-duplicate-export
dependabot/npm_and_yarn/lint-staged-17.6.0
dependabot/github_actions/actions/download-artifact-8.0.2
dependabot/npm_and_yarn/write-file-atomic-8.0.0
dependabot/npm_and_yarn/canonicalize-5.1.0
dependabot/github_actions/actions/cache/restore-6.1.0
dependabot/npm_and_yarn/toon-format/toon-4.4.0
dependabot/github_actions/actions/deploy-pages-5.0.1
codex/review-fixes-pr-2403
codex/review-fixes-pr-2400
codex/process-terminal-width
codex/test-brittleness-cleanup
verify/7167-browser-connect-macos
fix/7167-browser-connect
feat/native-matching-allowance-setting
feat/browser-scenario-dry-run
test/managed-caller-parameters-20261024
codex/parser-fortythird-boundaries
fix/6926-pinvoke-platform-decorations-20261023
fix/6889-har-body-encoding-20261023
fix/6888-sqlite-schema-budget-20261023
fix/dependency-security-gates-6708
cursor/evidence-overwrite-rebase-b4ac
feat/4057-verify-native-exports
fix/windows-acl-observer-overhead-20261022
cursor/erlang-beam-restack-4476
cursor/pe-certificate-review-bf32
fix/6116-javascript-application-fs-errors
fix/6061-unmapped-runtime-script-unknown
feat/browser-scenario-summary
fix/pe-certificates
codex/verify-3407
cursor/rebase-5699
cursor/rebase-5352
cursor/rebase-5600
cursor/fix-cli-schema-omission-10a8
codex/audit-hopper-shutdown-reasons-20261021
fix/demangle-swift-omitted-record
fix/nib-boolean-value-types
cursor/cancellation-partial-observation-a5ac
fix/ib-object-limit-reasons
fix/electron-fuse-limit-advice
fix/kuna-doctor-dangling-cache
fix/fixed-budget-refusal-advice
fix/js-ipc-listener-dedup
fix/js-return-member-provenance
fix/dmg-integrity-retry-advice
fix/ida-doctor-readiness
fix/kuna-version-probe-output-drain
fix/legacy-swift-runtime-name-matching
fix/dylib-lazy-load-commands
fix/browser-scenario-startup-cancel
fix/swift-nested-punycode-classification
fix/compare-unselected-scopes
fix/objc-method-budget-coverage
fix/dmg-mounted-root-observations
audit/ci-tests-20261019-round2
release/6.7.0
fix/wabt-cleanup-retry-resolution
fix/electron-fuse-elf-pe-architecture
fix/native-plist-descriptor-ownership
fix/kuna-caller-cleanup-ownership
experiment/atomic-codemode-20261016
review/first-contributor-3776
fix/call-resolution-full-read-cycles
fix/member-mutation-selection-repeat
experiment/atomic-re-tools-20261015
perf/member-mutation-lazy-origins
perf/static-network-mutation-index
fix/hopper-startup-lifecycle
audit/3050-windows-proof-20261015
feat/windows-window-capture-2183
audit/3049-windows-proof-20261015
audit/3735-macos-fixtures-20261015
fix/batch-decompile-null-pseudocode
fix/kuna-inventory-honors-jobs
experiment/browser-codemode-tool-design-20261015
fix/hopper-clean-runtime-argv
codex/review-2049-27f1b367
review-fixes/pr-2787-routing
fix/3360-scoped-this-emitters
fix/3367-replaced-global-effects
fix/3838-compact-any-value-items
test/hopper-download-probe-20261015
codex/windows-runtime-creation-red-green-20261015
codex/windows-readonly-root-red-green-20261015
codex/windows-root-share-red-green-20261015
codex/exp-1685-win-mcp-startup
codex/bug-sweep-20261015
investigate/2190-windows-package-phases
investigate/3543-macos-cli-deadline
perf/3281-reverse-tools-feedback-quality
fix/3406-hopper-doctor-fixture
fix/3341-wrapper-require-test-context
fix/3301-owned-uninstall-fixtures
fix/3164-compiler-input-ownership
fix/pcap-cleanup-stdout-newline-20261014
perf/interface-builder-summary
codex/review-fixes-pr-1977
fix/ghidra-effective-jvm-platform
fix/docs-turkish-rust-link-20261013
fix/main-maintenance-ci-20261010
fix/recover-binary-session-process-effects
codex/review-fixes-pr-2457
codex/ci-required-cancellation-gate
fix/main-format-baseline-20261013
fix/setup-preserve-entry-fields
cursor/restore-main-checks-1ffc
fix/ghidra-bridge-hashmap-import
codex/shared-ci-jadx-json-default-20261013
codex/shared-ci-ghidra-test-groups-20261013
codex/shared-ci-java-fixture-format-20261013
codex/fix-intrinsic-invocation-boundaries
test/release-preflight-reusable-jobs
test/js-parameter-provenance-expectation
fix/linux-hopper-verification-current
test/2419-ghidra-elf-probe-size
followup/pr-1544-native-ida
codex/fold-pr-2230-contribution
docs/readme-responsible-use
fix/issue-2064
fix/issue-2050
fix/issue-2053
fix/issue-1968
fix/issue-1945
experiment/codemode-bridge-20261011
fix/tool-effects-negative-wording-1904-20261012
fix/1842-electron-interop-provenance
feat/recover-javascript-multi-input
refactor/website-i18n-prep
fix/nativeaot-pe-virtual-extent
fix/browser-verifier-ci-hang
fix/pe-resources-input-selection
docs/beginner-friendly-guides
docs/beginner-setup-guide
fix/javascript-analysis-worker-1563
fix/1649-ghidra-duplicate-names
fix/javascript-export-keys-integration-1369
test/maintenance-boundary-contracts-20261011
review/javascript-export-keys-1369
review/process-live-progress-1483
test/javascript-1495-growth-ratio
fix/ci-escape-timing
docs/normalized-terminal-geometry-1450
release/6.3.0
archive/release-6.3.0-pr1381
release/6.2.0
fix/electron-preload-dynamic-override
fix/javascript-self-referential-bundler-edges
release/6.1.0
cursor/grok-client-cherry-pick-0b56
feat/signature-posture
feat/dyld-shared-cache
feat/artifact-integrity-policy
feat/pkg-inventory
release/6.0.0
chore/sync-release-5.0.0
release/5.0.0
cursor/cloud-agent-node-787e
fix/javascript-xhr-open-endpoints
review/pr-711-realpath-fallback-wording
cleanup/replay-confinement
codex/test-suite-performance
codex/issue-foundations
codex/address-open-issues-2026-07-20
agent/javascript-vite-rollup-esbuild-adapters
agent/process-cleanup
agent/triage-integration
agent/unknown-commands-registry
feat/mcp-identity-metadata
morlvps/issue_cleanup
codex/installer-redesign
morl/roadmap-investigation-core
morl/roadmap-investigation-docs
release-please--branches--main--components--rea
rea-agents-6.8.0
rea-agents-6.7.0
rea-agents-6.6.0
experiment/atomic-codemode-wait-runtime-20261016
rea-agents-6.5.0
rea-agents-6.4.0
rea-agents-6.3.0
rea-agents-6.2.0
rea-agents-6.1.0
rea-agents-6.0.0
rea-agents-5.0.0
rea-agents-4.1.0
rea-agents-4.0.1
rea-agents-4.0.0
rea-agents-3.2.1
rea-agents-3.2.0
rea-agents-3.1.0
rea-agents-3.0.0
rea-agents-2.7.0
rea-agents-2.6.0
rea-agents-2.5.1
rea-agents-2.5.0
rea-agents-2.4.0
rea-agents-2.3.0
rea-agents-2.2.0
rea-agents-2.1.0
rea-agents-2.0.0
rea-agents-1.7.0
rea-agents-1.6.0
rea-agents-1.5.0
rea-agents-1.4.0
rea-agents-1.3.0
rea-agents-1.2.0
rea-agents-1.1.0
rea-agents-1.0.0
rea-agents-0.5.0
rea-agents-0.4.0
rea-agents-0.3.0
rea-0.2.1
rea-0.2.0
Labels
Clear labels   
area: bridge

   
area: bridge

   
area: build

   
area: build

   
area: build

   
area: cli

   
area: domain

   
area: evidence

   
area: mcp

   
area: native

   
area: providers

   
area: runtime

   
area: server

   
area: setup

   
area: tests

   
bug

   
design

   
documentation

   
enhancement

   
enhancement

   
enhancement

   
experiment

   
help wanted

   
i18n

   
invalid

   
P0: critical

   
P1: high

   
P2: medium

   
P2: medium

   
P2: medium

   
P3: low

   
performance

   
provider-request

   
pull-request
Mirrored from GitHub Pull Request

  
question

   
roadmap

   
triage
No labels
area: bridge
area: bridge
area: build
area: build
area: build
area: cli
area: domain
area: evidence
area: mcp
area: native
area: providers
area: runtime
area: server
area: setup
area: tests
bug
design
documentation
enhancement
enhancement
enhancement
experiment
help wanted
i18n
invalid
P0: critical
P1: high
P2: medium
P2: medium
P2: medium
P3: low
performance
provider-request
pull-request
question
roadmap
triage
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#4654
No description provided.