morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3155] [Feature] Admit recordless legacy MIPS32 ELFs whose section header table lies past EOF (carved firmware) #1102

New issue

Open

opened 2026-10-15 18:31:04 +00:00 by bsakaguchi · 7 comments

bsakaguchi commented

2026-10-15 18:31:04 +00:00

Owner

Originally created by @bqw371 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3155

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

Thanks for #1898. It admits recordless legacy MIPS32 executables, but only when every section header could be read. A carved firmware ELF of that shape (EF_MIPS_ARCH_32, ABI field 0, no PT_MIPS_ABIFLAGS, section header table past EOF) is still refused. These are the two cases #1619 and #1620 reported, combined in one file. open_binary (provider ghidra) returns capability_unavailable with this Ghidra rejection (architecture_unsupported):

This recordless legacy admission requires a complete ABI-record inspection; ABI declarations in unavailable headers remain unknown. MIPS section header table at file offset 2952 extends beyond the captured file: examined 0 of 24 complete headers. ABI declarations in unavailable headers remain unknown.

The same file before it is cut opens normally at MIPS:LE:32:default.

Reproduction

Built from the repository's own conformance sources at 58300885 (Homebrew clang 23.1.0, LLD), with -march=mips32 and the other mips-fixture.mjs arguments:

clang --target=mipsel-linux-gnu -march=mips32 -mabi=32 -mno-abicalls -fno-pic -fuse-ld=lld \
  -nostdlib -static -O0 -g -Wl,-e,rea_mips_entry -Wl,--build-id=none \
  tests/conformance/c/mips.c tests/conformance/ghidra/mips-probe.S -o r1.elf
python3 make_carve.py r1.elf legacy.elf legacy_carved.elf

make_carve.py clears the header ABI field, retypes the ABI flags record away and cuts the file 1 byte before e_shoff. All PT_LOAD segments stay inside the file:

import struct, sys
b = bytearray(open(sys.argv[1], "rb").read())
struct.pack_into("<I", b, 36, 0x50000001)                 # EF_MIPS_ARCH_32 | noreorder, ABI field 0
phoff, shoff = struct.unpack_from("<II", b, 28)
phentsize, phnum, shentsize, shnum = struct.unpack_from("<HHHH", b, 42)
for i in range(phnum):                                    # PT_MIPS_ABIFLAGS -> PT_NULL
    o = phoff + i * phentsize
    if struct.unpack_from("<I", b, o)[0] == 0x70000003: struct.pack_into("<I", b, o, 0)
for i in range(shnum):                                    # SHT_MIPS_ABIFLAGS -> SHT_PROGBITS
    o = shoff + i * shentsize + 4
    if struct.unpack_from("<I", b, o)[0] == 0x7000002A: struct.pack_into("<I", b, o, 1)
open(sys.argv[2], "wb").write(b)                          # recordless legacy file (admitted by #1898)
open(sys.argv[3], "wb").write(b[:shoff - 1])              # the same file carved before its section table

GNU readelf 2.47 on legacy_carved.elf: Flags: 0x50000001, noreorder, mips32, the three LOAD segments intact, the former ABI flags program header shown as NULL, and Reading 960 bytes extends past end of file for section headers.

File	`open_binary` at `58300885`
`legacy.elf`	opens, `MIPS:LE:32:default`, 3 procedures
`legacy_carved.elf`	refused as above

Expected behavior

Admit the recordless legacy shape when the only inspection limitations come from a section header table past EOF. Keep that limitation in the profile (abi_flags_limitations) and in session Evidence, next to the existing o32-assumption limitations, so the unread headers stay unknown.

In MipsElfMetadata.ts the program header table is required and read whole. Only the optional section header table can produce this limitation. So for these files every program header has been checked and none is PT_MIPS_ABIFLAGS. What stays unknown is a record declared only in the unread section headers. Under this proposal Evidence would say exactly that, and o32 is already recorded as an assumption, not a declared fact.

We carry this rule as a local patch. With it, legacy_carved.elf opens at MIPS:LE:32:default with the 32-bit pin (Ghidra 12.1.3). It finds 2 procedures, not 3, because the symbol table is in the missing sections. The change is in hasLegacyMipsHeaderShape:

const isCarvedSectionTableLimitation = (limitation: string): boolean =>
  limitation.startsWith("MIPS section header table at file offset ");
// ...
    abiFlags === null &&
    (limitations === undefined ||
      limitations.every(isCarvedSectionTableLimitation))

It replaces the GhidraMipsProfile.test.ts row that expects this case to be refused with one that checks admission, the pin and the kept limitation. A program-header limitation is still refused. The src/ghidra, src/config, MIPS boundary and Ghidra launcher suites pass (654 passed, 4 skipped). Matching on the message prefix is the smallest change. A structured limitation kind from MipsElfMetadata would be cleaner if you prefer that. I'm happy to open a PR either way, with a verify:ghidra:mips round built like the snippet above.

Impact and scope

It affects legacy MIPS32 firmware carved from flash or memory dumps: the ELF keeps its original e_shoff, but the section table was never captured. Complete files are unaffected, and so are files that declare an ABI, since #1629 already covers carves that carry a record.

REA version

source revision 58300885 (rea --version: 6.4.0)

Originally created by @bqw371 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3155 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior Thanks for #1898. It admits recordless legacy MIPS32 executables, but only when every section header could be read. A carved firmware ELF of that shape (`EF_MIPS_ARCH_32`, ABI field 0, no `PT_MIPS_ABIFLAGS`, section header table past EOF) is still refused. These are the two cases #1619 and #1620 reported, combined in one file. `open_binary` (provider `ghidra`) returns `capability_unavailable` with this Ghidra rejection (`architecture_unsupported`): ``` This recordless legacy admission requires a complete ABI-record inspection; ABI declarations in unavailable headers remain unknown. MIPS section header table at file offset 2952 extends beyond the captured file: examined 0 of 24 complete headers. ABI declarations in unavailable headers remain unknown. ``` The same file before it is cut opens normally at `MIPS:LE:32:default`. ### Reproduction Built from the repository's own conformance sources at `58300885` (Homebrew clang 23.1.0, LLD), with `-march=mips32` and the other `mips-fixture.mjs` arguments: ```bash clang --target=mipsel-linux-gnu -march=mips32 -mabi=32 -mno-abicalls -fno-pic -fuse-ld=lld \ -nostdlib -static -O0 -g -Wl,-e,rea_mips_entry -Wl,--build-id=none \ tests/conformance/c/mips.c tests/conformance/ghidra/mips-probe.S -o r1.elf python3 make_carve.py r1.elf legacy.elf legacy_carved.elf ``` `make_carve.py` clears the header ABI field, retypes the ABI flags record away and cuts the file 1 byte before `e_shoff`. All `PT_LOAD` segments stay inside the file: ```python import struct, sys b = bytearray(open(sys.argv[1], "rb").read()) struct.pack_into("<I", b, 36, 0x50000001) # EF_MIPS_ARCH_32 | noreorder, ABI field 0 phoff, shoff = struct.unpack_from("<II", b, 28) phentsize, phnum, shentsize, shnum = struct.unpack_from("<HHHH", b, 42) for i in range(phnum): # PT_MIPS_ABIFLAGS -> PT_NULL o = phoff + i * phentsize if struct.unpack_from("<I", b, o)[0] == 0x70000003: struct.pack_into("<I", b, o, 0) for i in range(shnum): # SHT_MIPS_ABIFLAGS -> SHT_PROGBITS o = shoff + i * shentsize + 4 if struct.unpack_from("<I", b, o)[0] == 0x7000002A: struct.pack_into("<I", b, o, 1) open(sys.argv[2], "wb").write(b) # recordless legacy file (admitted by #1898) open(sys.argv[3], "wb").write(b[:shoff - 1]) # the same file carved before its section table ``` GNU readelf 2.47 on `legacy_carved.elf`: `Flags: 0x50000001, noreorder, mips32`, the three `LOAD` segments intact, the former ABI flags program header shown as `NULL`, and `Reading 960 bytes extends past end of file for section headers`. | File | `open_binary` at `58300885` | | --- | --- | | `legacy.elf` | opens, `MIPS:LE:32:default`, 3 procedures | | `legacy_carved.elf` | refused as above | ### Expected behavior Admit the recordless legacy shape when the only inspection limitations come from a section header table past EOF. Keep that limitation in the profile (`abi_flags_limitations`) and in session Evidence, next to the existing o32-assumption limitations, so the unread headers stay unknown. In `MipsElfMetadata.ts` the program header table is required and read whole. Only the optional section header table can produce this limitation. So for these files every program header has been checked and none is `PT_MIPS_ABIFLAGS`. What stays unknown is a record declared only in the unread section headers. Under this proposal Evidence would say exactly that, and o32 is already recorded as an assumption, not a declared fact. We carry this rule as a local patch. With it, `legacy_carved.elf` opens at `MIPS:LE:32:default` with the 32-bit pin (Ghidra 12.1.3). It finds 2 procedures, not 3, because the symbol table is in the missing sections. The change is in `hasLegacyMipsHeaderShape`: ```ts const isCarvedSectionTableLimitation = (limitation: string): boolean => limitation.startsWith("MIPS section header table at file offset "); // ... abiFlags === null && (limitations === undefined || limitations.every(isCarvedSectionTableLimitation)) ``` It replaces the `GhidraMipsProfile.test.ts` row that expects this case to be refused with one that checks admission, the pin and the kept limitation. A program-header limitation is still refused. The `src/ghidra`, `src/config`, MIPS boundary and Ghidra launcher suites pass (654 passed, 4 skipped). Matching on the message prefix is the smallest change. A structured limitation kind from `MipsElfMetadata` would be cleaner if you prefer that. I'm happy to open a PR either way, with a `verify:ghidra:mips` round built like the snippet above. ### Impact and scope It affects legacy MIPS32 firmware carved from flash or memory dumps: the ELF keeps its original `e_shoff`, but the section table was never captured. Complete files are unaffected, and so are files that declare an ABI, since #1629 already covers carves that carry a record. ### REA version source revision `58300885` (`rea --version`: 6.4.0)

bsakaguchi added the

enhancement

label

2026-10-15 18:31:04 +00:00

bsakaguchi commented

2026-10-15 18:31:05 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Claiming this. Scope I am taking: exactly the admission rule you proposed, no more — hasLegacyMipsHeaderShape admits the recordless legacy shape when every inspection limitation is a section-header-table-past-EOF limitation, program-header limitations still refuse, the kept limitation stays in the profile (abi_flags_limitations) and Evidence next to the o32 assumption, and the pinned GhidraMipsProfile.test.ts row flips from refusal to admission with the pin and the kept limitation. I will implement it as a structured limitation signal from MipsElfMetadata rather than matching the message prefix, unless that turns out to reshape more callers than the prefix match.

Out of scope, unchanged: the recordless rule for complete files (#1898), carves that carry a record (#1629), any non-MIPS lane, and extraction/inventory behavior.

Verification split, stated up front: I have no Ghidra or a MIPS-capable clang here, so my PR will claim the unit/boundary red→green plus the focused src/ghidra, src/config, and MIPS-adjacent suites, and cite your Ghidra 12.1.3 observation (legacy_carved.elf opening at MIPS:LE:32:default with the 32-bit pin, 2 procedures) as the real-provider evidence; I cannot run verify:ghidra:mips locally. If you are mid-PR on your local patch, say so and I will stand down — first claim per the house rule, but your carried patch and real-provider round outrank my mechanism-level one.

@akushonkamen commented on GitHub (Oct 14, 2026): Claiming this. Scope I am taking: exactly the admission rule you proposed, no more — `hasLegacyMipsHeaderShape` admits the recordless legacy shape when every inspection limitation is a section-header-table-past-EOF limitation, program-header limitations still refuse, the kept limitation stays in the profile (`abi_flags_limitations`) and Evidence next to the o32 assumption, and the pinned `GhidraMipsProfile.test.ts` row flips from refusal to admission with the pin and the kept limitation. I will implement it as a structured limitation signal from `MipsElfMetadata` rather than matching the message prefix, unless that turns out to reshape more callers than the prefix match. Out of scope, unchanged: the recordless rule for complete files (#1898), carves that carry a record (#1629), any non-MIPS lane, and extraction/inventory behavior. Verification split, stated up front: I have no Ghidra or a MIPS-capable clang here, so my PR will claim the unit/boundary red→green plus the focused `src/ghidra`, `src/config`, and MIPS-adjacent suites, and cite your Ghidra 12.1.3 observation (`legacy_carved.elf` opening at `MIPS:LE:32:default` with the 32-bit pin, 2 procedures) as the real-provider evidence; I cannot run `verify:ghidra:mips` locally. If you are mid-PR on your local patch, say so and I will stand down — first claim per the house rule, but your carried patch and real-provider round outrank my mechanism-level one.

bsakaguchi commented

2026-10-15 18:31:05 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Delivery update. The fix is complete and validated on branch akushonkamen:fix/mips-carved-recordless-admission (commit 3d6c80e3508724cf31e8b184bf70c474f6a8ed12, based on main at ec67aebed). Opening the pull request from this account is still rejected by the repository (GraphQL: CreatePullRequest permission error; REST returns 404, retried this session), while issue comments and pushes to my fork work — so the branch and this summary stand in for the PR.

What the branch contains

src/application/MipsElfMetadata.ts: the truncation wording now lives beside its producer as the exported MIPS_SECTION_TABLE_TRUNCATION_PREFIX, and each declaration table carries its own limitation prefix, so admission cannot drift from the recorded fact (the co-located-prefix variant rather than your message-prefix match; a fully structured limitation kind would ripple into the serialized BinaryTarget contract — limitations is a readonly string[] in binaryTargetTypes.ts consumed by worker schemas and Evidence — which this fix does not need).
src/ghidra/GhidraMipsProfile.ts: hasLegacyMipsHeaderShape accepts the recordless shape when limitations is empty or every entry is the section-table truncation; the pinned 32-bit language, mips_support_lane: elf32-exec-arch32-legacy-pinned-v1, mips_abi_basis: inferred-o32-assumption-v1 and the preserved abi_flags_limitations follow unchanged, and ghidraMipsUnsupportedReason still refuses any other limitation (program-header and mixed-limitation refusals are pinned by new test rows).
src/ghidra/GhidraMipsProfile.test.ts: the pinned refusal row for the section-table limitation is replaced by refusal rows for a program-header limitation and a section-table-plus-other-limitation mix, plus a new recordless legacy MIPS32 carve admission describe with your examined 0 of 24 carve shape asserting support, the pin, the lane, the o32-assumption basis and the kept limitation.

Executed validation (all commands run locally on the branch, macOS arm64, Node 24.14.0):

npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts — 3 files, 131 passed. On the unmodified tree the two new carve-admission tests fail (2 failed | 64 passed), reproducing the refusal you observed.
npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts src/ghidra/GhidraPeProfile.test.ts src/ghidra/GhidraRiscvProfile.test.ts src/ghidra/GhidraPowerPcProfile.test.ts — 4 files, 98 passed.
npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraDoctor.test.ts — 2 files, 81 passed.
npm run format:check — pass. npx tsc --noEmit -p tsconfig.json — clean. npm run lint — 0 errors, module boundaries pass. npx knip — clean. npm run jscpd — 3.19% under the 3.5% threshold. npm run docs:check — pass.

Not claimed: the end-to-end Ghidra effect. I have no Ghidra or MIPS-capable clang here, so the real-provider evidence remains your Ghidra 12.1.3 round (legacy_carved.elf opening at MIPS:LE:32:default with the 32-bit pin, 2 procedures); verify:ghidra:mips was not run and I have not claimed it. Your carried patch and this branch implement the same rule — if you open your own PR, the only deltas worth a look are the exported prefix (drift guard) and the two extra refusal-pinning test rows.

@akushonkamen commented on GitHub (Oct 14, 2026): **Delivery update.** The fix is complete and validated on branch `akushonkamen:fix/mips-carved-recordless-admission` (commit `3d6c80e3508724cf31e8b184bf70c474f6a8ed12`, based on `main` at `ec67aebed`). Opening the pull request from this account is still rejected by the repository (GraphQL: `CreatePullRequest` permission error; REST returns 404, retried this session), while issue comments and pushes to my fork work — so the branch and this summary stand in for the PR. **What the branch contains** - `src/application/MipsElfMetadata.ts`: the truncation wording now lives beside its producer as the exported `MIPS_SECTION_TABLE_TRUNCATION_PREFIX`, and each declaration table carries its own limitation prefix, so admission cannot drift from the recorded fact (the co-located-prefix variant rather than your message-prefix match; a fully structured limitation kind would ripple into the serialized `BinaryTarget` contract — `limitations` is a `readonly string[]` in `binaryTargetTypes.ts` consumed by worker schemas and Evidence — which this fix does not need). - `src/ghidra/GhidraMipsProfile.ts`: `hasLegacyMipsHeaderShape` accepts the recordless shape when `limitations` is empty or every entry is the section-table truncation; the pinned 32-bit language, `mips_support_lane: elf32-exec-arch32-legacy-pinned-v1`, `mips_abi_basis: inferred-o32-assumption-v1` and the preserved `abi_flags_limitations` follow unchanged, and `ghidraMipsUnsupportedReason` still refuses any other limitation (program-header and mixed-limitation refusals are pinned by new test rows). - `src/ghidra/GhidraMipsProfile.test.ts`: the pinned refusal row for the section-table limitation is replaced by refusal rows for a program-header limitation and a section-table-plus-other-limitation mix, plus a new `recordless legacy MIPS32 carve admission` describe with your `examined 0 of 24` carve shape asserting support, the pin, the lane, the o32-assumption basis and the kept limitation. **Executed validation** (all commands run locally on the branch, macOS arm64, Node 24.14.0): - `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts` — 3 files, **131 passed**. On the unmodified tree the two new carve-admission tests fail (`2 failed | 64 passed`), reproducing the refusal you observed. - `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts src/ghidra/GhidraPeProfile.test.ts src/ghidra/GhidraRiscvProfile.test.ts src/ghidra/GhidraPowerPcProfile.test.ts` — 4 files, **98 passed**. - `npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraDoctor.test.ts` — 2 files, **81 passed**. - `npm run format:check` — pass. `npx tsc --noEmit -p tsconfig.json` — clean. `npm run lint` — 0 errors, module boundaries pass. `npx knip` — clean. `npm run jscpd` — 3.19% under the 3.5% threshold. `npm run docs:check` — pass. **Not claimed:** the end-to-end Ghidra effect. I have no Ghidra or MIPS-capable clang here, so the real-provider evidence remains your Ghidra 12.1.3 round (`legacy_carved.elf` opening at `MIPS:LE:32:default` with the 32-bit pin, 2 procedures); `verify:ghidra:mips` was not run and I have not claimed it. Your carried patch and this branch implement the same rule — if you open your own PR, the only deltas worth a look are the exported prefix (drift guard) and the two extra refusal-pinning test rows.

bsakaguchi commented

2026-10-15 18:31:05 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Correction to my delivery comment above. The branch I announced there was accidentally stacked: commit 3d6c80e35 sat on top of my separate, still-open #3109 work (9ae7c7798 was a merge of 97c25283a — the setup-symlink fix — with main), so its diff against main carried ~148 lines of unrelated #3057 changes, and the comment's "based on main at ec67aebed" misdescribed that ancestry. That is fixed: the branch has been rebased onto current main (500e32399) and force-pushed.

Branch: akushonkamen:fix/mips-carved-recordless-admission, now exactly one commit 81b847e7573633175ae2af28558511a1295e3ec3 on main@500e32399.
git diff origin/main...HEAD --stat now contains only the three #3155 files: src/application/MipsElfMetadata.ts (+124/-40 across the rework), src/ghidra/GhidraMipsProfile.ts (+19/-1... exact counts below), src/ghidra/GhidraMipsProfile.test.ts (+55) — 158 insertions, 40 deletions, no other paths.
Content of the three files is byte-identical to the previously validated fix (git diff 3d6c80e35 HEAD over those paths is empty).
Test groups re-run on the rebased branch: npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts — 3 files, 131 passed; the four profile files (GhidraMipsProfile/GhidraPeProfile/GhidraRiscvProfile/GhidraPowerPcProfile) — 98 passed; GhidraProvider.behavior.test.ts + GhidraDoctor.test.ts — 81 passed. Gates re-run: npm run format:check pass, npx tsc --noEmit clean, npx oxlint on the three files 0 errors, npx knip clean.

Everything else in the delivery comment (scope, red→green evidence, the split of real-provider evidence — yours — versus my mechanism-level validation) is unchanged. PR creation from this account remains repository-blocked (GraphQL CreatePullRequest permission error; REST 404, retested this session), so the branch remains the deliverable.

@akushonkamen commented on GitHub (Oct 14, 2026): **Correction to my delivery comment above.** The branch I announced there was accidentally stacked: commit `3d6c80e35` sat on top of my separate, still-open #3109 work (`9ae7c7798` was a merge of `97c25283a` — the setup-symlink fix — with `main`), so its diff against `main` carried ~148 lines of unrelated #3057 changes, and the comment's "based on main at `ec67aebed`" misdescribed that ancestry. That is fixed: the branch has been rebased onto current `main` (`500e32399`) and force-pushed. - Branch: `akushonkamen:fix/mips-carved-recordless-admission`, now exactly one commit `81b847e7573633175ae2af28558511a1295e3ec3` on `main@500e32399`. - `git diff origin/main...HEAD --stat` now contains only the three #3155 files: `src/application/MipsElfMetadata.ts` (+124/-40 across the rework), `src/ghidra/GhidraMipsProfile.ts` (+19/-1... exact counts below), `src/ghidra/GhidraMipsProfile.test.ts` (+55) — 158 insertions, 40 deletions, no other paths. - Content of the three files is byte-identical to the previously validated fix (`git diff 3d6c80e35 HEAD` over those paths is empty). - Test groups re-run on the rebased branch: `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts` — 3 files, 131 passed; the four profile files (`GhidraMipsProfile`/`GhidraPeProfile`/`GhidraRiscvProfile`/`GhidraPowerPcProfile`) — 98 passed; `GhidraProvider.behavior.test.ts` + `GhidraDoctor.test.ts` — 81 passed. Gates re-run: `npm run format:check` pass, `npx tsc --noEmit` clean, `npx oxlint` on the three files 0 errors, `npx knip` clean. Everything else in the delivery comment (scope, red→green evidence, the split of real-provider evidence — yours — versus my mechanism-level validation) is unchanged. PR creation from this account remains repository-blocked (GraphQL `CreatePullRequest` permission error; REST 404, retested this session), so the branch remains the deliverable.

bsakaguchi commented

2026-10-15 18:31:06 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Exact per-file counts for the record (git diff origin/main...HEAD --stat at 81b847e75): src/application/MipsElfMetadata.ts 124 changed lines (84 insertions, 40 deletions — the file reworks the reader/table plumbing, so the line count exceeds the semantic delta), src/ghidra/GhidraMipsProfile.ts 20 (19 insertions, 1 deletion), src/ghidra/GhidraMipsProfile.test.ts 55 (all insertions). Total 199 changed lines: 158 insertions, 40 deletions; no other paths touched.

@akushonkamen commented on GitHub (Oct 14, 2026): Exact per-file counts for the record (`git diff origin/main...HEAD --stat` at `81b847e75`): `src/application/MipsElfMetadata.ts` 124 changed lines (84 insertions, 40 deletions — the file reworks the reader/table plumbing, so the line count exceeds the semantic delta), `src/ghidra/GhidraMipsProfile.ts` 20 (19 insertions, 1 deletion), `src/ghidra/GhidraMipsProfile.test.ts` 55 (all insertions). Total 199 changed lines: 158 insertions, 40 deletions; no other paths touched.

bsakaguchi commented

2026-10-15 18:31:06 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

Correction to my previous comment: I transcribed the per-file split from the summary bar instead of --numstat. Verified numbers at 81b847e75 (git diff origin/main...HEAD --numstat): src/application/MipsElfMetadata.ts 86 insertions, 38 deletions; src/ghidra/GhidraMipsProfile.test.ts 55 insertions, 0 deletions; src/ghidra/GhidraMipsProfile.ts 17 insertions, 2 deletions. Totals stand at 158 insertions, 40 deletions, three files, no other paths.

@akushonkamen commented on GitHub (Oct 14, 2026): Correction to my previous comment: I transcribed the per-file split from the summary bar instead of `--numstat`. Verified numbers at `81b847e75` (`git diff origin/main...HEAD --numstat`): `src/application/MipsElfMetadata.ts` **86 insertions, 38 deletions**; `src/ghidra/GhidraMipsProfile.test.ts` 55 insertions, 0 deletions; `src/ghidra/GhidraMipsProfile.ts` **17 insertions, 2 deletions**. Totals stand at 158 insertions, 40 deletions, three files, no other paths.

bsakaguchi commented

2026-10-15 18:31:06 +00:00

Author

Owner

@akushonkamen commented on GitHub (Oct 14, 2026):

PR attempt + fresh validation (this session). Opening the PR from this account was retried just now and is still rejected — GraphQL: CreatePullRequest permission error; REST returns 404 — so the PR could not be opened from here. akushonkamen:fix/mips-carved-recordless-admission at 81b847e75 is ready for a maintainer to open against main (500e32399; branch re-verified in sync with main at fetch time — same commit, patch-id-identical to the previously announced 3d6c80e35). Closes #3155.

All gates re-run green on the rebased branch in this session:

npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts — 3 files, 131 passed.
npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts src/ghidra/GhidraPeProfile.test.ts src/ghidra/GhidraRiscvProfile.test.ts src/ghidra/GhidraPowerPcProfile.test.ts — 4 files, 98 passed.
npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraDoctor.test.ts — 2 files, 81 passed.
npm run format:check pass; npx tsc --noEmit clean; npm run lint 0 errors + module boundaries pass; npx knip clean; npm run jscpd 3.19% (< 3.5%); npm run docs:check 5 passed.

Red-first re-verified first-hand: with this branch's test file applied to the unmodified base 500e32399, npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts gives 2 failed | 64 passed (66); on the branch, 66/66.

Not claimed: no real Ghidra headless run and no real carved-firmware artifact in this environment — coverage is fixture-level inspection/admission tests; the reporter's end-to-end confirmation on Ghidra 12.1.3 stands in #3155.

@akushonkamen commented on GitHub (Oct 14, 2026): **PR attempt + fresh validation (this session).** Opening the PR from this account was retried just now and is still rejected — GraphQL: `CreatePullRequest` permission error; REST returns 404 — so the PR could not be opened from here. `akushonkamen:fix/mips-carved-recordless-admission` at `81b847e75` is ready for a maintainer to open against `main` (`500e32399`; branch re-verified in sync with `main` at fetch time — same commit, patch-id-identical to the previously announced `3d6c80e35`). Closes #3155. All gates re-run green on the rebased branch in this session: - `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts tests/boundary/filesystem/mipsAbiFlags.test.ts tests/boundary/filesystem/mipsSharedLoad.test.ts` — 3 files, **131 passed**. - `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts src/ghidra/GhidraPeProfile.test.ts src/ghidra/GhidraRiscvProfile.test.ts src/ghidra/GhidraPowerPcProfile.test.ts` — 4 files, **98 passed**. - `npm run test:focused -- src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraDoctor.test.ts` — 2 files, **81 passed**. - `npm run format:check` pass; `npx tsc --noEmit` clean; `npm run lint` 0 errors + module boundaries pass; `npx knip` clean; `npm run jscpd` 3.19% (< 3.5%); `npm run docs:check` 5 passed. Red-first re-verified first-hand: with this branch's test file applied to the unmodified base `500e32399`, `npm run test:focused -- src/ghidra/GhidraMipsProfile.test.ts` gives **2 failed | 64 passed (66)**; on the branch, 66/66. Not claimed: no real Ghidra headless run and no real carved-firmware artifact in this environment — coverage is fixture-level inspection/admission tests; the reporter's end-to-end confirmation on Ghidra 12.1.3 stands in #3155.

bsakaguchi commented

2026-10-15 18:31:06 +00:00

Author

Owner

@bqw371 commented on GitHub (Oct 14, 2026):

TL;DR: Verified 81b847e75 on real Ghidra (12.1.3 and 12.1.4, both byte orders, CLI and MCP): the recordless carve opens pinned to 32-bit, its entry function decompiles cleanly, and the unread section table is recorded as a limitation in the profile and in Evidence, exactly as the issue asks. Red→green reproduces. Rebased onto main@29d3dcb7e (after 938728227; one import-block conflict, every patch line unchanged) and verified again there. verify:ghidra:mips is red on the branch, but identically red on its base: that is d4bd65c53's documented refusal of record-carrying carves (already noted in #3096), not this change. Two notes at the end: one for maintainers, one about who opens the PR.

Thanks for taking it, @akushonkamen. Your branch is the same rule we carry locally, with the prefix exported from its producer, which is better than our prefix match. I built it and ran the real-provider side you could not.

What I ran on the branch (no local patch)

Fixtures: your tree's tests/conformance/c/mips.c + mips-probe.S, the issue's clang line with -march=mips32, once for mipsel-linux-gnu and once for mips-linux-gnu, then the issue's make_carve.py (ABI field cleared, ABI-flags record retyped away, cut at e_shoff - 1; program headers intact, 24 section headers unread). Each file went through rea analyze --provider ghidra --json and through the MCP server (open_binary with provider_id: ghidra, inspect_native_load_image, list_procedures, procedure_assembly and analyze_function at e_entry 0x201b0, close_binary), the surface this issue was reported against.

File	12.1.3	12.1.4
LE uncut	`MIPS:LE:32:default`, 3 procedures	same
LE carve	`MIPS:LE:32:default`, 2 procedures, limitation kept	same
BE uncut	`MIPS:BE:32:default`, 3 procedures	same
BE carve	`MIPS:BE:32:default`, 2 procedures, limitation kept	same

Every carve reports language_selection: provider-pinned-v1, mips_support_lane: elf32-exec-arch32-legacy-pinned-v1, mips_abi_basis: inferred-o32-assumption-v1, the section-table limitation in mips_elf.abi_flags_limitations, and the Evidence envelope carries it once, next to the existing o32-assumption and FP-unverified lines. The carves lose one procedure because the symbol table is in the missing sections; the entry is then FUN_000201b0.

Decode health on the carves: the entry function decompiles without halt_baddata(), and inspect_native_instruction at 0x20230 reports jr ra decoded (0800e003 LE, 03e00008 BE, flow return). That is the word #1620 saw fail to decode under the auto-selected 64-bit language, so the pin is doing its job on the carved shape too.

Cross-check: the same two carves through our installed build (main@58300885 plus our local rule) give the same language, lane, basis, procedures and limitations. The branch and the local patch agree on every observable.

Real firmware: on our own carved MIPS32 images of this shape (recordless legacy header, section table past EOF, refused by main today), every file opens at MIPS:LE:32:default with function entry sets equal to our plain-analyzeHeadless reference lists; a complete file of the same family still matches its reference; a caller-selected MIPS:LE:64:64-32R6addr is still refused as unsupported_target.

Tests: your three focused files, 131 passed. Your test file against GhidraMipsProfile.ts from 500e32399: 2 failed / 64 passed, the two new carve-admission tests. Rebased onto main@29d3dcb7e, which brought 938728227 (GNU ABI attributes) into both files you touch: the only conflict was the import block, the added and removed lines of all three files are identical to 81b847e75, tsc is clean, your three files plus MipsGnuAttributes.test.ts give 161 passed, your test file against today's GhidraMipsProfile.ts gives 2 failed / 65 passed, and the fixture round and our real-firmware check repeat on the rebased tree with the same results.

`verify:ghidra:mips`: red on the branch and on its base

On both Ghidra versions the verifier stops at the "MIPS little hard-float" carved step: its own mips32r2 + o32 + ABI-record file, cut before the section table, is refused:

MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown. MIPS section header table at file offset 3636 extends beyond the captured file: examined 0 of 24 complete headers. ABI declarations in unavailable headers remain unknown.

Control: a clean checkout of 500e32399, same Ghidra 12.1.3, fails at the same step with the same reason. The refusal is d4bd65c53 (fix(providers)!), which refuses any inspection limitation in the declared-ABI path even with a valid program-header record and rewrote ghidra-mips.md to say so; #3096 already notes that this leaves the verifier red on main. So the proof lags a decision, and the branch is not the cause. Reproduced outside the verifier: a -march=mips32r2 build from your tree opens whole on elf32-exec-o32-arch32r2-standard-v3, and the same file cut at e_shoff - 1 is refused with the message above.

Because the verifier aborts there, its later steps did not run on the branch. The fixture rounds above stand in for them.

One note, not blocking

The program-header limitationPrefix and the two new refusal rows describe inputs readMipsElfMetadata cannot emit. The program header table is optional: false, so count is always table.count for it; a short table is a read error that BinaryTargetResolver.ts returns whole, never a limitation. The only limitations.push is the section-table one, and a record past EOF is also a read error, so "MIPS ABI record at file offset 4096 extends beyond the captured file" never occurs. Keep the rows as fail-closed pins for limitation kinds that do not exist yet, but the describe names and the comment in GhidraMipsProfile.ts read as if they were live shapes.

Two notes

For maintainers: is the recordless carve acceptable under d4bd65c53's line? That commit fails closed on a record-carrying carve because an unread section header could contradict the declared ABI. The recordless carve declares nothing to contradict: its lane already records o32 as an assumption (mips_abi_basis: inferred-o32-assumption-v1), and the unread headers stay in abi_flags_limitations and in Evidence, so the output asserts nothing it has not seen. If you want the same fail-closed line for both, say so and we keep carrying the rule locally.
The PR. @akushonkamen, your account opened #3092 and #3109 from the same fork earlier today, so the CreatePullRequest error may have been transient. If it works now, please open it yourself: the branch rebases onto current main with one conflict in the import block (keep both imports), and I will post the real-provider round on the PR. If it still fails, say so here and I will open it from my fork with 81b847e75 rebased the same way, every patch line yours, authorship kept. Either way I will re-run the fixture round on whatever lands.

Reproduction details

LE: legacy.elf 03f288b0…, legacy_carved.elf 1116ed74… (3799 bytes, e_shoff 3800). BE: legacy_be.elf ad445850…, legacy_carved_be.elf 50af4b5d… (2963 bytes, e_shoff 2964). e_flags 0x50000001, e_entry 0x201b0 in all four.
Record-carrying control: -march=mips32r2, e_flags 0x70001001, cut at 3799 bytes.
verify:ghidra:mips needs GNU readelf on PATH; with only LLVM tools present it dies with spawn readelf ENOENT before any Ghidra run.
Box: macOS arm64, Node 26.8.1, Homebrew Ghidra 12.1.3 and Ghidra 12.1.4 with JDK 21, Homebrew clang/LLD 23.1.0, GNU readelf 2.47 (mips-linux-gnu-binutils).

@bqw371 commented on GitHub (Oct 14, 2026): **TL;DR:** Verified `81b847e75` on real Ghidra (12.1.3 and 12.1.4, both byte orders, CLI and MCP): the recordless carve opens pinned to 32-bit, its entry function decompiles cleanly, and the unread section table is recorded as a limitation in the profile and in Evidence, exactly as the issue asks. Red→green reproduces. Rebased onto `main@29d3dcb7e` (after `938728227`; one import-block conflict, every patch line unchanged) and verified again there. `verify:ghidra:mips` is red on the branch, but identically red on its base: that is `d4bd65c53`'s documented refusal of record-carrying carves (already noted in #3096), not this change. Two notes at the end: one for maintainers, one about who opens the PR. Thanks for taking it, @akushonkamen. Your branch is the same rule we carry locally, with the prefix exported from its producer, which is better than our prefix match. I built it and ran the real-provider side you could not. ### What I ran on the branch (no local patch) Fixtures: your tree's `tests/conformance/c/mips.c` + `mips-probe.S`, the issue's `clang` line with `-march=mips32`, once for `mipsel-linux-gnu` and once for `mips-linux-gnu`, then the issue's `make_carve.py` (ABI field cleared, ABI-flags record retyped away, cut at `e_shoff - 1`; program headers intact, 24 section headers unread). Each file went through `rea analyze --provider ghidra --json` and through the MCP server (`open_binary` with `provider_id: ghidra`, `inspect_native_load_image`, `list_procedures`, `procedure_assembly` and `analyze_function` at `e_entry` `0x201b0`, `close_binary`), the surface this issue was reported against. | File | 12.1.3 | 12.1.4 | | --- | --- | --- | | LE uncut | `MIPS:LE:32:default`, 3 procedures | same | | LE carve | `MIPS:LE:32:default`, 2 procedures, limitation kept | same | | BE uncut | `MIPS:BE:32:default`, 3 procedures | same | | BE carve | `MIPS:BE:32:default`, 2 procedures, limitation kept | same | Every carve reports `language_selection: provider-pinned-v1`, `mips_support_lane: elf32-exec-arch32-legacy-pinned-v1`, `mips_abi_basis: inferred-o32-assumption-v1`, the section-table limitation in `mips_elf.abi_flags_limitations`, and the Evidence envelope carries it once, next to the existing o32-assumption and FP-unverified lines. The carves lose one procedure because the symbol table is in the missing sections; the entry is then `FUN_000201b0`. Decode health on the carves: the entry function decompiles without `halt_baddata()`, and `inspect_native_instruction` at `0x20230` reports `jr ra` decoded (`0800e003` LE, `03e00008` BE, flow `return`). That is the word #1620 saw fail to decode under the auto-selected 64-bit language, so the pin is doing its job on the carved shape too. Cross-check: the same two carves through our installed build (`main@58300885` plus our local rule) give the same language, lane, basis, procedures and limitations. The branch and the local patch agree on every observable. Real firmware: on our own carved MIPS32 images of this shape (recordless legacy header, section table past EOF, refused by `main` today), every file opens at `MIPS:LE:32:default` with function entry sets equal to our plain-`analyzeHeadless` reference lists; a complete file of the same family still matches its reference; a caller-selected `MIPS:LE:64:64-32R6addr` is still refused as `unsupported_target`. Tests: your three focused files, 131 passed. Your test file against `GhidraMipsProfile.ts` from `500e32399`: 2 failed / 64 passed, the two new carve-admission tests. Rebased onto `main@29d3dcb7e`, which brought `938728227` (GNU ABI attributes) into both files you touch: the only conflict was the import block, the added and removed lines of all three files are identical to `81b847e75`, `tsc` is clean, your three files plus `MipsGnuAttributes.test.ts` give 161 passed, your test file against today's `GhidraMipsProfile.ts` gives 2 failed / 65 passed, and the fixture round and our real-firmware check repeat on the rebased tree with the same results. ### `verify:ghidra:mips`: red on the branch and on its base On both Ghidra versions the verifier stops at the "MIPS little hard-float" carved step: its own `mips32r2` + o32 + ABI-record file, cut before the section table, is refused: ``` MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown. MIPS section header table at file offset 3636 extends beyond the captured file: examined 0 of 24 complete headers. ABI declarations in unavailable headers remain unknown. ``` Control: a clean checkout of `500e32399`, same Ghidra 12.1.3, fails at the same step with the same reason. The refusal is `d4bd65c53` (`fix(providers)!`), which refuses any inspection limitation in the declared-ABI path even with a valid program-header record and rewrote [ghidra-mips.md](https://github.com/morluto/rea/blob/main/docs/ghidra-mips.md) to say so; #3096 already notes that this leaves the verifier red on `main`. So the proof lags a decision, and the branch is not the cause. Reproduced outside the verifier: a `-march=mips32r2` build from your tree opens whole on `elf32-exec-o32-arch32r2-standard-v3`, and the same file cut at `e_shoff - 1` is refused with the message above. Because the verifier aborts there, its later steps did not run on the branch. The fixture rounds above stand in for them. ### One note, not blocking The program-header `limitationPrefix` and the two new refusal rows describe inputs `readMipsElfMetadata` cannot emit. The program header table is `optional: false`, so `count` is always `table.count` for it; a short table is a read error that `BinaryTargetResolver.ts` returns whole, never a limitation. The only `limitations.push` is the section-table one, and a record past EOF is also a read error, so "MIPS ABI record at file offset 4096 extends beyond the captured file" never occurs. Keep the rows as fail-closed pins for limitation kinds that do not exist yet, but the describe names and the comment in `GhidraMipsProfile.ts` read as if they were live shapes. ### Two notes 1. **For maintainers: is the recordless carve acceptable under `d4bd65c53`'s line?** That commit fails closed on a record-carrying carve because an unread section header could contradict the declared ABI. The recordless carve declares nothing to contradict: its lane already records o32 as an assumption (`mips_abi_basis: inferred-o32-assumption-v1`), and the unread headers stay in `abi_flags_limitations` and in Evidence, so the output asserts nothing it has not seen. If you want the same fail-closed line for both, say so and we keep carrying the rule locally. 2. **The PR.** @akushonkamen, your account opened #3092 and #3109 from the same fork earlier today, so the `CreatePullRequest` error may have been transient. If it works now, please open it yourself: the branch rebases onto current `main` with one conflict in the import block (keep both imports), and I will post the real-provider round on the PR. If it still fails, say so here and I will open it from my fork with `81b847e75` rebased the same way, every patch line yours, authorship kept. Either way I will re-run the fixture round on whatever lands. <details> <summary>Reproduction details</summary> - LE: `legacy.elf` `03f288b0…`, `legacy_carved.elf` `1116ed74…` (3799 bytes, `e_shoff` 3800). BE: `legacy_be.elf` `ad445850…`, `legacy_carved_be.elf` `50af4b5d…` (2963 bytes, `e_shoff` 2964). `e_flags` `0x50000001`, `e_entry` `0x201b0` in all four. - Record-carrying control: `-march=mips32r2`, `e_flags` `0x70001001`, cut at 3799 bytes. - `verify:ghidra:mips` needs GNU `readelf` on `PATH`; with only LLVM tools present it dies with `spawn readelf ENOENT` before any Ghidra run. - Box: macOS arm64, Node 26.8.1, Homebrew Ghidra 12.1.3 and Ghidra 12.1.4 with JDK 21, Homebrew clang/LLD 23.1.0, GNU readelf 2.47 (`mips-linux-gnu-binutils`). </details>

bsakaguchi referenced this issue

2026-10-16 21:02:34 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:08:39 +00:00

[PR #1102] [MERGED] fix(javascript): parse plain TypeScript artifact dialects #2650

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-10-31 00:29:57 +00:00