morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #4626] feat(native): inspect 16-bit NE executables #6338

New issue

Open

opened 2026-10-31 01:13:41 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:13:41 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/4626
Author: @dfl
Created: 10/17/2026
Status: 🔄 Open

Base: main ← Head: feature/ne-format

📝 Commits (3)

6059bb8 feat(native): inspect 16-bit NE executables
4e10f5a feat(formats): classify NE images as artifacts for NE inspection
45d502d fix(native): accept NE resources as VB3 linkers write them

📊 Changes

38 files changed (+1981 additions, -6 deletions)

View changed files

📝 .agents/skills/reverse-engineer-anything/SKILL.md (+3 -0)
📝 .agents/skills/reverse-engineer-anything/references/native-and-artifacts.md (+12 -0)
📝 docs/binary-diagnostics.md (+54 -0)
📝 docs/ghidra-dos.md (+4 -1)
📝 scripts/lib/catalog-builders.mjs (+9 -1)
📝 src/application/BinaryTargetResolver.ts (+46 -0)
📝 src/application/CapabilityInventory.ts (+1 -1)
📝 src/application/InvestigationProviders.ts (+7 -0)
➕ src/application/binaryDiagnostics/NeExecutablePort.ts (+20 -0)
➕ src/application/binaryDiagnostics/NeExecutableService.test.ts (+77 -0)
➕ src/application/binaryDiagnostics/NeExecutableService.ts (+150 -0)
📝 src/artifacts/inventory/ArtifactGraphConstruction.ts (+6 -0)
📝 src/cli/binaryDiagnosticsCommands.ts (+36 -0)
📝 src/cliCommandNames.ts (+1 -0)
📝 src/composition/binaryDiagnostics.ts (+6 -0)
📝 src/contracts/native/binaryDiagnosticsToolContracts.ts (+19 -0)
📝 src/contracts/toolEffects.test.ts (+1 -0)
📝 src/contracts/toolEffects.ts (+1 -0)
📝 src/contracts/toolOutputSchemaPrimitives.ts (+1 -0)
📝 src/domain/analysisSnapshot.ts (+1 -0)

...and 18 more files

📄 Description

Summary

REA rejected every 16-bit Windows (NE) executable during target resolution
("unsupported NE executable"), so Windows 3.x programs and DLLs, including
Visual Basic 3 applications, had no inspection path at all. This adds a
portable, provider-free inspect_ne_executable / rea inspect-ne-executable
and routes NE images to it through artifact inventory.

Problem and expected behavior

Before: an MZ image whose new-header pointer declares NE failed admission;
inventory reported unknown.
After: inventory classifies it as ne, inspect_artifact proposes
inspect_ne_executable, and that operation reports the NE header, module
name and description, segments (with relocation-record counts and ranges),
resources, resident and non-resident exported names, module references and
the entry table, each with its original file offset and length.
open_binary still does not load NE: no session provider analyzes it, so the
provider rejections are reported. LE and LX remain unsupported.

Change and scope

src/native/ne/NeExecutableParser.ts parses the NE tables with bounded reads,
row and 8 MiB metadata budgets and cancellation, following the
inspect_pe_resources stack (port, service output invariants, provider,
CLI/MCP registration).
Names are decoded byte-for-byte as latin1 (the image does not record its code
page). Segment data, relocation records and resource payloads stay opaque, so
imported function names are not reported yet.
Malformed tables fail without partial success. Two real-world linker
behaviours are accepted rather than rejected (third commit): resource lengths
are stored in alignment units, so a final resource may declare up to one unit
past the end of the file (VB3 does this); its location then covers only the
bytes present, with declared_bytes and a limitation. A resource with
offset 0 has location: null, and a zero name word is a none identity
instead of a name read from the table's alignment word.
Not included: relocation decoding, a Ghidra NE loader, VB3 p-code analysis.

Scope: read-only static format inspection, which CONTRIBUTING lists as a
supported contribution. It never executes, attaches to or modifies the target.
No maintainer scope decision is linked; maintainers may judge one is needed for
a new tool.

Contract and boundary impact

Semantic owner and earliest changed stage: src/native/ne/NeExecutableParser.ts
owns NE table interpretation; src/domain/dosMz.ts consumers and
BinaryTargetResolver route NE images to it.
Contribution scope fit and linked maintainer decision: format inspection; none linked.
CLI and MCP/tool-catalog contract: new inspect_ne_executable tool and
inspect-ne-executable command; ne added to artifact, analysis snapshot,
target format and Evidence subject format enums.
Provider, bridge, target-format, or platform compatibility: NE now resolves as
an artifact instead of failing admission; Hopper and Ghidra session behaviour
unchanged.
Evidence, artifact, provenance, or reconstruction contract: Evidence records
the stable artifact path, byte count and SHA-256; every fact carries its file range.
Target selection, authority, upstream behavior, and cleanup details: reads an
explicit local path through the existing artifact reader and resource scope.
Generated metadata, package, or installation impact: catalog builder entry
updated in scripts/lib/catalog-builders.mjs; skill reference updated in
.agents/skills/reverse-engineer-anything/.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated: parser tests with hand-authored fixtures (including
corrupt tables, budgets, a 64 KiB zero-length segment and the three linker
cases above), service output invariants, filesystem boundary, inventory and
target diagnostics, and CLI/MCP acceptance.
Base reproduction: on main, inspect_artifact on an NE image reports it as
unsupported. Before the third commit, a VB3-compiled 14,095-byte EXE failed
with NE resource at 14080 + 256 leaves the artifact.; the new parser test
truncates the fixture inside its last alignment unit and fails without the fix.
Executed on real files (not committed): 110 NE images, including the VB3 IDE
and runtime, 29 VBX controls, Polaris (a VB3 application) and its DLLs, and a
compiled VB3 test program; all parse, and segment counts and RCDATA offsets
agree with an independent parser.
User-visible output (rea inspect-ne-executable TYPES.EXE --format json, excerpt
of normalized_result):

{
  "module_references": [{ "index": 1, "name": "VBRUN300", "location": { "offset": 1787, "bytes": 9 } }],
  "resources": [
    { "type": { "kind": "id", "id": 10 }, "name": { "kind": "id", "id": 5 }, "flags": 7216,
      "location": { "offset": 14080, "bytes": 15 }, "declared_bytes": 256,
      "entry_location": { "offset": 1746, "bytes": 12 } },
    { "type": { "kind": "id", "id": 10 }, "name": { "kind": "none" }, "flags": 0,
      "location": null, "declared_bytes": 0, "entry_location": { "offset": 1758, "bytes": 12 } }
  ],
  "coverage": { "status": "complete", "examined_rows": 16 },
  "limitations": ["…", "1 resource's declared length passes the end of the file by less than one alignment unit; its location covers only the bytes present."]
}

Remaining proof gaps: no NE binaries are committed (REA does not commit
binaries), so real-file coverage is the manual run above; OS/2 NE images were
not available.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

On macOS 27.0.1 (arm64), Node 24.18.0, branch rebased onto main at 0c7b6d8:

npm run check:pr: typecheck, lint:built, format:check, knip and
metadata:check passed; test:run 17,026 passed, 344 skipped, 3 failed. The 3
failures (tests/process-global/binaryTargetAdmissionCancellation.test.ts,
tests/boundary/providers/kuna/kunaProvider.test.ts,
tests/boundary/providers/managed/managedHashCancellation.test.ts, one test
each) fail identically on unmodified main at 0c7b6d8 in this environment.
The steps check:pr skips after a failure, run separately: npm run test:python
passed (35 and 71 tests), npm run docs:check passed, npm run jscpd passed
(3.34% duplicated lines), npm run scan:todos passed.
rea inspect-ne-executable on the 110 real NE files above: all succeeded.

Compatibility, safety, and release

Breaking changes or migration steps: none; NE images previously failed admission.
Real Hopper/Ghidra, browser, or OS coverage: not applicable; no provider involved.
Package or release metadata impact: one new tool in the catalog.
Security, privacy, process, or containment review: parses untrusted bytes with
bounded reads and budgets; no execution, network or writes.

Author readiness

Human owner:

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/4626 **Author:** [@dfl](https://github.com/dfl) **Created:** 10/17/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feature/ne-format` --- ### 📝 Commits (3) - [`6059bb8`](https://github.com/morluto/rea/commit/6059bb8ebfb063ca6319cb1ea2a69364101a9f97) feat(native): inspect 16-bit NE executables - [`4e10f5a`](https://github.com/morluto/rea/commit/4e10f5ad400f351d5eba20b26cb07b48afee1f9b) feat(formats): classify NE images as artifacts for NE inspection - [`45d502d`](https://github.com/morluto/rea/commit/45d502dc13c475771b10b92010bcea0b7f08adcb) fix(native): accept NE resources as VB3 linkers write them ### 📊 Changes **38 files changed** (+1981 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `.agents/skills/reverse-engineer-anything/SKILL.md` (+3 -0) 📝 `.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md` (+12 -0) 📝 `docs/binary-diagnostics.md` (+54 -0) 📝 `docs/ghidra-dos.md` (+4 -1) 📝 `scripts/lib/catalog-builders.mjs` (+9 -1) 📝 `src/application/BinaryTargetResolver.ts` (+46 -0) 📝 `src/application/CapabilityInventory.ts` (+1 -1) 📝 `src/application/InvestigationProviders.ts` (+7 -0) ➕ `src/application/binaryDiagnostics/NeExecutablePort.ts` (+20 -0) ➕ `src/application/binaryDiagnostics/NeExecutableService.test.ts` (+77 -0) ➕ `src/application/binaryDiagnostics/NeExecutableService.ts` (+150 -0) 📝 `src/artifacts/inventory/ArtifactGraphConstruction.ts` (+6 -0) 📝 `src/cli/binaryDiagnosticsCommands.ts` (+36 -0) 📝 `src/cliCommandNames.ts` (+1 -0) 📝 `src/composition/binaryDiagnostics.ts` (+6 -0) 📝 `src/contracts/native/binaryDiagnosticsToolContracts.ts` (+19 -0) 📝 `src/contracts/toolEffects.test.ts` (+1 -0) 📝 `src/contracts/toolEffects.ts` (+1 -0) 📝 `src/contracts/toolOutputSchemaPrimitives.ts` (+1 -0) 📝 `src/domain/analysisSnapshot.ts` (+1 -0) _...and 18 more files_ </details> ### 📄 Description ## Summary REA rejected every 16-bit Windows (NE) executable during target resolution ("unsupported NE executable"), so Windows 3.x programs and DLLs, including Visual Basic 3 applications, had no inspection path at all. This adds a portable, provider-free `inspect_ne_executable` / `rea inspect-ne-executable` and routes NE images to it through artifact inventory. ## Problem and expected behavior - Before: an MZ image whose new-header pointer declares `NE` failed admission; inventory reported `unknown`. - After: inventory classifies it as `ne`, `inspect_artifact` proposes `inspect_ne_executable`, and that operation reports the NE header, module name and description, segments (with relocation-record counts and ranges), resources, resident and non-resident exported names, module references and the entry table, each with its original file offset and length. - `open_binary` still does not load NE: no session provider analyzes it, so the provider rejections are reported. LE and LX remain unsupported. ## Change and scope - `src/native/ne/NeExecutableParser.ts` parses the NE tables with bounded reads, row and 8 MiB metadata budgets and cancellation, following the `inspect_pe_resources` stack (port, service output invariants, provider, CLI/MCP registration). - Names are decoded byte-for-byte as latin1 (the image does not record its code page). Segment data, relocation records and resource payloads stay opaque, so imported function names are not reported yet. - Malformed tables fail without partial success. Two real-world linker behaviours are accepted rather than rejected (third commit): resource lengths are stored in alignment units, so a final resource may declare up to one unit past the end of the file (VB3 does this); its location then covers only the bytes present, with `declared_bytes` and a limitation. A resource with offset 0 has `location: null`, and a zero name word is a `none` identity instead of a name read from the table's alignment word. - Not included: relocation decoding, a Ghidra NE loader, VB3 p-code analysis. Scope: read-only static format inspection, which CONTRIBUTING lists as a supported contribution. It never executes, attaches to or modifies the target. No maintainer scope decision is linked; maintainers may judge one is needed for a new tool. ## Contract and boundary impact - Semantic owner and earliest changed stage: `src/native/ne/NeExecutableParser.ts` owns NE table interpretation; `src/domain/dosMz.ts` consumers and `BinaryTargetResolver` route NE images to it. - Contribution scope fit and linked maintainer decision: format inspection; none linked. - CLI and MCP/tool-catalog contract: new `inspect_ne_executable` tool and `inspect-ne-executable` command; `ne` added to artifact, analysis snapshot, target format and Evidence subject format enums. - Provider, bridge, target-format, or platform compatibility: NE now resolves as an artifact instead of failing admission; Hopper and Ghidra session behaviour unchanged. - Evidence, artifact, provenance, or reconstruction contract: Evidence records the stable artifact path, byte count and SHA-256; every fact carries its file range. - Target selection, authority, upstream behavior, and cleanup details: reads an explicit local path through the existing artifact reader and resource scope. - Generated metadata, package, or installation impact: catalog builder entry updated in `scripts/lib/catalog-builders.mjs`; skill reference updated in `.agents/skills/reverse-engineer-anything/`. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: parser tests with hand-authored fixtures (including corrupt tables, budgets, a 64 KiB zero-length segment and the three linker cases above), service output invariants, filesystem boundary, inventory and target diagnostics, and CLI/MCP acceptance. - Base reproduction: on `main`, `inspect_artifact` on an NE image reports it as unsupported. Before the third commit, a VB3-compiled 14,095-byte EXE failed with `NE resource at 14080 + 256 leaves the artifact.`; the new parser test truncates the fixture inside its last alignment unit and fails without the fix. - Executed on real files (not committed): 110 NE images, including the VB3 IDE and runtime, 29 VBX controls, Polaris (a VB3 application) and its DLLs, and a compiled VB3 test program; all parse, and segment counts and RCDATA offsets agree with an independent parser. - User-visible output (`rea inspect-ne-executable TYPES.EXE --format json`, excerpt of `normalized_result`): ```json { "module_references": [{ "index": 1, "name": "VBRUN300", "location": { "offset": 1787, "bytes": 9 } }], "resources": [ { "type": { "kind": "id", "id": 10 }, "name": { "kind": "id", "id": 5 }, "flags": 7216, "location": { "offset": 14080, "bytes": 15 }, "declared_bytes": 256, "entry_location": { "offset": 1746, "bytes": 12 } }, { "type": { "kind": "id", "id": 10 }, "name": { "kind": "none" }, "flags": 0, "location": null, "declared_bytes": 0, "entry_location": { "offset": 1758, "bytes": 12 } } ], "coverage": { "status": "complete", "examined_rows": 16 }, "limitations": ["…", "1 resource's declared length passes the end of the file by less than one alignment unit; its location covers only the bytes present."] } ``` - Remaining proof gaps: no NE binaries are committed (REA does not commit binaries), so real-file coverage is the manual run above; OS/2 NE images were not available. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed On macOS 27.0.1 (arm64), Node 24.18.0, branch rebased onto `main` at `0c7b6d8`: - `npm run check:pr`: typecheck, `lint:built`, `format:check`, knip and `metadata:check` passed; `test:run` 17,026 passed, 344 skipped, 3 failed. The 3 failures (`tests/process-global/binaryTargetAdmissionCancellation.test.ts`, `tests/boundary/providers/kuna/kunaProvider.test.ts`, `tests/boundary/providers/managed/managedHashCancellation.test.ts`, one test each) fail identically on unmodified `main` at `0c7b6d8` in this environment. - The steps `check:pr` skips after a failure, run separately: `npm run test:python` passed (35 and 71 tests), `npm run docs:check` passed, `npm run jscpd` passed (3.34% duplicated lines), `npm run scan:todos` passed. - `rea inspect-ne-executable` on the 110 real NE files above: all succeeded. ## Compatibility, safety, and release - Breaking changes or migration steps: none; NE images previously failed admission. - Real Hopper/Ghidra, browser, or OS coverage: not applicable; no provider involved. - Package or release metadata impact: one new tool in the catalog. - Security, privacy, process, or containment review: parses untrusted bytes with bounded reads and budgets; no execution, network or writes. ## Author readiness Human owner:  - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-31 01:13:41 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-11-01 21:11:00 +00:00

[GH-ISSUE #5653] [Tracker] New issue intake by area (October 2026) #3423