[PR #503] [MERGED] fix: correct native inventories and bridge error classification #2030

New issue

Closed

opened 2026-10-16 21:13:32 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:13:32 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/503
Author: @morluto
Created: 10/4/2026
Status: ✅ Merged
Merged: 10/4/2026
Merged by: @morluto

Base: main ← Head: codex/fix-hopper-inventory-addresses

📝 Commits (6)

c9ea330 fix(hopper): preserve normalized inventory addresses
064b1c1 test(hopper): isolate real inventory verification
24a3165 fix(native): preserve otool addresses and section alignment
e60ef19 fix: preserve address spaces in call path evidence
a56b2ef fix(hopper): distinguish request validation from provider faults
3075fdf fix(native): retain tool inventory facts and reject failed signatures

📊 Changes

34 files changed (+1009 additions, -161 deletions)

View changed files

📝 bridge/hopper_bridge.py (+40 -25)
📝 docs/product-catalog.json (+2 -2)
📝 docs/verification/managed-conformance-manifest.json (+2 -2)
➕ scripts/lib/real-hopper-inventory.mjs (+47 -0)
📝 scripts/verify-real-hopper.mjs (+3 -0)
📝 skills/reverse-engineer-anything/SKILL.md (+1 -1)
📝 src/domain/callPath.test.ts (+20 -0)
📝 src/domain/callPathSchemas.ts (+12 -4)
📝 src/generatedMcpToolCatalog.ts (+1 -1)
📝 src/native/NativeMacOSProvider.ts (+16 -5)
📝 src/native/NativeMachoInspection.ts (+13 -1)
📝 src/native/parsers/dyldInfo.ts (+61 -22)
📝 src/native/parsers/otool.ts (+9 -5)
📝 tests/boundary/mcp/investigationAnalysisMcp.test.ts (+42 -10)
📝 tests/boundary/providers/hopper/hopperBridgeFacade.test.ts (+65 -0)
➕ tests/boundary/providers/native/nativeInventory.golden.test.ts (+117 -0)
📝 tests/boundary/providers/native/nativeProvider.test.ts (+43 -79)
📝 tests/fixtures/hopperBridgeFacadeProbe.py (+91 -1)
📝 tests/fixtures/native-macos/dyld-imports.txt (+4 -3)
➕ tests/fixtures/native-macos/dyld-inventory/README.md (+43 -0)

...and 14 more files

📄 Description

Summary

Fixes #501 and additional address, format and error-handling boundary bugs found while auditing the same failure class.

Hopper inventories: _search_inventory already returns normalized hexadecimal addresses in numeric order. Preserve those keys and that ordering in list_strings and list_names instead of formatting/sorting them again. Release 3.2.1 reproduces both reported -32000 invalid_request failures; main had only partially repaired strings, with names still failing and string addresses sorted lexicographically.
Native Mach-O inspection: otool segment/section addresses above JavaScript's safe integer range were returned as null. Normalize address text using BigInt, retaining exact 64-bit offsets.
Native section alignment: real otool prints 2^N (bytes), which the numeric-only parser discarded. Decode that notation while preserving support for the existing numeric exponent form.
Call-path reconstruction: valid Ghidra dossiers containing qualified addresses such as EXTERNAL:0x1 were rejected by a plain-hex-only schema. Accept encoded address-space prefixes and normalize only offsets. Keep equal offsets in distinct spaces separate; regenerate canonical MCP/product metadata.

Further boundary audit

Distinguish explicit Hopper request-validation failures from internal provider TypeError, ValueError and KeyError faults. Internal faults now remain bridge_exception; absence of a loaded document is capability unavailability.
Validate JSON request objects, method strings and parameter objects before dispatch. Invalid Unicode tokens produce authorization failures instead of incidental Python comparison errors.
Parse actual Apple dyld_info imports and annotated weak/absolute/reexport rows. Import binding ordinals are not reported as addresses. Project export offsets using the observed file-backed image base; keep absolute values unchanged and report unknown addresses when no base was observed.
Preserve richer dyld export facts when merging names from nm, which previously replaced addresses and flags with null.
Reject failed codesign inspections instead of returning signed: true. Preserve successful unsigned observations and propagate failures from signed-artifact requirement/entitlement queries.

Regression coverage

Extend the existing Python bridge/socket integration with the actual cache and dispatch handlers, using a Hopper API fixture. Require complete inventories, numeric ordering across address widths, exact filters and empty results for absent addresses. This offline boundary test does not prove a live Hopper engine claim.
Extend the real-Hopper stdio MCP verifier with full inventories, canonical addresses, numeric ordering and filtered lookups on its existing session.
Add a captured real Apple otool golden and a reproducible local Mach-O generation recipe; verify normalized addresses and alignment through the production native provider.
Strengthen an existing SDK-connected MCP call-path test with an external callee, successful ordinary zero-hop reconstruction and a cited path to the qualified address. Cover distinct spaces, encoded names and malformed encodings in domain tests.
Extend actual bridge/socket coverage for malformed JSON/envelope types, boolean parameter confusion, non-ASCII tokens, provider faults and successful requests after rejection.
Replay a complete captured host-native inventory through the production provider, plus separate real library row samples and signed-inspection failure/unsigned stderr captures. Replace the former assumed import fixture with captured Apple output and share the existing fixture runner.
Failures were reproduced before their fixes; the focused regressions pass afterward.

Verification

npm run check:pr: 352 files passed; 1,798 tests passed, two skipped. Typecheck, lint, formatting, dead-code and generated-document checks passed (70 existing lint warnings, zero errors).
Focused bridge/socket, native golden and native-provider regressions: 17 tests passed; the full gate also covers the call-path domain and SDK MCP regressions.
Real Apple otool output was passed directly through the compiled parser. The compiled production native provider was also checked with unmocked Apple tools on a generated host-native fixture: imports, export virtual addresses, weak/absolute facts, and signed/unsigned/failure distinctions passed.
Generated catalog comparison confirms only build_call_path changed; auxiliary provider descriptors are unchanged.
Real Hopper/Ghidra engine verification was not run. No Hopper windows were opened. The updated Hopper lane is available as npm run verify:hopper.

Audit limits

No further double conversion bugs were confirmed in reviewed Hopper cache consumers, numeric reference sorting or Ghidra unsigned-address serialization. The misleading Hopper error classification is now fixed. The reviewed TypeScript transport and Ghidra integer/boolean validators showed no additional confirmed coercion failures.

No binaries, provider project documents or credentials are committed.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/503 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/4/2026 **Status:** ✅ Merged **Merged:** 10/4/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `codex/fix-hopper-inventory-addresses` --- ### 📝 Commits (6) - [`c9ea330`](https://github.com/morluto/rea/commit/c9ea330afefe26c09a08e646387d3e53d938e876) fix(hopper): preserve normalized inventory addresses - [`064b1c1`](https://github.com/morluto/rea/commit/064b1c12c59e754155971e845697c8f49954b18c) test(hopper): isolate real inventory verification - [`24a3165`](https://github.com/morluto/rea/commit/24a31657dfee17bac2b22a69e359bafad5ace3c5) fix(native): preserve otool addresses and section alignment - [`e60ef19`](https://github.com/morluto/rea/commit/e60ef19b7a08ea96f595ee6509a675cb6f764df7) fix: preserve address spaces in call path evidence - [`a56b2ef`](https://github.com/morluto/rea/commit/a56b2ef237ae4a70b5ff8032b94cf3defaa47544) fix(hopper): distinguish request validation from provider faults - [`3075fdf`](https://github.com/morluto/rea/commit/3075fdf7f7e0b25b46e69ff191db0ef48624208b) fix(native): retain tool inventory facts and reject failed signatures ### 📊 Changes **34 files changed** (+1009 additions, -161 deletions) <details> <summary>View changed files</summary> 📝 `bridge/hopper_bridge.py` (+40 -25) 📝 `docs/product-catalog.json` (+2 -2) 📝 `docs/verification/managed-conformance-manifest.json` (+2 -2) ➕ `scripts/lib/real-hopper-inventory.mjs` (+47 -0) 📝 `scripts/verify-real-hopper.mjs` (+3 -0) 📝 `skills/reverse-engineer-anything/SKILL.md` (+1 -1) 📝 `src/domain/callPath.test.ts` (+20 -0) 📝 `src/domain/callPathSchemas.ts` (+12 -4) 📝 `src/generatedMcpToolCatalog.ts` (+1 -1) 📝 `src/native/NativeMacOSProvider.ts` (+16 -5) 📝 `src/native/NativeMachoInspection.ts` (+13 -1) 📝 `src/native/parsers/dyldInfo.ts` (+61 -22) 📝 `src/native/parsers/otool.ts` (+9 -5) 📝 `tests/boundary/mcp/investigationAnalysisMcp.test.ts` (+42 -10) 📝 `tests/boundary/providers/hopper/hopperBridgeFacade.test.ts` (+65 -0) ➕ `tests/boundary/providers/native/nativeInventory.golden.test.ts` (+117 -0) 📝 `tests/boundary/providers/native/nativeProvider.test.ts` (+43 -79) 📝 `tests/fixtures/hopperBridgeFacadeProbe.py` (+91 -1) 📝 `tests/fixtures/native-macos/dyld-imports.txt` (+4 -3) ➕ `tests/fixtures/native-macos/dyld-inventory/README.md` (+43 -0) _...and 14 more files_ </details> ### 📄 Description ## Summary Fixes #501 and additional address, format and error-handling boundary bugs found while auditing the same failure class. - Hopper inventories: `_search_inventory` already returns normalized hexadecimal addresses in numeric order. Preserve those keys and that ordering in `list_strings` and `list_names` instead of formatting/sorting them again. Release 3.2.1 reproduces both reported `-32000 invalid_request` failures; main had only partially repaired strings, with names still failing and string addresses sorted lexicographically. - Native Mach-O inspection: `otool` segment/section addresses above JavaScript's safe integer range were returned as `null`. Normalize address text using `BigInt`, retaining exact 64-bit offsets. - Native section alignment: real `otool` prints `2^N (bytes)`, which the numeric-only parser discarded. Decode that notation while preserving support for the existing numeric exponent form. - Call-path reconstruction: valid Ghidra dossiers containing qualified addresses such as `EXTERNAL:0x1` were rejected by a plain-hex-only schema. Accept encoded address-space prefixes and normalize only offsets. Keep equal offsets in distinct spaces separate; regenerate canonical MCP/product metadata. ## Further boundary audit - Distinguish explicit Hopper request-validation failures from internal provider `TypeError`, `ValueError` and `KeyError` faults. Internal faults now remain `bridge_exception`; absence of a loaded document is capability unavailability. - Validate JSON request objects, method strings and parameter objects before dispatch. Invalid Unicode tokens produce authorization failures instead of incidental Python comparison errors. - Parse actual Apple `dyld_info` imports and annotated weak/absolute/reexport rows. Import binding ordinals are not reported as addresses. Project export offsets using the observed file-backed image base; keep absolute values unchanged and report unknown addresses when no base was observed. - Preserve richer dyld export facts when merging names from `nm`, which previously replaced addresses and flags with `null`. - Reject failed `codesign` inspections instead of returning `signed: true`. Preserve successful unsigned observations and propagate failures from signed-artifact requirement/entitlement queries. ## Regression coverage - Extend the existing Python bridge/socket integration with the actual cache and dispatch handlers, using a Hopper API fixture. Require complete inventories, numeric ordering across address widths, exact filters and empty results for absent addresses. This offline boundary test does not prove a live Hopper engine claim. - Extend the real-Hopper stdio MCP verifier with full inventories, canonical addresses, numeric ordering and filtered lookups on its existing session. - Add a captured real Apple `otool` golden and a reproducible local Mach-O generation recipe; verify normalized addresses and alignment through the production native provider. - Strengthen an existing SDK-connected MCP call-path test with an external callee, successful ordinary zero-hop reconstruction and a cited path to the qualified address. Cover distinct spaces, encoded names and malformed encodings in domain tests. - Extend actual bridge/socket coverage for malformed JSON/envelope types, boolean parameter confusion, non-ASCII tokens, provider faults and successful requests after rejection. - Replay a complete captured host-native inventory through the production provider, plus separate real library row samples and signed-inspection failure/unsigned stderr captures. Replace the former assumed import fixture with captured Apple output and share the existing fixture runner. - Failures were reproduced before their fixes; the focused regressions pass afterward. ## Verification - `npm run check:pr`: 352 files passed; 1,798 tests passed, two skipped. Typecheck, lint, formatting, dead-code and generated-document checks passed (70 existing lint warnings, zero errors). - Focused bridge/socket, native golden and native-provider regressions: 17 tests passed; the full gate also covers the call-path domain and SDK MCP regressions. - Real Apple `otool` output was passed directly through the compiled parser. The compiled production native provider was also checked with unmocked Apple tools on a generated host-native fixture: imports, export virtual addresses, weak/absolute facts, and signed/unsigned/failure distinctions passed. - Generated catalog comparison confirms only `build_call_path` changed; auxiliary provider descriptors are unchanged. - Real Hopper/Ghidra engine verification was **not run**. No Hopper windows were opened. The updated Hopper lane is available as `npm run verify:hopper`. ## Audit limits No further double conversion bugs were confirmed in reviewed Hopper cache consumers, numeric reference sorting or Ghidra unsigned-address serialization. The misleading Hopper error classification is now fixed. The reviewed TypeScript transport and Ghidra integer/boolean validators showed no additional confirmed coercion failures. No binaries, provider project documents or credentials are committed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-16 21:13:32 +00:00