morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #8193] [Design] Analyze an explicitly selected universal Mach-O slice with Kuna #6477

New issue

Open

opened 2026-11-01 21:11:27 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Owner

Originally created by @morluto on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/8193

Before submitting

I searched the open and closed issues and pull requests for a related discussion. #7381 covers the Ghidra universal-image boundary; this asks about the distinct Kuna path and cross-provider selection/provenance.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This discussion does not include credentials, private samples, or sensitive target data.

Decision needed

Should REA allow an explicit selected architecture of a universal Mach-O to be analyzed with Kuna, while keeping the original container, selected slice, coordinates and derived artifacts unambiguous?

Context and evidence

On macOS arm64, I built an unexecuted x86-64/arm64 universal Mach-O from the same small owned C fixture (container SHA-256 705614f3df4c590fee5e0bbe7639e7108886f4d47138d20ff63ec59829019538).

Direct Kuna 1.986 decompile ... _adjust --slice arm64 --json returned the arm64 function at 0x100000410; selecting x86_64 returned the x86-64 function at 0x100000420. Omitting --slice returned the x86-64 result on this container, even though the host is arm64. This is one container ordering, not a claim about all default choices.
REA MCP open_binary with provider_id: "kuna" refused the universal image with a typed unsupported-target error and advised opening a supported extracted binary. A failed explicit open left no target in that new session, so list_architectures and inspect_macho then reported no app open.
Opening the same input without explicitly selecting a provider allowed REA's list_architectures to report both slices. With Hopper configured, auto selection used its verified private thin-image workflow and its Evidence described the original container. In a separate session with Hopper disabled, metadata inspection still worked, while deep procedure listing was unavailable.
I manually used lipo -thin arm64 to create a separate file. Its bytes matched the range [16384, 49952) of the original container. REA/Kuna opened that thin file and listed the function, but its Evidence subject was the thin file and its SHA-256 (20b1bb58e22badbb817bd8939ad3e8a84b1a61f181f3a86dce4456d6c2245d36); the manual parent-to-slice relationship was not part of that Evidence.

The current rejection matches REA's documented thin-only Kuna boundary; it is not a claim that universal input was promised. It does show why simply forwarding Kuna's default would be unsafe for a caller who selected a different architecture.

Expected interaction with targets

Read a caller-selected local container and launch the existing static provider; an owned temporary slice may be written. Neither original target nor target code is executed or modified.

Boundary details (optional)

Require or otherwise make the selected architecture unambiguous. Validate slice identity, offsets, container bounds and target changes; preserve original and derived digest, selected architecture, coordinate mapping, and cleanup ownership. Do not assume slice addresses, file offsets, or the provider default are interchangeable. The existing Hopper verified-image path is useful design evidence, not proof that its implementation can be reused unchanged for Kuna.

Options and tradeoffs

Compare a shared, verified owned-slice preparation workflow against passing an explicit --slice with independently checked identity and output semantics. Consider an agent's recovery after an explicit provider refusal and whether a provider-neutral selection operation could be shared with #7381. Do not expose upstream flags without preserving evidence and lifecycle behavior.

Verification available so far

I have a prototype or observed results.

Success criteria and open questions

Test both requested slices, ambiguous/default selection, malformed and duplicated FAT entries, identity changes, different file offsets, cache separation, CLI/MCP parity, cancellation and cleanup. Preserve the selected user input and verify cold and project-export paths; do not infer that a universal container's default architecture equals the host or that the slice digest is the container digest.

References

Related: #7381 (Ghidra), #8173 (function matching across distinct coordinate contexts), #8168 (Kuna inventory classifications). REA source d48e472fa plus the 1.986 baseline update in #8187.

Universal Mach-O byte provenance control

On macOS arm64, I built an unexecuted x86-64/arm64 universal Mach-O with _audit_value in both slices. lipo -detailed_info reported x86-64 at container offset 0x1000 and arm64 at 0x4000. Direct released Kuna 1.988 read --slice returned the expected 16 bytes for each architecture. An unverified debug source build at Kuna merge commit b8eae8b660ceb0dc2a9162d0df2e63272c9280ed, using locally supplied 1.988 specs, returned the same bytes and file provenance offsets 0x1340 for x86-64 and 0x4328 for arm64. Those offsets locate the bytes in the original universal container.

After explicit lipo -thin extraction, the same source build reported slice-file offsets 0x340 and 0x328, respectively. The original container bytes at the reported container offsets and the extracted files at their reported slice offsets matched independently. Real REA/Ghidra 12.1.4 through compact and full stdio MCP returned the same bytes and slice-file offsets for both extracted binaries; successful read_bytes and address_to_file_offset results validated against their advertised schemas. REA explicitly refused the universal file for both Ghidra and verified Kuna 1.988 with provider-specific unsupported-target reasons.

The original universal SHA-256 was d5374df0bffb2d6c05a30d431a37b70cc4877f16bad1c11638d748797c0252ef; extracted arm64 and x86-64 hashes were 967e8358527e0fd4dea88282e6cd4b9cf56604c9f786f5c1d75ed1ba642d991b and b33bfb04782a7b1570eb9d54cf28226dd67e2d4223608d23b979d299c89065f7. All remained unchanged and were not run. This is one constructed container and ordinary file-backed reads, not a claim about all offsets, fixups, malformed containers or released provenance support. It reinforces that an adapter must name the coordinate owner and preserve the selected slice and container identity; it must not add a slice base twice or treat an extracted file's offset as a container offset.

Recovery after refusal with an existing target

On #8608 head 331f2dd82, macOS arm64, Ghidra 12.1.4/JDK 21 and Kuna 1.988, I opened the extracted arm64 file with Ghidra, read 16 bytes at 0x100000328, then attempted to open the universal container explicitly with Ghidra. Both compact and full stdio MCP returned capability_unavailable / target_unsupported; binary_session(detail: complete) still showed an open Ghidra binding, and repeating the read succeeded with the previous thin file's subject SHA-256 967e8358527e0fd4dea88282e6cd4b9cf56604c9f786f5c1d75ed1ba642d991b. A failed switch therefore preserves the previous target in this control.

Explicitly reopening the universal container without a provider then succeeded; list_architectures reported both slices and their container offsets. All successful results were validated against the fetched MCP output schemas; source hashes remained unchanged and no target was executed. This is one recovery sequence, not a general lifecycle guarantee. It adds an agent-UX concern to #8402: after a failed switch, a follow-up query without checking its subject may still answer about the old artifact. Recovery guidance should make the preserved selection and explicit reopen/selection step understandable while retaining the useful previous session.

Originally created by @morluto on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/8193 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related discussion. #7381 covers the Ghidra universal-image boundary; this asks about the distinct Kuna path and cross-provider selection/provenance. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This discussion does not include credentials, private samples, or sensitive target data. ### Decision needed Should REA allow an explicit selected architecture of a universal Mach-O to be analyzed with Kuna, while keeping the original container, selected slice, coordinates and derived artifacts unambiguous? ### Context and evidence On macOS arm64, I built an unexecuted x86-64/arm64 universal Mach-O from the same small owned C fixture (container SHA-256 `705614f3df4c590fee5e0bbe7639e7108886f4d47138d20ff63ec59829019538`). - Direct Kuna 1.986 `decompile ... _adjust --slice arm64 --json` returned the arm64 function at `0x100000410`; selecting `x86_64` returned the x86-64 function at `0x100000420`. Omitting `--slice` returned the x86-64 result on this container, even though the host is arm64. This is one container ordering, not a claim about all default choices. - REA MCP `open_binary` with `provider_id: "kuna"` refused the universal image with a typed unsupported-target error and advised opening a supported extracted binary. A failed explicit open left no target in that new session, so `list_architectures` and `inspect_macho` then reported no app open. - Opening the same input without explicitly selecting a provider allowed REA's `list_architectures` to report both slices. With Hopper configured, auto selection used its verified private thin-image workflow and its Evidence described the original container. In a separate session with Hopper disabled, metadata inspection still worked, while deep procedure listing was unavailable. - I manually used `lipo -thin arm64` to create a separate file. Its bytes matched the range `[16384, 49952)` of the original container. REA/Kuna opened that thin file and listed the function, but its Evidence subject was the thin file and its SHA-256 (`20b1bb58e22badbb817bd8939ad3e8a84b1a61f181f3a86dce4456d6c2245d36`); the manual parent-to-slice relationship was not part of that Evidence. The current rejection matches REA's documented thin-only Kuna boundary; it is not a claim that universal input was promised. It does show why simply forwarding Kuna's default would be unsafe for a caller who selected a different architecture. ### Expected interaction with targets Read a caller-selected local container and launch the existing static provider; an owned temporary slice may be written. Neither original target nor target code is executed or modified. ### Boundary details (optional) Require or otherwise make the selected architecture unambiguous. Validate slice identity, offsets, container bounds and target changes; preserve original and derived digest, selected architecture, coordinate mapping, and cleanup ownership. Do not assume slice addresses, file offsets, or the provider default are interchangeable. The existing Hopper verified-image path is useful design evidence, not proof that its implementation can be reused unchanged for Kuna. ### Options and tradeoffs Compare a shared, verified owned-slice preparation workflow against passing an explicit `--slice` with independently checked identity and output semantics. Consider an agent's recovery after an explicit provider refusal and whether a provider-neutral selection operation could be shared with #7381. Do not expose upstream flags without preserving evidence and lifecycle behavior. ### Verification available so far I have a prototype or observed results. ### Success criteria and open questions Test both requested slices, ambiguous/default selection, malformed and duplicated FAT entries, identity changes, different file offsets, cache separation, CLI/MCP parity, cancellation and cleanup. Preserve the selected user input and verify cold and project-export paths; do not infer that a universal container's default architecture equals the host or that the slice digest is the container digest. ### References Related: #7381 (Ghidra), #8173 (function matching across distinct coordinate contexts), #8168 (Kuna inventory classifications). REA source `d48e472fa` plus the 1.986 baseline update in #8187. ### Universal Mach-O byte provenance control On macOS arm64, I built an unexecuted x86-64/arm64 universal Mach-O with `_audit_value` in both slices. `lipo -detailed_info` reported x86-64 at container offset `0x1000` and arm64 at `0x4000`. Direct released Kuna 1.988 `read --slice` returned the expected 16 bytes for each architecture. An unverified debug source build at Kuna merge commit `b8eae8b660ceb0dc2a9162d0df2e63272c9280ed`, using locally supplied 1.988 specs, returned the same bytes and `file` provenance offsets `0x1340` for x86-64 and `0x4328` for arm64. Those offsets locate the bytes in the original universal container. After explicit `lipo -thin` extraction, the same source build reported slice-file offsets `0x340` and `0x328`, respectively. The original container bytes at the reported container offsets and the extracted files at their reported slice offsets matched independently. Real REA/Ghidra 12.1.4 through compact and full stdio MCP returned the same bytes and slice-file offsets for both extracted binaries; successful `read_bytes` and `address_to_file_offset` results validated against their advertised schemas. REA explicitly refused the universal file for both Ghidra and verified Kuna 1.988 with provider-specific unsupported-target reasons. The original universal SHA-256 was `d5374df0bffb2d6c05a30d431a37b70cc4877f16bad1c11638d748797c0252ef`; extracted arm64 and x86-64 hashes were `967e8358527e0fd4dea88282e6cd4b9cf56604c9f786f5c1d75ed1ba642d991b` and `b33bfb04782a7b1570eb9d54cf28226dd67e2d4223608d23b979d299c89065f7`. All remained unchanged and were not run. This is one constructed container and ordinary file-backed reads, not a claim about all offsets, fixups, malformed containers or released provenance support. It reinforces that an adapter must name the coordinate owner and preserve the selected slice and container identity; it must not add a slice base twice or treat an extracted file's offset as a container offset. ### Recovery after refusal with an existing target On #8608 head `331f2dd82`, macOS arm64, Ghidra 12.1.4/JDK 21 and Kuna 1.988, I opened the extracted arm64 file with Ghidra, read 16 bytes at `0x100000328`, then attempted to open the universal container explicitly with Ghidra. Both compact and full stdio MCP returned `capability_unavailable` / `target_unsupported`; `binary_session(detail: complete)` still showed an open Ghidra binding, and repeating the read succeeded with the *previous thin file's* subject SHA-256 `967e8358527e0fd4dea88282e6cd4b9cf56604c9f786f5c1d75ed1ba642d991b`. A failed switch therefore preserves the previous target in this control. Explicitly reopening the universal container without a provider then succeeded; `list_architectures` reported both slices and their container offsets. All successful results were validated against the fetched MCP output schemas; source hashes remained unchanged and no target was executed. This is one recovery sequence, not a general lifecycle guarantee. It adds an agent-UX concern to #8402: after a failed switch, a follow-up query without checking its subject may still answer about the old artifact. Recovery guidance should make the preserved selection and explicit reopen/selection step understandable while retaining the useful previous session.

bsakaguchi added the

enhancement

triage

labels

2026-11-01 21:11:27 +00:00

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

@morluto commented on GitHub (Oct 31, 2026): Looping in @gensokyo-dev @kaoru0822-kitauji @N0zoM1z0 for visibility on the Ghidra, Kuna, and REA tool-design investigation.

bsakaguchi commented

2026-11-01 21:11:27 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Additional selected-slice controls on main e6d061897, macOS arm64, direct Kuna 1.986 and real REA MCP with Ghidra 12.1.4/Kuna 1.986. This uses the original two-slice, unexecuted Mach-O; independent lipo -detailed_info reported x86-64 at container offset 4096 (size 8568) and arm64 at offset 16384 (size 33568). I verified each full container range byte-for-byte against its corresponding thin file.

For each architecture, direct Kuna functions, individual decompile, outgoing xrefs and disassemble returned equal parsed results for the selected FAT slice and its thin file after excluding the input path. _adjust was 0x100000410 on arm64 and 0x100000420 on x86-64. The returned binary identifies the supplied file; the selection is still caller context.
With selected one-function decompile-project --progress, the function spans matched between FAT and thin output for each architecture, but complete C-file digests differed because their generated include/file names differed. On x86-64 the function's output-file offset was 30 for FAT versus 31 for thin. Output-file spans must therefore remain tied to the actual produced file, even when function text matches.
REA/Ghidra and REA/Kuna both analyzed each explicitly extracted thin input, retaining its own SHA-256 as the Evidence subject. REA/Ghidra mapped _adjust to thin offsets 1040 (arm64) and 1056 (x86-64); adding the independently verified slice starts produced container offsets 17424 and 5152, whose bytes matched Kuna's first decoded instruction. REA/Kuna reported address-to-file-offset capability unavailable; it did not establish that mapping itself.
In a shared isolated REA/Kuna cache, completed project-export Evidence used different keys for the two thin-file identities. Fresh servers reopening each thin file reported the corresponding same key. The Kuna analysis-profile digest was equal across these runs, so it is not a substitute for target/slice identity. This check records matching keys, not a process-level measurement of whether each cache was rebuilt.

The container and thin-file hashes were unchanged. These are two slices of one fixture, not proof of general FAT loader equivalence or cross-architecture behavioral equivalence. They support preserving both container and slice identity plus coordinate context; they do not make project-output offsets interchangeable or justify forwarding an omitted --slice default.

@morluto commented on GitHub (Oct 31, 2026): Additional selected-slice controls on main `e6d061897`, macOS arm64, direct Kuna 1.986 and real REA MCP with Ghidra 12.1.4/Kuna 1.986. This uses the original two-slice, unexecuted Mach-O; independent `lipo -detailed_info` reported x86-64 at container offset 4096 (size 8568) and arm64 at offset 16384 (size 33568). I verified each full container range byte-for-byte against its corresponding thin file. * For each architecture, direct Kuna `functions`, individual `decompile`, outgoing `xrefs` and `disassemble` returned equal parsed results for the selected FAT slice and its thin file after excluding the input path. `_adjust` was `0x100000410` on arm64 and `0x100000420` on x86-64. The returned `binary` identifies the supplied file; the selection is still caller context. * With selected one-function `decompile-project --progress`, the function spans matched between FAT and thin output for each architecture, but complete C-file digests differed because their generated include/file names differed. On x86-64 the function's output-file offset was 30 for FAT versus 31 for thin. Output-file spans must therefore remain tied to the actual produced file, even when function text matches. * REA/Ghidra and REA/Kuna both analyzed each explicitly extracted thin input, retaining its own SHA-256 as the Evidence subject. REA/Ghidra mapped `_adjust` to thin offsets 1040 (arm64) and 1056 (x86-64); adding the independently verified slice starts produced container offsets 17424 and 5152, whose bytes matched Kuna's first decoded instruction. REA/Kuna reported address-to-file-offset capability unavailable; it did not establish that mapping itself. * In a shared isolated REA/Kuna cache, completed project-export Evidence used different keys for the two thin-file identities. Fresh servers reopening each thin file reported the corresponding same key. The Kuna analysis-profile digest was equal across these runs, so it is not a substitute for target/slice identity. This check records matching keys, not a process-level measurement of whether each cache was rebuilt. The container and thin-file hashes were unchanged. These are two slices of one fixture, not proof of general FAT loader equivalence or cross-architecture behavioral equivalence. They support preserving both container and slice identity plus coordinate context; they do not make project-output offsets interchangeable or justify forwarding an omitted `--slice` default.

bsakaguchi commented

2026-11-01 21:11:28 +00:00

Author

Owner

@morluto commented on GitHub (Oct 31, 2026):

Shipped-artifact control on current REA a989c0e: Xcode 26.5's universal libSwiftToolsSupport.dylib was refused by explicit open_binary(provider_id: "kuna") with the documented target_format_unsupported reason. Direct Kuna 1.986 on that original file, explicitly using --slice arm64, successfully decompiled the selected address 0x6d868.

I created a separate caller-owned arm64 file with lipo -thin arm64; its bytes exactly matched the original slice at container offset 1,228,800 with length 1,190,496. REA/Kuna opened that derivative and decompiled the same address, but its Evidence subject and SHA-256 identified the derivative, not the original universal container. Original SHA-256 a0c202bf8ea127eb1d0c23f305c9f926e54cd29399d095c71bb37a265ad4144c; derivative SHA-256 0898d938a516e7168c2a19b2bce36fda6e0b440dadcf3c436babf52b486a1875. Both hashes stayed unchanged during analysis; neither target was executed.

The chosen export name was also absent from Kuna's procedure inventory despite being present in the source's export trie; that independent name handoff is #8328 / Kuna #1349. This one Apple-shipped library supports the importance of preserving container/slice identity; it does not establish that forwarding --slice is sufficient for REA's cached project and evidence contracts.

@morluto commented on GitHub (Oct 31, 2026): Shipped-artifact control on current REA `a989c0e`: Xcode 26.5's universal `libSwiftToolsSupport.dylib` was refused by explicit `open_binary(provider_id: "kuna")` with the documented `target_format_unsupported` reason. Direct Kuna 1.986 on that **original** file, explicitly using `--slice arm64`, successfully decompiled the selected address `0x6d868`. I created a separate caller-owned arm64 file with `lipo -thin arm64`; its bytes exactly matched the original slice at container offset 1,228,800 with length 1,190,496. REA/Kuna opened that derivative and decompiled the same address, but its Evidence subject and SHA-256 identified the derivative, not the original universal container. Original SHA-256 `a0c202bf8ea127eb1d0c23f305c9f926e54cd29399d095c71bb37a265ad4144c`; derivative SHA-256 `0898d938a516e7168c2a19b2bce36fda6e0b440dadcf3c436babf52b486a1875`. Both hashes stayed unchanged during analysis; neither target was executed. The chosen export name was also absent from Kuna's procedure inventory despite being present in the source's export trie; that independent name handoff is #8328 / [Kuna #1349](https://github.com/Noelo-Lab/kuna/issues/1349). This one Apple-shipped library supports the importance of preserving container/slice identity; it does not establish that forwarding `--slice` is sufficient for REA's cached project and evidence contracts.

bsakaguchi referenced this issue

2026-11-01 21:47:19 +00:00

[PR #6477] feat(mcp): add opt-in release notifications #7362

bsakaguchi referenced this issue

2026-11-03 00:29:36 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425

bsakaguchi added

enhancement

and removed

enhancement

labels

2026-11-03 00:29:41 +00:00