[PR #6257] [MERGED] feat(native): decode Swift nominal type declarations from __swift5_types #7231

New issue

Closed

opened 2026-11-01 21:46:21 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:46:21 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6257
Author: @kaoru0822-kitauji
Created: 10/22/2026
Status: ✅ Merged
Merged: 10/22/2026
Merged by: @kaoru0822-kitauji

Base: main ← Head: feat/5341-swift-nominal-decls

📝 Commits (2)

4e08dc0 feat(native): decode Swift nominal type declarations from __swift5_types
3f6179a fix(native): bound Swift context walks and decode vtables in record order

📊 Changes

15 files changed (+1087 additions, -110 deletions)

View changed files

📝 docs/native-investigation.md (+17 -3)
📝 scripts/verify/apple/dispatch.mjs (+73 -0)
📝 src/domain/native/objcSwiftMetadata.ts (+16 -28)
📝 src/native/AppleDispatchCoverage.ts (+14 -1)
📝 src/native/AppleDispatchDecodeFacts.ts (+1 -0)
📝 src/native/AppleDispatchMetadata.test.ts (+5 -1)
📝 src/native/AppleDispatchMetadata.ts (+27 -11)
📝 src/native/AppleObjcProtocols.ts (+2 -2)
📝 src/native/AppleSwiftDispatchFacets.ts (+6 -2)
➕ src/native/AppleSwiftNominalTypes.test.ts (+482 -0)
➕ src/native/AppleSwiftNominalTypes.ts (+342 -0)
📝 src/native/AppleSwiftRelativePointer.test.ts (+1 -1)
📝 src/native/AppleSwiftVtables.test.ts (+49 -10)
📝 src/native/AppleSwiftVtables.ts (+40 -51)
📝 tests/conformance/native/dispatch.swift (+12 -0)

📄 Description

What broke: swift_decls in inspect_native_dispatch_metadata was always [], so a stripped Swift binary showed no type list even though __swift5_types survives stripping.
Fix: one shared nominal-descriptor reader fills swift_decls (kind, name, Outer.Inner, module, accessor, descriptor location) and feeds the existing class-vtable decoder.
Risk: low. Read-only static parsing; the swift_decls item schema changes (never-populated fields removed); full output grows ~20–25% on Swift apps because of the new records.

Summary

Fixes #5341. The Mach-O byte reader behind inspect_native_dispatch_metadata / rea inspect-native-dispatch-metadata now decodes __swift5_types nominal type context descriptors into swift_decls. Before this change nothing produced that field. The only other Swift type path was class vtables, which skipped structs and enums.

Problem and expected behavior

When you open a stripped Swift app, you want to know which modules and types it contains and where their accessors are. You want that before reading any code. Each swift_decls entry now carries:

kind: class, struct, enum, or actor (taken from the class IsActor descriptor flag)
name (the descriptor's own name), qualified_name (Outer.Inner) and module (found by walking parent contexts)
generic, plus anonymous_context for private/local types whose anonymous context is left out of qualified_name
accessor_address (the metadata access function), the descriptor location, decode, and evidence citing the __swift5_types record

If a parent context can't be resolved, qualified_name stays null rather than holding a shortened guess, and the entry is partial with a reason. Examples are an extension of a type named only by a mangled name, a missing module, or a cyclic or over-deep chain. Coverage uses the existing swift_declarations facet name, which the symbol-only path already uses.

Change and scope

New src/native/AppleSwiftNominalTypes.ts. It walks each context once (memoized), so a shared module or outer type adds one relative-pointer record no matter how many types it contains. Depth is capped at 32 contexts.
decodeSwiftClassVtables now uses the reader's descriptors instead of re-reading records and names, so the two paths can't drift (the issue's "Possible direction"). If a type record can't be read, swift_declarations and swift_class_vtable_descriptors both stay partial, because that record might have been a class.
Cleanup in touched code:
- removed the unused classDescriptorName relative-field alias (it duplicated typeName)
- removed a failures string array in the vtable decoder that was never read
- AppleObjcProtocols now takes its reader type from the shared reader
Not included (non-goals in the issue): field descriptors, demangling superclass, extension or conformance names, and access level.

Contract and boundary impact

Semantic owner and earliest changed stage: Apple Mach-O metadata reader (src/native). The domain schema swiftDeclSchema changes.
Contribution scope fit and linked maintainer decision: static parsing of the target file only. The issue's direction was endorsed in the thread, and the plan was posted on #5341.
CLI and MCP/tool-catalog contract: same tool and inputs. swift_decls items change from {kind, name, module, access_level=internal, super_class, protocols=[], is_final…} to the descriptor-backed fields above. The old fields were never populated (the array was always empty), and defaults like access_level: "internal" would have been made up. There is no data migration. I didn't mark this ! because no previously produced value changes, but maintainers may judge the output-schema change differently.
Provider/platform compatibility: Mach-O 64-bit LE byte reader only. The symbol-only provider path is unchanged.
Evidence/provenance: every declaration cites its type record with artifact path and SHA-256. Relative pointers read during the walk are kept in relative_pointers, the same as for the other Swift readers.
Generated metadata: none committed.
Read existing files or captured data.

Evidence and regression coverage

Tests added or updated:
- src/native/AppleSwiftNominalTypes.test.ts runs the production decodeAppleDispatchMetadata on a synthetic Mach-O. It covers:
  - module, nested struct/enum, actor flag, generic type with a null accessor, and an anonymous (private) context
  - an extension context, an orphan with no parent, an unsupported descriptor kind, and a parent cycle
  - each shared context read once, the record-budget prefix, an indirect parent pointer, and an unreadable record keeping vtable coverage partial
- Vtable tests were updated to the shared-descriptor input.
Real toolchain: verify:apple-dispatch now builds dispatch.swift with an extra enum, nested struct, generic struct, and actor, in both -no_fixup_chains and chained-fixup link modes. For each build it checks:
- the expected declarations, all decoded, with swift_declarations complete
- the declaration set is identical before and after strip -x
- every descriptor and accessor address equals a compiler …Mn / …Ma symbol from nm

User-visible output (mpv, filtered to swift_decls):

{"kind":"class","name":"Application","qualified_name":"Application","module":"swift","generic":false,"anonymous_context":false,"accessor_address":"0x100014428","location":{"address":"0x100383910","file_offset":3684624},"decode":{"status":"decoded","reason":null},"evidence":[{"kind":"binary_metadata","description":"Swift __swift5_types record of the nominal type descriptor at location","location":{"address":"0x10038b730","file_offset":3716912},…}]}

Real-app before/after (`rea inspect-native-dispatch-metadata <bin> --max-records 20000 --format json`, origin/main `c537c52c9` vs this branch, 3 warm runs each, macOS arm64)

App (types)	swift_decls	swift_declarations	vtable facet	wall s (base → new)	peak RSS MB	full output tokens
PureMac (111)	0 → 111	complete 111/111	unchanged 104/31	1.33–1.48 → 1.37–1.53	331–382 → 331–348	304,122 → 380,054
mpv (209)	0 → 209	complete 209/209	unchanged 676/546	1.42–1.49 → 1.43–1.49	341–359 → 346–359	505,759 → 606,327
Ghostty (323)	0 → 323	partial 323/320 (3 types declared in an extension of an external type)	unchanged 1362/308	1.66–1.90 → 1.66–1.86	450–451 → 457–473	not measured (bytes 6.30 MB → 7.02 MB)

Wall time, CPU time and peak RSS are within run-to-run noise. ObjC, conformance and vtable counts are identical to main.
Tokens: the full envelope grows 20–25%. About two thirds of that is the new relative-pointer records (name, parent, accessor), which keep the reader's existing "every decoded relative pointer is recorded" behavior. Requesting only swift_decls (--filter-output normalized_result.result.swift_decls) costs 37,596 tokens for mpv's 209 declarations.
Remaining proof gaps:
- On BetterDisplay (964 types), Raycast (7,097) and Microsoft To Do (94), even --max-records 20000 is used up by ObjC and conformance records before the reader reaches __swift5_types, both on main and here. The facet then correctly reports partial with max_records_reached. Letting callers pick facets so the budget isn't shared is a separate change.
- No arm64e Swift fixture: the verifier builds Swift only for the arm64 link modes, as before.
Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npm run test:local -- src/native/AppleSwiftNominalTypes.test.ts src/native/AppleSwiftVtables.test.ts src/native/AppleDispatchMetadata.test.ts src/native/AppleSwiftRelativePointer.test.ts src/domain/native/objcSwiftMetadata.test.ts src/domain/native/nativeInvestigationGraph.test.ts src/native/AppleSwiftConformanceTypeReference.test.ts src/application/native/EnhancedNativeInvestigation.test.ts src/application/EnhancedTools.objcClasses.test.ts: 9 files, 161 tests passed. The new indirect-parent test was added afterwards; the 3 native files rerun passed, 97 tests.
Review follow-up (3f6179a64) fixes the four Greptile findings: an intrinsic depth bound that works with the cache, a 4096-character qualified_name cap, vtables decoded in record order, a requirement that accessors be in executable mapped code, and name failures forwarded to vtable coverage only for class or unknown-kind records. There are 5 new regression tests, and all 5 fail on 4e08dc0e2. After the follow-up: npm run test:local -- src/native/AppleSwiftNominalTypes.test.ts src/native/AppleSwiftVtables.test.ts src/native/AppleDispatchMetadata.test.ts src/native/AppleSwiftRelativePointer.test.ts passed 110 tests; check:fast, format:check and verify:apple-dispatch passed; the real-app outputs for PureMac, mpv and Ghostty are byte-identical to the first push.
npm run verify:apple-dispatch: ok: true, target_executed: false; swift-{dyld-info,chained}-{symbols,stripped} each have 7 declarations.
npm run check:fast: passed.
npm run format:check: passed.
npm run docs:check: passed.
Real-app runs: see the table above.

Compatibility, safety, and release

Breaking changes or migration steps: see the contract note. The swift_decls item schema changed, but no produced data changes.
Real Hopper/Ghidra, browser, or OS coverage: macOS byte reader only. No provider change.
Package or release metadata impact: none.
Security, privacy, process, or containment review: static reads of the selected file. No target execution; the verifier reports target_executed: false.

Author readiness

Human owner: @kaoru0822-kitauji

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6257 **Author:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Created:** 10/22/2026 **Status:** ✅ Merged **Merged:** 10/22/2026 **Merged by:** [@kaoru0822-kitauji](https://github.com/kaoru0822-kitauji) **Base:** `main` ← **Head:** `feat/5341-swift-nominal-decls` --- ### 📝 Commits (2) - [`4e08dc0`](https://github.com/morluto/rea/commit/4e08dc0e24b5343bdfe2a99d865e96a111175ab7) feat(native): decode Swift nominal type declarations from __swift5_types - [`3f6179a`](https://github.com/morluto/rea/commit/3f6179a64b35a1187b6764bf631ba9730bad6119) fix(native): bound Swift context walks and decode vtables in record order ### 📊 Changes **15 files changed** (+1087 additions, -110 deletions) <details> <summary>View changed files</summary> 📝 `docs/native-investigation.md` (+17 -3) 📝 `scripts/verify/apple/dispatch.mjs` (+73 -0) 📝 `src/domain/native/objcSwiftMetadata.ts` (+16 -28) 📝 `src/native/AppleDispatchCoverage.ts` (+14 -1) 📝 `src/native/AppleDispatchDecodeFacts.ts` (+1 -0) 📝 `src/native/AppleDispatchMetadata.test.ts` (+5 -1) 📝 `src/native/AppleDispatchMetadata.ts` (+27 -11) 📝 `src/native/AppleObjcProtocols.ts` (+2 -2) 📝 `src/native/AppleSwiftDispatchFacets.ts` (+6 -2) ➕ `src/native/AppleSwiftNominalTypes.test.ts` (+482 -0) ➕ `src/native/AppleSwiftNominalTypes.ts` (+342 -0) 📝 `src/native/AppleSwiftRelativePointer.test.ts` (+1 -1) 📝 `src/native/AppleSwiftVtables.test.ts` (+49 -10) 📝 `src/native/AppleSwiftVtables.ts` (+40 -51) 📝 `tests/conformance/native/dispatch.swift` (+12 -0) </details> ### 📄 Description > **What broke:** `swift_decls` in `inspect_native_dispatch_metadata` was always `[]`, so a stripped Swift binary showed no type list even though `__swift5_types` survives stripping. > **Fix:** one shared nominal-descriptor reader fills `swift_decls` (kind, name, `Outer.Inner`, module, accessor, descriptor location) and feeds the existing class-vtable decoder. > **Risk:** low. Read-only static parsing; the `swift_decls` item schema changes (never-populated fields removed); full output grows ~20–25% on Swift apps because of the new records. ## Summary Fixes #5341. The Mach-O byte reader behind `inspect_native_dispatch_metadata` / `rea inspect-native-dispatch-metadata` now decodes `__swift5_types` nominal type context descriptors into `swift_decls`. Before this change nothing produced that field. The only other Swift type path was class vtables, which skipped structs and enums. ## Problem and expected behavior When you open a stripped Swift app, you want to know which modules and types it contains and where their accessors are. You want that before reading any code. Each `swift_decls` entry now carries: - `kind`: `class`, `struct`, `enum`, or `actor` (taken from the class `IsActor` descriptor flag) - `name` (the descriptor's own name), `qualified_name` (`Outer.Inner`) and `module` (found by walking parent contexts) - `generic`, plus `anonymous_context` for private/local types whose anonymous context is left out of `qualified_name` - `accessor_address` (the metadata access function), the descriptor `location`, `decode`, and `evidence` citing the `__swift5_types` record If a parent context can't be resolved, `qualified_name` stays `null` rather than holding a shortened guess, and the entry is `partial` with a reason. Examples are an extension of a type named only by a mangled name, a missing module, or a cyclic or over-deep chain. Coverage uses the existing `swift_declarations` facet name, which the symbol-only path already uses. ## Change and scope - New `src/native/AppleSwiftNominalTypes.ts`. It walks each context once (memoized), so a shared module or outer type adds one relative-pointer record no matter how many types it contains. Depth is capped at 32 contexts. - `decodeSwiftClassVtables` now uses the reader's descriptors instead of re-reading records and names, so the two paths can't drift (the issue's "Possible direction"). If a type record can't be read, `swift_declarations` and `swift_class_vtable_descriptors` both stay partial, because that record might have been a class. - Cleanup in touched code: - removed the unused `classDescriptorName` relative-field alias (it duplicated `typeName`) - removed a `failures` string array in the vtable decoder that was never read - `AppleObjcProtocols` now takes its reader type from the shared reader - Not included (non-goals in the issue): field descriptors, demangling superclass, extension or conformance names, and access level. ## Contract and boundary impact - Semantic owner and earliest changed stage: Apple Mach-O metadata reader (`src/native`). The domain schema `swiftDeclSchema` changes. - Contribution scope fit and linked maintainer decision: static parsing of the target file only. The issue's direction was endorsed in the thread, and the plan was posted on #5341. - CLI and MCP/tool-catalog contract: same tool and inputs. `swift_decls` items change from `{kind, name, module, access_level=internal, super_class, protocols=[], is_final…}` to the descriptor-backed fields above. The old fields were never populated (the array was always empty), and defaults like `access_level: "internal"` would have been made up. There is no data migration. I didn't mark this `!` because no previously produced value changes, but maintainers may judge the output-schema change differently. - Provider/platform compatibility: Mach-O 64-bit LE byte reader only. The symbol-only provider path is unchanged. - Evidence/provenance: every declaration cites its type record with artifact path and SHA-256. Relative pointers read during the walk are kept in `relative_pointers`, the same as for the other Swift readers. - Generated metadata: none committed. - [x] Read existing files or captured data. ## Evidence and regression coverage - Tests added or updated: - `src/native/AppleSwiftNominalTypes.test.ts` runs the production `decodeAppleDispatchMetadata` on a synthetic Mach-O. It covers: - module, nested struct/enum, actor flag, generic type with a null accessor, and an anonymous (private) context - an extension context, an orphan with no parent, an unsupported descriptor kind, and a parent cycle - each shared context read once, the record-budget prefix, an indirect parent pointer, and an unreadable record keeping vtable coverage partial - Vtable tests were updated to the shared-descriptor input. - Real toolchain: `verify:apple-dispatch` now builds `dispatch.swift` with an extra enum, nested struct, generic struct, and actor, in both `-no_fixup_chains` and chained-fixup link modes. For each build it checks: - the expected declarations, all `decoded`, with `swift_declarations` complete - the declaration set is identical before and after `strip -x` - every descriptor and accessor address equals a compiler `…Mn` / `…Ma` symbol from `nm` - User-visible output (mpv, filtered to `swift_decls`): ```json {"kind":"class","name":"Application","qualified_name":"Application","module":"swift","generic":false,"anonymous_context":false,"accessor_address":"0x100014428","location":{"address":"0x100383910","file_offset":3684624},"decode":{"status":"decoded","reason":null},"evidence":[{"kind":"binary_metadata","description":"Swift __swift5_types record of the nominal type descriptor at location","location":{"address":"0x10038b730","file_offset":3716912},…}]} ``` ### Real-app before/after (`rea inspect-native-dispatch-metadata <bin> --max-records 20000 --format json`, origin/main `c537c52c9` vs this branch, 3 warm runs each, macOS arm64) | App (types) | swift_decls | swift_declarations | vtable facet | wall s (base → new) | peak RSS MB | full output tokens | |---|---|---|---|---|---|---| | PureMac (111) | 0 → 111 | complete 111/111 | unchanged 104/31 | 1.33–1.48 → 1.37–1.53 | 331–382 → 331–348 | 304,122 → 380,054 | | mpv (209) | 0 → 209 | complete 209/209 | unchanged 676/546 | 1.42–1.49 → 1.43–1.49 | 341–359 → 346–359 | 505,759 → 606,327 | | Ghostty (323) | 0 → 323 | partial 323/320 (3 types declared in an extension of an external type) | unchanged 1362/308 | 1.66–1.90 → 1.66–1.86 | 450–451 → 457–473 | not measured (bytes 6.30 MB → 7.02 MB) | - Wall time, CPU time and peak RSS are within run-to-run noise. ObjC, conformance and vtable counts are identical to main. - Tokens: the full envelope grows 20–25%. About two thirds of that is the new relative-pointer records (name, parent, accessor), which keep the reader's existing "every decoded relative pointer is recorded" behavior. Requesting only `swift_decls` (`--filter-output normalized_result.result.swift_decls`) costs 37,596 tokens for mpv's 209 declarations. - Remaining proof gaps: - On BetterDisplay (964 types), Raycast (7,097) and Microsoft To Do (94), even `--max-records 20000` is used up by ObjC and conformance records before the reader reaches `__swift5_types`, both on main and here. The facet then correctly reports `partial` with `max_records_reached`. Letting callers pick facets so the budget isn't shared is a separate change. - No arm64e Swift fixture: the verifier builds Swift only for the arm64 link modes, as before. - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm run test:local -- src/native/AppleSwiftNominalTypes.test.ts src/native/AppleSwiftVtables.test.ts src/native/AppleDispatchMetadata.test.ts src/native/AppleSwiftRelativePointer.test.ts src/domain/native/objcSwiftMetadata.test.ts src/domain/native/nativeInvestigationGraph.test.ts src/native/AppleSwiftConformanceTypeReference.test.ts src/application/native/EnhancedNativeInvestigation.test.ts src/application/EnhancedTools.objcClasses.test.ts`: 9 files, 161 tests passed. The new indirect-parent test was added afterwards; the 3 native files rerun passed, 97 tests. - Review follow-up (3f6179a64) fixes the four Greptile findings: an intrinsic depth bound that works with the cache, a 4096-character `qualified_name` cap, vtables decoded in record order, a requirement that accessors be in executable mapped code, and name failures forwarded to vtable coverage only for class or unknown-kind records. There are 5 new regression tests, and all 5 fail on 4e08dc0e2. After the follow-up: `npm run test:local -- src/native/AppleSwiftNominalTypes.test.ts src/native/AppleSwiftVtables.test.ts src/native/AppleDispatchMetadata.test.ts src/native/AppleSwiftRelativePointer.test.ts` passed 110 tests; `check:fast`, `format:check` and `verify:apple-dispatch` passed; the real-app outputs for PureMac, mpv and Ghostty are byte-identical to the first push. - `npm run verify:apple-dispatch`: `ok: true`, `target_executed: false`; `swift-{dyld-info,chained}-{symbols,stripped}` each have 7 declarations. - `npm run check:fast`: passed. - `npm run format:check`: passed. - `npm run docs:check`: passed. - Real-app runs: see the table above. ## Compatibility, safety, and release - Breaking changes or migration steps: see the contract note. The `swift_decls` item schema changed, but no produced data changes. - Real Hopper/Ghidra, browser, or OS coverage: macOS byte reader only. No provider change. - Package or release metadata impact: none. - Security, privacy, process, or containment review: static reads of the selected file. No target execution; the verifier reports `target_executed: false`. ## Author readiness Human owner: @kaoru0822-kitauji - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:46:21 +00:00