morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5683] [Bug] Swift dispatch metadata labels imported descriptors as unsupported kinds #3434

New issue

Closed

opened 2026-10-31 00:40:26 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:40:26 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5683

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-native-dispatch-metadata exits 0 but drops two Swift conformances involving imported descriptors, reporting Unsupported Swift type descriptor kind and Unsupported Swift protocol descriptor kind. Independent Apple symbol/fixup inspection identifies valid external bindings to Swift.Int and Swift.Equatable. The wholly local conformance succeeds. Coverage is partial, but the failure explanation loses the external identity and implies that an external descriptor's kind was inspected.

Reproduction

With the checkout's locked dependencies installed and CLI built, run from its root on macOS arm64:

work_dir="$(mktemp -d)"
S="$work_dir"
mkdir -p "$S/home" "$S/cfg" "$S/data" "$S/cache" "$S/state" "$S/tmp"
cat > "$work_dir/imported.swift" <<'SWIFT'
protocol Local { func h() }
struct LocalType: Local { func h() {} }
extension Int: Local { func h() {} }
struct Eq: Equatable { let v: Int }
SWIFT
swiftc -Onone -g -parse-as-library -emit-library \
  -o "$work_dir/libimported.dylib" "$work_dir/imported.swift"
env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" \
  XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" \
  XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \
  node scripts/rea.mjs inspect-native-dispatch-metadata \
  "$work_dir/libimported.dylib" --json > "$work_dir/out.json"
jq '.normalized_result.result | {
  conformances: [.swift_conformances[] | [.type_name, .protocol_name]],
  coverage: [.coverage[] | select(.facet == "swift_conformances_static_witness_slots" or .facet == "pointer_fixups")]
}' "$work_dir/out.json"
nm "$work_dir/libimported.dylib" | xcrun swift-demangle --compact | \
  grep 'protocol conformance descriptor for'
xcrun dyld_info -fixups "$work_dir/libimported.dylib" | \
  grep -E 'SQMp|SiMn'

The independent verifier ran this fixture twice; the submitter freshly replayed it once with the same result. Only this reduced fixture was tested. The dylib was inspected without executing target code.

Expected behavior

If imported descriptors cannot be decoded from this image, retain their unresolved external type/protocol identity, recoverable binding evidence, and metadata locations, with partial coverage. Full external witness-layout decoding is not required. Do not report an unsupported descriptor kind after reading unrelated local bytes.

Contract: native investigation documentation documents simple Swift conformances, metadata locations and fixup-aware pointers; its external bind-name example specifically concerns Objective-C superclasses. Architecture and Cleanup requires explicit unresolved identity, partial facts and meaningful failure reasons.

Impact and scope

P2: two of three valid conformance cases lose their rows and external identities and receive misleading failure evidence. The CLI remains usable and marks coverage partial. nm, swift-demangle, and dyld_info recover the names/bindings for this fixture; they do not establish an equivalent complete REA metadata/witness inventory. Population-wide frequency, other architectures, shipped frameworks, MCP, and security impact were not tested.

REA version

rea-agents 6.7.0; reproduced at source commit 9d8e7803326a382cbbb3ffdb900f5e93e5442d68. Fetched and rechecked origin/main at 60956c8c56475e7aaab9559f780908851677ced6: no intervening commits change the cited implementation or contract files. Runtime reproduction was run at the baseline; a fresh current-main build was not run.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1 arm64; Apple Swift 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1). Native macOS provider; Hopper/Ghidra are not involved.

Target and artifact context

Self-built thin arm64 Mach-O dylib with a local conformance control, Swift.Int: Local, and Eq: Swift.Equatable, linked with DYLD_CHAINED_PTR_64_OFFSET fixups.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Observed submitter replay (CLI exit 0):

swift_conformances: [["LocalType", "Local"]]
swift_conformances_static_witness_slots:
  status: partial; examined: 3; decoded: 1
  reason: conformance_decode_failed at 0x9c0: Unsupported Swift type descriptor kind;
          conformance_decode_failed at 0x9c4: Unsupported Swift protocol descriptor kind
pointer_fixups:
  status: complete; reason: chained fixups: DYLD_CHAINED_PTR_64_OFFSET

Independent Apple tools on the same replay:

0x888  protocol conformance descriptor for imported.LocalType : imported.Local in imported
0x898  protocol conformance descriptor for Swift.Int : imported.Local in imported
0x8a8  protocol conformance descriptor for imported.Eq : Swift.Equatable in imported
0x4068 bind libswiftCore/_$sSQMp
0x4080 bind libswiftCore/_$sSiMn

Addresses vary with compiler/linker layout. Relative-pointer evidence and coverage error positions still lead back to the metadata; not every location is erased. The verifier also observed an imported-protocol indirect pointer with null target and decode.status: decoded. Evidence provenance: observed; local verification: independently reproduced; reproduction completeness: complete.

Typed outcome or recovery expected

Decoded supported facts or an accurate unresolved-external-reference result retaining binding/location evidence. Coverage must remain partial where external layout or witness slots are unresolved. Proposed restoration checks: preserve the local conformance and synchronous witness slot; both imported cases must avoid false kind diagnoses, while a genuinely unsupported local descriptor must retain its meaningful kind diagnostic. No fix or restoration was tested.

Analysis or suspected cause

Source-derived reasoning: AppleDispatchMetadata.ts:63-68 projects bind pointers to 0n, although the adjacent bound() reader can retain bind symbols. AppleSwiftDispatchFacets.ts:47-80 carries that zero through indirect relative pointers. The conformance decoder checks kind bits at zero before emitting a row. On this dylib, zero maps to the Mach-O header rather than an imported descriptor. This is a REA reader/consumer identity-loss path, not an established bundled-dependency defect.

Additional context

Prepared with AI assistance. No earlier known-good version was established. Related #4368 and PRs #4429/#4472 concern empty names escaping recovery and invalidating the whole result; this case returns a successful partial result with nonempty external identities lost before name decoding. Earlier fixup PRs #953/#1709 cover Objective-C bind recovery and chained-pointer membership/ordinals, respectively; neither addresses this Swift consumer path. Separate Swift vtable counter report #5681 has a different restoration signal.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5683 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior `inspect-native-dispatch-metadata` exits 0 but drops two Swift conformances involving imported descriptors, reporting `Unsupported Swift type descriptor kind` and `Unsupported Swift protocol descriptor kind`. Independent Apple symbol/fixup inspection identifies valid external bindings to `Swift.Int` and `Swift.Equatable`. The wholly local conformance succeeds. Coverage is partial, but the failure explanation loses the external identity and implies that an external descriptor's kind was inspected. ### Reproduction With the checkout's locked dependencies installed and CLI built, run from its root on macOS arm64: ```sh work_dir="$(mktemp -d)" S="$work_dir" mkdir -p "$S/home" "$S/cfg" "$S/data" "$S/cache" "$S/state" "$S/tmp" cat > "$work_dir/imported.swift" <<'SWIFT' protocol Local { func h() } struct LocalType: Local { func h() {} } extension Int: Local { func h() {} } struct Eq: Equatable { let v: Int } SWIFT swiftc -Onone -g -parse-as-library -emit-library \ -o "$work_dir/libimported.dylib" "$work_dir/imported.swift" env -i PATH="$PATH" HOME="$S/home" XDG_CONFIG_HOME="$S/cfg" \ XDG_DATA_HOME="$S/data" XDG_CACHE_HOME="$S/cache" \ XDG_STATE_HOME="$S/state" TMPDIR="$S/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-native-dispatch-metadata \ "$work_dir/libimported.dylib" --json > "$work_dir/out.json" jq '.normalized_result.result | { conformances: [.swift_conformances[] | [.type_name, .protocol_name]], coverage: [.coverage[] | select(.facet == "swift_conformances_static_witness_slots" or .facet == "pointer_fixups")] }' "$work_dir/out.json" nm "$work_dir/libimported.dylib" | xcrun swift-demangle --compact | \ grep 'protocol conformance descriptor for' xcrun dyld_info -fixups "$work_dir/libimported.dylib" | \ grep -E 'SQMp|SiMn' ``` The independent verifier ran this fixture twice; the submitter freshly replayed it once with the same result. Only this reduced fixture was tested. The dylib was inspected without executing target code. ### Expected behavior If imported descriptors cannot be decoded from this image, retain their unresolved external type/protocol identity, recoverable binding evidence, and metadata locations, with partial coverage. Full external witness-layout decoding is not required. Do not report an unsupported descriptor kind after reading unrelated local bytes. Contract: [native investigation documentation](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/docs/native-investigation.md#L419-L434) documents simple Swift conformances, metadata locations and fixup-aware pointers; its external bind-name example specifically concerns Objective-C superclasses. [Architecture and Cleanup](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/AGENTS.md#architecture-and-cleanup) requires explicit unresolved identity, partial facts and meaningful failure reasons. ### Impact and scope P2: two of three valid conformance cases lose their rows and external identities and receive misleading failure evidence. The CLI remains usable and marks coverage partial. `nm`, `swift-demangle`, and `dyld_info` recover the names/bindings for this fixture; they do not establish an equivalent complete REA metadata/witness inventory. Population-wide frequency, other architectures, shipped frameworks, MCP, and security impact were not tested. ### REA version `rea-agents` 6.7.0; reproduced at source commit `9d8e7803326a382cbbb3ffdb900f5e93e5442d68`. Fetched and rechecked `origin/main` at `60956c8c56475e7aaab9559f780908851677ced6`: no intervening commits change the cited implementation or contract files. Runtime reproduction was run at the baseline; a fresh current-main build was not run. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1 arm64; Apple Swift 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1). Native macOS provider; Hopper/Ghidra are not involved. ### Target and artifact context Self-built thin arm64 Mach-O dylib with a local conformance control, `Swift.Int: Local`, and `Eq: Swift.Equatable`, linked with `DYLD_CHAINED_PTR_64_OFFSET` fixups. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Observed submitter replay (CLI exit 0): ```text swift_conformances: [["LocalType", "Local"]] swift_conformances_static_witness_slots: status: partial; examined: 3; decoded: 1 reason: conformance_decode_failed at 0x9c0: Unsupported Swift type descriptor kind; conformance_decode_failed at 0x9c4: Unsupported Swift protocol descriptor kind pointer_fixups: status: complete; reason: chained fixups: DYLD_CHAINED_PTR_64_OFFSET ``` Independent Apple tools on the same replay: ```text 0x888 protocol conformance descriptor for imported.LocalType : imported.Local in imported 0x898 protocol conformance descriptor for Swift.Int : imported.Local in imported 0x8a8 protocol conformance descriptor for imported.Eq : Swift.Equatable in imported 0x4068 bind libswiftCore/_$sSQMp 0x4080 bind libswiftCore/_$sSiMn ``` Addresses vary with compiler/linker layout. Relative-pointer evidence and coverage error positions still lead back to the metadata; not every location is erased. The verifier also observed an imported-protocol indirect pointer with null target and `decode.status: decoded`. Evidence provenance: observed; local verification: independently reproduced; reproduction completeness: complete. ### Typed outcome or recovery expected Decoded supported facts or an accurate unresolved-external-reference result retaining binding/location evidence. Coverage must remain partial where external layout or witness slots are unresolved. Proposed restoration checks: preserve the local conformance and synchronous witness slot; both imported cases must avoid false kind diagnoses, while a genuinely unsupported local descriptor must retain its meaningful kind diagnostic. No fix or restoration was tested. ### Analysis or suspected cause Source-derived reasoning: [AppleDispatchMetadata.ts:63-68](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/AppleDispatchMetadata.ts#L63-L68) projects bind pointers to `0n`, although the adjacent `bound()` reader can retain bind symbols. [AppleSwiftDispatchFacets.ts:47-80](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/AppleSwiftDispatchFacets.ts#L47-L80) carries that zero through indirect relative pointers. [The conformance decoder](https://github.com/morluto/rea/blob/9d8e7803326a382cbbb3ffdb900f5e93e5442d68/src/native/AppleSwiftDispatchFacets.ts#L109-L133) checks kind bits at zero before emitting a row. On this dylib, zero maps to the Mach-O header rather than an imported descriptor. This is a REA reader/consumer identity-loss path, not an established bundled-dependency defect. ### Additional context Prepared with AI assistance. No earlier known-good version was established. Related #4368 and PRs #4429/#4472 concern empty names escaping recovery and invalidating the whole result; this case returns a successful partial result with nonempty external identities lost before name decoding. Earlier fixup PRs #953/#1709 cover Objective-C bind recovery and chained-pointer membership/ordinals, respectively; neither addresses this Swift consumer path. Separate Swift vtable counter report #5681 has a different restoration signal.

bsakaguchi closed this issue

2026-10-31 00:40:26 +00:00

bsakaguchi commented

2026-10-31 00:40:27 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 20, 2026):

Additional verified trigger for the same source-derived bind-to-zero cause: a Swift executable with a nonzero image base drops both Point: Equatable and Point: CustomStringConvertible, reporting unmapped zero instead of the dylib's unsupported-kind errors. Freshly reproduced at 9d8e7803326a382cbbb3ffdb900f5e93e5442d68 on macOS arm64 with Apple Swift 6.4; fetched main at ccc7da309d7db385d5c6d73b0a3a8e093be33504 has intervening changes to AppleDispatchMetadata.ts and the native-investigation docs, so main was not re-run. Save this as f.swift, then run the commands below from the built checkout (the target is compiled and inspected, never executed). REA exits 0, retains only C: Local, and reports partial conformance coverage (examined: 3, decoded: 1); dyld_info confirms binds to libswiftCore/_$sSQMp and libswiftCore/_$ss23CustomStringConvertibleMp. Expected: preserve both Point conformance identities and binding/location evidence as partial records when external descriptors cannot be decoded, while retaining the local control.

import Foundation
struct Point: Equatable, CustomStringConvertible {
  var x: Int
  var description: String { "\(x)" }
}
protocol Local { func f() }
final class C: Local { func f() {} }
print(Point(x: 1), C())

swiftc -module-cache-path swift-module-cache -o fixture f.swift
mkdir -p home cfg data cache state tmp
env -i PATH="$PATH" HOME="$PWD/home" XDG_CONFIG_HOME="$PWD/cfg" \
  XDG_DATA_HOME="$PWD/data" XDG_CACHE_HOME="$PWD/cache" \
  XDG_STATE_HOME="$PWD/state" TMPDIR="$PWD/tmp" REA_LOG_LEVEL=silent \
  node scripts/rea.mjs inspect-native-dispatch-metadata fixture --json
xcrun dyld_info -fixups fixture

conformance_decode_failed at 0x1000012b4: Unmapped or ambiguous metadata pointer 0x0;
conformance_decode_failed at 0x1000012b8: Unmapped or ambiguous metadata pointer 0x0

@coygeek commented on GitHub (Oct 20, 2026): Additional verified trigger for the same source-derived bind-to-zero cause: a Swift executable with a nonzero image base drops both `Point: Equatable` and `Point: CustomStringConvertible`, reporting unmapped zero instead of the dylib's unsupported-kind errors. Freshly reproduced at `9d8e7803326a382cbbb3ffdb900f5e93e5442d68` on macOS arm64 with Apple Swift 6.4; fetched main at `ccc7da309d7db385d5c6d73b0a3a8e093be33504` has intervening changes to `AppleDispatchMetadata.ts` and the native-investigation docs, so main was not re-run. Save this as `f.swift`, then run the commands below from the built checkout (the target is compiled and inspected, never executed). REA exits 0, retains only `C: Local`, and reports partial conformance coverage (`examined: 3`, `decoded: 1`); `dyld_info` confirms binds to `libswiftCore/_$sSQMp` and `libswiftCore/_$ss23CustomStringConvertibleMp`. Expected: preserve both `Point` conformance identities and binding/location evidence as partial records when external descriptors cannot be decoded, while retaining the local control. ```swift import Foundation struct Point: Equatable, CustomStringConvertible { var x: Int var description: String { "\(x)" } } protocol Local { func f() } final class C: Local { func f() {} } print(Point(x: 1), C()) ``` ```sh swiftc -module-cache-path swift-module-cache -o fixture f.swift mkdir -p home cfg data cache state tmp env -i PATH="$PATH" HOME="$PWD/home" XDG_CONFIG_HOME="$PWD/cfg" \ XDG_DATA_HOME="$PWD/data" XDG_CACHE_HOME="$PWD/cache" \ XDG_STATE_HOME="$PWD/state" TMPDIR="$PWD/tmp" REA_LOG_LEVEL=silent \ node scripts/rea.mjs inspect-native-dispatch-metadata fixture --json xcrun dyld_info -fixups fixture ``` ```text conformance_decode_failed at 0x1000012b4: Unmapped or ambiguous metadata pointer 0x0; conformance_decode_failed at 0x1000012b8: Unmapped or ambiguous metadata pointer 0x0 ```

bsakaguchi commented

2026-10-31 00:40:27 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with canonical issue #4368 ([Bug] Empty Swift conformance names escape entry recovery and invalidate dispatch metadata) and sibling Swift dispatch metadata reports #5424, #5529, #5543, #5681, #5682, #5685.

Consolidated under Cluster AZ (Native Dispatch Metadata Inspection & Provider Fallbacks) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with canonical issue #4368 (`[Bug] Empty Swift conformance names escape entry recovery and invalidate dispatch metadata`) and sibling Swift dispatch metadata reports #5424, #5529, #5543, #5681, #5682, #5685. Consolidated under **Cluster AZ (Native Dispatch Metadata Inspection & Provider Fallbacks)** in tracker #4330.

bsakaguchi referenced this issue

2026-10-31 01:09:06 +00:00

[PR #3440] [CLOSED] fix(security): pin CI actions, protect Hopper xauth and downloads, flag old apktool #5743

bsakaguchi referenced this issue

2026-10-31 01:12:01 +00:00

[PR #4199] [MERGED] ci: pin Cutter actions and guard workflow-only changes #6121