morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7445] [Bug] Retroactive Swift conformance flag suppresses a readable static witness slot #4450

New issue

Open

opened 2026-10-31 00:49:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:49:54 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7445

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

inspect-native-dispatch-metadata returns a partial conformance and an empty swift_dispatch_slots array for a compiler-generated executable declaring a retroactive conformance between two statically linked Swift modules. It resolves RetroT: RetroP and the witness-table location, but reports Conditional, generic, resilient or missing static witness table is not decoded and conformance coverage examined 1 / decoded 0.

The conformance flags are 0x48: type-reference kind 1 plus retroactivity. Apple’s fixup output independently resolves the first witness slot to the local synchronous method. Changing only the retroactivity bit in a diagnostic copy makes the same REA command decode that slot and report complete conformance coverage. Both CLI calls exit 0.

Reproduction

Run this script from a clean, built checkout of source commit f8f7ac96de8a66a4ab9a23d87e04779bfcd11045, with its locked dependencies, Node.js 24.22.1, Python 3 and the Apple Swift toolchain on PATH. The script creates all synthetic inputs and isolates HOME, XDG directories and TMPDIR. It compiles and statically inspects the binaries; it never executes them. The control copy is for diagnosis only.

The complete script below was independently run successfully. Its assertions characterize the failure and the differential control, rather than a repaired REA implementation.

#!/bin/sh
set -eu
export REPO="$PWD"
export WORK="$(mktemp -d)"
mkdir -p "$WORK/home" "$WORK/config" "$WORK/data" "$WORK/cache" "$WORK/state" "$WORK/tmp"
export HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config"
export XDG_DATA_HOME="$WORK/data" XDG_CACHE_HOME="$WORK/cache"
export XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/tmp"
cd "$WORK"
cat > a.swift <<'SWIFT'
public protocol RetroP { func pValue() -> Int }
SWIFT
cat > b.swift <<'SWIFT'
public struct RetroT { public init() {} }
SWIFT
cat > main.swift <<'SWIFT'
import A
import B
extension RetroT: @retroactive RetroP {
  public func pValue() -> Int { 5 }
}
@_cdecl("retro_entry")
public func retroEntry() -> Int { RetroT().pValue() }
SWIFT
xcrun swiftc -emit-library -static -emit-module -module-name A -O \
  -module-cache-path "$WORK/cache/swift" a.swift -o libA.a
xcrun swiftc -emit-library -static -emit-module -module-name B -O \
  -module-cache-path "$WORK/cache/swift" b.swift -o libB.a
xcrun swiftc -O -I. -L. -lA -lB -module-cache-path "$WORK/cache/swift" main.swift -o retro
rea_isolated() {
  env -i PATH="$PATH" HOME="$HOME" XDG_CONFIG_HOME="$XDG_CONFIG_HOME" \
    XDG_DATA_HOME="$XDG_DATA_HOME" XDG_CACHE_HOME="$XDG_CACHE_HOME" \
    XDG_STATE_HOME="$XDG_STATE_HOME" TMPDIR="$TMPDIR" \
    node "$REPO/scripts/rea.mjs" "$@"
}
rea_isolated --version
rea_isolated inspect-native-dispatch-metadata "$WORK/retro" --json > original.json
xcrun dyld_info -fixups retro > fixups.txt
xcrun nm -n retro > symbols.txt
python3 -I - <<'PY'
import json, os, pathlib
w = pathlib.Path(os.environ['WORK'])
r = json.loads((w/'original.json').read_text())['normalized_result']['result']
c = r['swift_conformances'][0]
assert (c['type_name'], c['protocol_name']) == ('RetroT', 'RetroP')
assert c['decode']['status'] == 'partial' and r['swift_dispatch_slots'] == []
b = bytearray((w/'retro').read_bytes())
offset = c['location']['file_offset'] + 12
flags = int.from_bytes(b[offset:offset+4], 'little')
assert flags == 0x48, hex(flags)
b[offset:offset+4] = (flags & ~0x40).to_bytes(4, 'little')
(w/'retro-control').write_bytes(b)
assert sum(a != z for a,z in zip((w/'retro').read_bytes(), b)) == 1
print('Original flags: 0x48; control flags: 0x08; exactly one byte changed')
PY
rea_isolated inspect-native-dispatch-metadata "$WORK/retro-control" --json > control.json
python3 -I - <<'PY'
import json, os, pathlib
w = pathlib.Path(os.environ['WORK'])
for label in ['original', 'control']:
    r = json.loads((w/(label+'.json')).read_text())['normalized_result']['result']
    facet = next(x for x in r['coverage'] if x['facet'] == 'swift_conformances_static_witness_slots')
    print(label, json.dumps({'decode': r['swift_conformances'][0]['decode'],
          'slots': [(s['slot_index'], s['implementation_address']) for s in r['swift_dispatch_slots']],
          'coverage': facet}))
    if label == 'control':
        assert r['swift_conformances'][0]['decode']['status'] == 'decoded'
        assert len(r['swift_dispatch_slots']) == 1
        s = r['swift_dispatch_slots'][0]
        assert s['slot_index'] == 1 and s['decode']['status'] == 'decoded'
        assert facet['status'] == 'complete' and facet['examined'] == facet['decoded'] == 1
PY

Expected behavior

Return the supported static synchronous witness slot for this non-generic struct/protocol conformance. Retroactivity alone should not cause a readable local witness table to be classified as conditional, generic, resilient or missing. The conformance should decode, slot 1 should identify its in-image implementation, and swift_conformances_static_witness_slots should be complete for this one-conformance fixture.

Expected source: docs/native-investigation.md:726 advertises simple Swift conformances and static synchronous witness slots, including exact metadata addresses. Expected source: docs/native-investigation.md:834 lists generic, resilient, async/coroutine tables and inherited overrides as partial or unsupported. This fixture has none of those properties. If retroactivity is an intentional additional limitation, its partial reason should identify it accurately.

Impact and scope

P2: a bounded valid-input family loses a recoverable protocol dispatch relationship, and callers receive an explanation that does not identify the actual exclusion. The conformance identities and table location remain available, and aggregate coverage explicitly warns that decoding is partial. No crash, target execution, artifact loss or security-boundary crossing was observed.

For this unstripped fixture, nm -n and xcrun dyld_info -fixups provide a demonstrated manual way to recover the method and witness-pointer relationship. They do not repair REA's returned inventory. Clearing metadata bits is a diagnostic control, not a recommended production workaround. Production prevalence, stripped artifacts, other architectures and MCP execution were not established.

REA version

Exact node scripts/rea.mjs --version output: 6.8.0.
Source commit: f8f7ac96de8a66a4ab9a23d87e04779bfcd11045.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1 arm64; Apple Swift 6.4 (swiftlang-6.4.0.34.1, swift-driver 1.168.6). Native Mach-O byte reader; no Hopper, Ghidra or IDA session.

Target and artifact context

Self-built arm64 Mach-O executable. Module A defines a public protocol with one synchronous method; module B defines a public non-generic struct; the executable module declares its @retroactive conformance. Both libraries are linked statically. The binary uses DYLD_CHAINED_PTR_64_OFFSET fixups and has a local witness table and local implementation. The control changes exactly one byte of a scratch copy without executing it or renewing its signature.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The independent run of the script printed these results; addresses may vary with compiler/linker layout:

6.8.0
Original flags: 0x48; control flags: 0x08; exactly one byte changed
original: conformance partial; slots []; conformance coverage partial, examined 1, decoded 0
control: conformance decoded; slot 1 -> 0x100000728; conformance coverage complete, examined 1, decoded 1

The original conformance reason was exactly Conditional, generic, resilient or missing static witness table is not decoded. Its facet reason included conformance_decode_partial and swift_records_unresolved.

On the same original artifact, xcrun dyld_info -fixups reported:

__DATA_CONST __const 0x100004000 rebase 0x10000077C
__DATA_CONST __const 0x100004008 rebase 0x100000728

The conformance descriptor was at file offset 1916. Its flags at offset 1928 changed from 0x48 to 0x08; an independent byte comparison found exactly one changed byte. nm -n identified the conforming method and its protocol witness thunk at 0x100000728, matching the recovered slot. The table, names, pointer fixups and implementation bytes were unchanged in the control.

Failure source: src/native/AppleSwiftDispatchFacets.ts:379 requires (flags & ~0x38) === 0 before static-witness decoding. Failure source: src/native/AppleSwiftDispatchFacets.ts:387 supplies the fallback explanation, and src/native/AppleSwiftDispatchFacets.ts:444 skips the slot loop when that explanation is non-null. Failure source: src/native/AppleDispatchCoverage.ts:186 derives the partial facet from partial conformance/slot facts.

Typed outcome or recovery expected

Preserve the currently resolved identities and locations, decode the supported local slot, and retain truthful partial coverage for actual conditional, generic, resilient, asynchronous or unresolved cases. The differential control establishes decodability for this fixture; no changed-REA restoration run is claimed.

Analysis or suspected cause

The guard admits only type-reference-kind bits and rejects every other conformance flag. Swift's ABI ConformanceFlags declaration assigns retroactivity to bit 6, separately from conditional requirements and resilient/generic witness flags. Consequently 0x48 fails the gate while 0x08 passes. The observed one-bit comparison isolates that exclusion without changing the witness table or implementation.

Additional context

The reviewed related reports cover different failures: #5738 concerns inline Objective-C class names, #5683 external descriptor bindings, #5682 qualified names, #4368 empty-name validation and #6000 kind-3 rebase-to-zero preservation. None reports this flag-based exclusion. No known-good REA version or regression range was established.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7445 ## Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ## Area Native analysis or decompilation ## Observed behavior `inspect-native-dispatch-metadata` returns a partial conformance and an empty `swift_dispatch_slots` array for a compiler-generated executable declaring a retroactive conformance between two statically linked Swift modules. It resolves `RetroT: RetroP` and the witness-table location, but reports `Conditional, generic, resilient or missing static witness table is not decoded` and conformance coverage examined 1 / decoded 0. The conformance flags are `0x48`: type-reference kind 1 plus retroactivity. Apple’s fixup output independently resolves the first witness slot to the local synchronous method. Changing only the retroactivity bit in a diagnostic copy makes the same REA command decode that slot and report complete conformance coverage. Both CLI calls exit 0. ## Reproduction Run this script from a clean, built checkout of source commit `f8f7ac96de8a66a4ab9a23d87e04779bfcd11045`, with its locked dependencies, Node.js 24.22.1, Python 3 and the Apple Swift toolchain on PATH. The script creates all synthetic inputs and isolates HOME, XDG directories and TMPDIR. It compiles and statically inspects the binaries; it never executes them. The control copy is for diagnosis only. The complete script below was independently run successfully. Its assertions characterize the failure and the differential control, rather than a repaired REA implementation. ```sh #!/bin/sh set -eu export REPO="$PWD" export WORK="$(mktemp -d)" mkdir -p "$WORK/home" "$WORK/config" "$WORK/data" "$WORK/cache" "$WORK/state" "$WORK/tmp" export HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config" export XDG_DATA_HOME="$WORK/data" XDG_CACHE_HOME="$WORK/cache" export XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/tmp" cd "$WORK" cat > a.swift <<'SWIFT' public protocol RetroP { func pValue() -> Int } SWIFT cat > b.swift <<'SWIFT' public struct RetroT { public init() {} } SWIFT cat > main.swift <<'SWIFT' import A import B extension RetroT: @retroactive RetroP { public func pValue() -> Int { 5 } } @_cdecl("retro_entry") public func retroEntry() -> Int { RetroT().pValue() } SWIFT xcrun swiftc -emit-library -static -emit-module -module-name A -O \ -module-cache-path "$WORK/cache/swift" a.swift -o libA.a xcrun swiftc -emit-library -static -emit-module -module-name B -O \ -module-cache-path "$WORK/cache/swift" b.swift -o libB.a xcrun swiftc -O -I. -L. -lA -lB -module-cache-path "$WORK/cache/swift" main.swift -o retro rea_isolated() { env -i PATH="$PATH" HOME="$HOME" XDG_CONFIG_HOME="$XDG_CONFIG_HOME" \ XDG_DATA_HOME="$XDG_DATA_HOME" XDG_CACHE_HOME="$XDG_CACHE_HOME" \ XDG_STATE_HOME="$XDG_STATE_HOME" TMPDIR="$TMPDIR" \ node "$REPO/scripts/rea.mjs" "$@" } rea_isolated --version rea_isolated inspect-native-dispatch-metadata "$WORK/retro" --json > original.json xcrun dyld_info -fixups retro > fixups.txt xcrun nm -n retro > symbols.txt python3 -I - <<'PY' import json, os, pathlib w = pathlib.Path(os.environ['WORK']) r = json.loads((w/'original.json').read_text())['normalized_result']['result'] c = r['swift_conformances'][0] assert (c['type_name'], c['protocol_name']) == ('RetroT', 'RetroP') assert c['decode']['status'] == 'partial' and r['swift_dispatch_slots'] == [] b = bytearray((w/'retro').read_bytes()) offset = c['location']['file_offset'] + 12 flags = int.from_bytes(b[offset:offset+4], 'little') assert flags == 0x48, hex(flags) b[offset:offset+4] = (flags & ~0x40).to_bytes(4, 'little') (w/'retro-control').write_bytes(b) assert sum(a != z for a,z in zip((w/'retro').read_bytes(), b)) == 1 print('Original flags: 0x48; control flags: 0x08; exactly one byte changed') PY rea_isolated inspect-native-dispatch-metadata "$WORK/retro-control" --json > control.json python3 -I - <<'PY' import json, os, pathlib w = pathlib.Path(os.environ['WORK']) for label in ['original', 'control']: r = json.loads((w/(label+'.json')).read_text())['normalized_result']['result'] facet = next(x for x in r['coverage'] if x['facet'] == 'swift_conformances_static_witness_slots') print(label, json.dumps({'decode': r['swift_conformances'][0]['decode'], 'slots': [(s['slot_index'], s['implementation_address']) for s in r['swift_dispatch_slots']], 'coverage': facet})) if label == 'control': assert r['swift_conformances'][0]['decode']['status'] == 'decoded' assert len(r['swift_dispatch_slots']) == 1 s = r['swift_dispatch_slots'][0] assert s['slot_index'] == 1 and s['decode']['status'] == 'decoded' assert facet['status'] == 'complete' and facet['examined'] == facet['decoded'] == 1 PY ``` ## Expected behavior Return the supported static synchronous witness slot for this non-generic struct/protocol conformance. Retroactivity alone should not cause a readable local witness table to be classified as conditional, generic, resilient or missing. The conformance should decode, slot 1 should identify its in-image implementation, and `swift_conformances_static_witness_slots` should be complete for this one-conformance fixture. Expected source: `docs/native-investigation.md:726` advertises simple Swift conformances and static synchronous witness slots, including exact metadata addresses. Expected source: `docs/native-investigation.md:834` lists generic, resilient, async/coroutine tables and inherited overrides as partial or unsupported. This fixture has none of those properties. If retroactivity is an intentional additional limitation, its partial reason should identify it accurately. ## Impact and scope P2: a bounded valid-input family loses a recoverable protocol dispatch relationship, and callers receive an explanation that does not identify the actual exclusion. The conformance identities and table location remain available, and aggregate coverage explicitly warns that decoding is partial. No crash, target execution, artifact loss or security-boundary crossing was observed. For this unstripped fixture, `nm -n` and `xcrun dyld_info -fixups` provide a demonstrated manual way to recover the method and witness-pointer relationship. They do not repair REA's returned inventory. Clearing metadata bits is a diagnostic control, not a recommended production workaround. Production prevalence, stripped artifacts, other architectures and MCP execution were not established. ## REA version Exact `node scripts/rea.mjs --version` output: `6.8.0`. Source commit: `f8f7ac96de8a66a4ab9a23d87e04779bfcd11045`. ## Runtime and platform Node.js v24.22.1; macOS 27.0.1 arm64; Apple Swift 6.4 (`swiftlang-6.4.0.34.1`, swift-driver 1.168.6). Native Mach-O byte reader; no Hopper, Ghidra or IDA session. ## Target and artifact context Self-built arm64 Mach-O executable. Module A defines a public protocol with one synchronous method; module B defines a public non-generic struct; the executable module declares its `@retroactive` conformance. Both libraries are linked statically. The binary uses `DYLD_CHAINED_PTR_64_OFFSET` fixups and has a local witness table and local implementation. The control changes exactly one byte of a scratch copy without executing it or renewing its signature. ## Interface CLI ## Earliest failure stage Native analysis or reconstruction ## Relevant output or evidence The independent run of the script printed these results; addresses may vary with compiler/linker layout: ```text 6.8.0 Original flags: 0x48; control flags: 0x08; exactly one byte changed original: conformance partial; slots []; conformance coverage partial, examined 1, decoded 0 control: conformance decoded; slot 1 -> 0x100000728; conformance coverage complete, examined 1, decoded 1 ``` The original conformance reason was exactly `Conditional, generic, resilient or missing static witness table is not decoded`. Its facet reason included `conformance_decode_partial` and `swift_records_unresolved`. On the same original artifact, `xcrun dyld_info -fixups` reported: ```text __DATA_CONST __const 0x100004000 rebase 0x10000077C __DATA_CONST __const 0x100004008 rebase 0x100000728 ``` The conformance descriptor was at file offset 1916. Its flags at offset 1928 changed from `0x48` to `0x08`; an independent byte comparison found exactly one changed byte. `nm -n` identified the conforming method and its protocol witness thunk at `0x100000728`, matching the recovered slot. The table, names, pointer fixups and implementation bytes were unchanged in the control. Failure source: `src/native/AppleSwiftDispatchFacets.ts:379` requires `(flags & ~0x38) === 0` before static-witness decoding. Failure source: `src/native/AppleSwiftDispatchFacets.ts:387` supplies the fallback explanation, and `src/native/AppleSwiftDispatchFacets.ts:444` skips the slot loop when that explanation is non-null. Failure source: `src/native/AppleDispatchCoverage.ts:186` derives the partial facet from partial conformance/slot facts. ## Typed outcome or recovery expected Preserve the currently resolved identities and locations, decode the supported local slot, and retain truthful partial coverage for actual conditional, generic, resilient, asynchronous or unresolved cases. The differential control establishes decodability for this fixture; no changed-REA restoration run is claimed. ## Analysis or suspected cause The guard admits only type-reference-kind bits and rejects every other conformance flag. [Swift's ABI ConformanceFlags declaration](https://github.com/swiftlang/swift/blob/main/include/swift/ABI/MetadataValues.h) assigns retroactivity to bit 6, separately from conditional requirements and resilient/generic witness flags. Consequently `0x48` fails the gate while `0x08` passes. The observed one-bit comparison isolates that exclusion without changing the witness table or implementation. ## Additional context The reviewed related reports cover different failures: #5738 concerns inline Objective-C class names, #5683 external descriptor bindings, #5682 qualified names, #4368 empty-name validation and #6000 kind-3 rebase-to-zero preservation. None reports this flag-based exclusion. No known-good REA version or regression range was established.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4450

No description provided.

Rows
Columns