[GH-ISSUE #4307] [Feature] Inspect the dyld shared cache and resolve system library loads #1711

New issue

Open

opened 2026-10-16 21:09:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:09:19 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4307

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Native analysis or decompilation

Current limitation

Since macOS 11, system libraries exist only inside the dyld shared cache, so trace_dylib_resolution cannot evaluate any /usr/lib or /System load and REA cannot look inside the cache. Add a read-only way to inventory a caller-selected cache file set and its images, and let trace_dylib_resolution resolve absolute system candidates against that cache. A complete implementation (PR #947) was closed unmerged by its author without a stated reason. This request asks the maintainers to decide whether they want this direction before anyone re-implements it.

An analyst tracing an app's dependencies cannot tell whether a system dependency exists on a given OS, which image would satisfy an @rpath that lists /usr/lib/swift first, or what a cached library itself links. Measured with REA 6.5.0 (npx -y rea-agents@6.5.0, tag rea-agents-6.5.0, commit 3b79af28e80785afce9f5998d8f03289402a1847) on macOS 27.0.1 (26A434) arm64 with Xcode 27.0 (27A266a) and Node.js v26.10.0:

For a fixture built with printf 'int main(void){return 0;}\n' > m.c && clang -o plain m.c, rea trace-dylib-resolution ./plain --json reports the /usr/lib/libSystem.B.dylib edge with one candidate, outcome outside-target, and resolution undetermined, while coverage.status is complete.
On the same host, /usr/lib/libSystem.B.dylib does not exist as a file. dyld_info -linked_dylibs /usr/lib/libSystem.B.dylib reads it from the cache and lists its re-exported /usr/lib/system/* libraries, and dyld_info -uuid prints its UUID. The cache itself is the file set dyld_shared_cache_arm64e plus subcaches, .map and .atlas files (82 files) under /System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/.

The release documents this as a fixed limitation: "Absolute install names and rpaths are outside the analyzed root and are not evaluated, including /System and /usr/lib libraries that the dyld shared cache usually provides." (dylibResolutionFindings.ts, line 92) and "On macOS 11 and later most of them live in the dyld shared cache rather than on disk, so REA does not check them against the host." (docs/apple-application-analysis.md). Neither the release nor main at d52a488 contains shared-cache code, and the cited source files are unchanged there; searches of src and docs for dyld_shared_cache, subcache and DyldSharedCache return nothing.

Use cases

No response

Desired outcome and success criteria

Inventory a caller-selected cache file set (the main cache file plus its subcaches) without an open target and without launching a process. Report the cache UUID, architecture, platform and OS version, its mappings, each expected subcache as present, missing or uuid-mismatch, and the complete image list with install names and addresses. Anything other than present makes coverage partial and names the affected subcache.
For requested images, read their load commands through the cache mappings and return them with the same dependency fields trace_dylib_resolution already reports (install name, weak, re-export, upward, delayed-init, versions), each with a status of parsed, absent, unmapped, malformed or unsupported.
trace_dylib_resolution accepts an optional cache. An absolute candidate found in a cache that serves the process's architecture family resolves as a shared-cache load and stops the search, as dyld does. The result names the cache path, UUID, architecture and OS version used. Without a cache, or for a process of another architecture family, results stay undetermined with a limitation, exactly as today.
Results bind to the files that were read, by digest and cache UUID, not to "the running OS". Since macOS 26.1 a Background Security Improvement can replace cryptex content between OS updates, and the measurement host's cache lives on the preboot volume's cryptex path.
Read-only. The format is undocumented on developer.apple.com, so unknown header generations, mappings or image entries fail closed with a typed format error or a counted partial result, never a silent omission.

Acceptance criteria

Inventory of the host cache: the image count and every install name equal the set listed by dyld_info -all_dyld_cache -linked_dylibs, and for every image the dependencies and their re-export, weak-link, upward and delay-init attributes equal that output. PR #947 reports this comparison for 3,649 images on macOS 26.6.
Requesting /usr/lib/libSystem.B.dylib returns the UUID printed by dyld_info -uuid /usr/lib/libSystem.B.dylib and the dependency list printed by dyld_info -linked_dylibs.
trace_dylib_resolution on the plain fixture with the host cache reports the libSystem edge as a shared-cache load naming the cache UUID. Without the cache the result is unchanged from the release build.
An x86_64 fixture (clang -arch x86_64 -o plain_x86 m.c) traced against the arm64e host cache keeps undetermined with a limitation that names the architecture-family mismatch.
A copy of the cache file set with one subcache removed reports that subcache missing and partial coverage. A non-cache file returns a typed format error.
CLI and MCP results match. No process is launched and no file is written.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Affected area

A new or extended Apple artifact operation for cache inventory, trace_dylib_resolution input and output schemas, the dylib-resolution limitation list and docs/apple-application-analysis.md.

Non-goals

Extracting images or dylibs from the cache, local symbols, slide info, or analyzing cached images with Hopper, Ghidra or IDA.
Verifying cache code signatures, cryptex seals or trust caches.
Finding or downloading caches for other OS versions or devices, such as from IPSW files. Callers supply the files.
Modeling DYLD_* overrides, library validation or other runtime conditions beyond the existing trace_dylib_resolution limitations.

Verification available so far

I can suggest a sample or manual check

Verification details (optional)

No response

Alternatives considered

dyld_info on the host answers per-path questions, but it reads only the running system's cache, produces text outside Evidence, and is not joined to a trace.
Leaving system edges undetermined is accurate but makes every system dependency, and every @rpath that lists a system path first, unanswerable.

Possible direction

No response

References

Prior art. PR #947, "feat(apple): inspect dyld shared caches and resolve system paths" (head feat/dyld-shared-cache, 3,425 additions), added inspect_dyld_shared_cache and trace_dylib_resolution --shared-cache. Its body reports a real-lane check of 3,649 host-cache images against dyld_info -all_dyld_cache -linked_dylibs. #947 was closed unmerged by its author. Follow-up fix PR #1090 ("admit only compatible dyld subcache mappings") was merged into #947's branch, not main. Merged PR #1100 says shared-cache inventory and system-path resolution "remain outside this change".
Question for maintainers: is shared-cache inventory and system-path resolution a direction you want in REA? If so, should #947's branch (with #1090) be revived, or should a new implementation follow constraints you name first, such as tool shape, whether host-only caches are enough, or test-lane requirements? If not, documenting the decision would stop repeat proposals.
macOS Big Sur 11.0.1 Release Notes: "the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail."
Background Security Improvements (Apple Platform Security): "supported and enabled for future releases starting with iOS 26.1, iPadOS 26.1, and macOS 26.1" for "the Safari browser, WebKit framework stack, and other system libraries", and "Content that can be patched using the Background Security Improvement mechanism has been moved into cryptexes, which are optimized, cryptographically sealed disk images that reside on the preboot volume". Background Security Improvements "ship both a patch and an antipatch", so the cache an analyst reads can change between OS updates.
The Apple documentation corpus checked for this draft has no file-format specification for the shared cache. PR #947 read header fields "by the offsets in Apple's open-source dyld".
Apple .ips crash reports mark images loaded from the cache with source S or C, so cache inventory would also let crash frames in system libraries be joined by UUID (see the companion .ips request).
GitHub searches for "dyld shared cache" and "shared cache" found only #947, #1090, #1100 and #927 (the original trace_dylib_resolution PR); none is open.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4307 ### Before submitting - [X] I searched the open and closed issues and pull requests for a related proposal. - [X] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [X] This request does not include credentials, private samples, or sensitive target data. ### Area Native analysis or decompilation ### Current limitation Since macOS 11, system libraries exist only inside the dyld shared cache, so `trace_dylib_resolution` cannot evaluate any `/usr/lib` or `/System` load and REA cannot look inside the cache. Add a read-only way to inventory a caller-selected cache file set and its images, and let `trace_dylib_resolution` resolve absolute system candidates against that cache. A complete implementation (PR #947) was closed unmerged by its author without a stated reason. This request asks the maintainers to decide whether they want this direction before anyone re-implements it. An analyst tracing an app's dependencies cannot tell whether a system dependency exists on a given OS, which image would satisfy an `@rpath` that lists `/usr/lib/swift` first, or what a cached library itself links. Measured with REA 6.5.0 (`npx -y rea-agents@6.5.0`, tag `rea-agents-6.5.0`, commit `3b79af28e80785afce9f5998d8f03289402a1847`) on macOS 27.0.1 (26A434) arm64 with Xcode 27.0 (27A266a) and Node.js v26.10.0: - For a fixture built with `printf 'int main(void){return 0;}\n' > m.c && clang -o plain m.c`, `rea trace-dylib-resolution ./plain --json` reports the `/usr/lib/libSystem.B.dylib` edge with one candidate, outcome `outside-target`, and resolution `undetermined`, while `coverage.status` is `complete`. - On the same host, `/usr/lib/libSystem.B.dylib` does not exist as a file. `dyld_info -linked_dylibs /usr/lib/libSystem.B.dylib` reads it from the cache and lists its re-exported `/usr/lib/system/*` libraries, and `dyld_info -uuid` prints its UUID. The cache itself is the file set `dyld_shared_cache_arm64e` plus subcaches, `.map` and `.atlas` files (82 files) under `/System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/`. The release documents this as a fixed limitation: "Absolute install names and rpaths are outside the analyzed root and are not evaluated, including /System and /usr/lib libraries that the dyld shared cache usually provides." ([dylibResolutionFindings.ts](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/src/domain/apple/dylibResolutionFindings.ts#L89-L98), line 92) and "On macOS 11 and later most of them live in the dyld shared cache rather than on disk, so REA does not check them against the host." ([docs/apple-application-analysis.md](https://github.com/morluto/rea/blob/3b79af28e80785afce9f5998d8f03289402a1847/docs/apple-application-analysis.md#L200-L202)). Neither the release nor main at `d52a488` contains shared-cache code, and the cited source files are unchanged there; searches of `src` and `docs` for `dyld_shared_cache`, `subcache` and `DyldSharedCache` return nothing. ### Use cases _No response_ ### Desired outcome and success criteria - Inventory a caller-selected cache file set (the main cache file plus its subcaches) without an open target and without launching a process. Report the cache UUID, architecture, platform and OS version, its mappings, each expected subcache as `present`, `missing` or `uuid-mismatch`, and the complete image list with install names and addresses. Anything other than `present` makes coverage `partial` and names the affected subcache. - For requested images, read their load commands through the cache mappings and return them with the same dependency fields `trace_dylib_resolution` already reports (install name, weak, re-export, upward, delayed-init, versions), each with a status of `parsed`, `absent`, `unmapped`, `malformed` or `unsupported`. - `trace_dylib_resolution` accepts an optional cache. An absolute candidate found in a cache that serves the process's architecture family resolves as a shared-cache load and stops the search, as dyld does. The result names the cache path, UUID, architecture and OS version used. Without a cache, or for a process of another architecture family, results stay `undetermined` with a limitation, exactly as today. - Results bind to the files that were read, by digest and cache UUID, not to "the running OS". Since macOS 26.1 a Background Security Improvement can replace cryptex content between OS updates, and the measurement host's cache lives on the preboot volume's cryptex path. - Read-only. The format is undocumented on developer.apple.com, so unknown header generations, mappings or image entries fail closed with a typed format error or a counted partial result, never a silent omission. **Acceptance criteria** - Inventory of the host cache: the image count and every install name equal the set listed by `dyld_info -all_dyld_cache -linked_dylibs`, and for every image the dependencies and their `re-export`, `weak-link`, `upward` and `delay-init` attributes equal that output. PR #947 reports this comparison for 3,649 images on macOS 26.6. - Requesting `/usr/lib/libSystem.B.dylib` returns the UUID printed by `dyld_info -uuid /usr/lib/libSystem.B.dylib` and the dependency list printed by `dyld_info -linked_dylibs`. - `trace_dylib_resolution` on the `plain` fixture with the host cache reports the libSystem edge as a shared-cache load naming the cache UUID. Without the cache the result is unchanged from the release build. - An x86_64 fixture (`clang -arch x86_64 -o plain_x86 m.c`) traced against the arm64e host cache keeps `undetermined` with a limitation that names the architecture-family mismatch. - A copy of the cache file set with one subcache removed reports that subcache `missing` and partial coverage. A non-cache file returns a typed format error. - CLI and MCP results match. No process is launched and no file is written. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) **Affected area** A new or extended Apple artifact operation for cache inventory, `trace_dylib_resolution` input and output schemas, the dylib-resolution limitation list and `docs/apple-application-analysis.md`. **Non-goals** - Extracting images or dylibs from the cache, local symbols, slide info, or analyzing cached images with Hopper, Ghidra or IDA. - Verifying cache code signatures, cryptex seals or trust caches. - Finding or downloading caches for other OS versions or devices, such as from IPSW files. Callers supply the files. - Modeling `DYLD_*` overrides, library validation or other runtime conditions beyond the existing `trace_dylib_resolution` limitations. ### Verification available so far I can suggest a sample or manual check ### Verification details (optional) _No response_ ### Alternatives considered - `dyld_info` on the host answers per-path questions, but it reads only the running system's cache, produces text outside Evidence, and is not joined to a trace. - Leaving system edges `undetermined` is accurate but makes every system dependency, and every `@rpath` that lists a system path first, unanswerable. ### Possible direction _No response_ ### References - Prior art. PR #947, "feat(apple): inspect dyld shared caches and resolve system paths" (head `feat/dyld-shared-cache`, 3,425 additions), added `inspect_dyld_shared_cache` and `trace_dylib_resolution --shared-cache`. Its body reports a real-lane check of 3,649 host-cache images against `dyld_info -all_dyld_cache -linked_dylibs`. #947 was closed unmerged by its author. Follow-up fix PR #1090 ("admit only compatible dyld subcache mappings") was merged into #947's branch, not main. Merged PR #1100 says shared-cache inventory and system-path resolution "remain outside this change". - Question for maintainers: is shared-cache inventory and system-path resolution a direction you want in REA? If so, should #947's branch (with #1090) be revived, or should a new implementation follow constraints you name first, such as tool shape, whether host-only caches are enough, or test-lane requirements? If not, documenting the decision would stop repeat proposals. - [macOS Big Sur 11.0.1 Release Notes](https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes): "the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail." - [Background Security Improvements](https://support.apple.com/guide/security/background-security-improvements-sec87fc038c2/1/web/1) (Apple Platform Security): "supported and enabled for future releases starting with iOS 26.1, iPadOS 26.1, and macOS 26.1" for "the Safari browser, WebKit framework stack, and other system libraries", and "Content that can be patched using the Background Security Improvement mechanism has been moved into cryptexes, which are optimized, cryptographically sealed disk images that reside on the preboot volume". Background Security Improvements "ship both a patch and an antipatch", so the cache an analyst reads can change between OS updates. - The Apple documentation corpus checked for this draft has no file-format specification for the shared cache. PR #947 read header fields "by the offsets in Apple's open-source dyld". - Apple `.ips` crash reports mark images loaded from the cache with `source` `S` or `C`, so cache inventory would also let crash frames in system libraries be joined by UUID (see the companion `.ips` request). - GitHub searches for "dyld shared cache" and "shared cache" found only #947, #1090, #1100 and #927 (the original `trace_dylib_resolution` PR); none is open.

bsakaguchi referenced this issue

2026-10-31 00:41:46 +00:00

[GH-ISSUE #5856] [Bug] Interactive CLI-only skill selection is dropped when only Claude Code is detected #3550

bsakaguchi referenced this issue

2026-10-31 00:55:07 +00:00

[PR #1744] [CLOSED] fix(setup): verify independently selected skill destinations #4794