[GH-ISSUE #2218] [Feature] Support caller-controlled caches for macOS ownership-helper compilation

bsakaguchi commented

2026-10-15 18:27:51 +00:00

Owner

Originally created by @xthunder0 on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2218

Before submitting

I searched open and closed issues and pull requests for related proposals.
I removed credentials, private paths, and sensitive target data.

Area

CLI / macOS process ownership inspection

Current limitation

Please provide a documented, opt-in way to keep the caches used by macOS ownership-helper compilation within a caller-selected writable directory, while preserving the current reusable-cache behavior by default.

The use case is static JavaScript analysis in a restricted environment where writes are allowed only within a dedicated working directory. Setting TMPDIR controls REA's helper root and compiler temporary objects, but does not establish containment of all compiler/tool-lookup caches.

In rea-agents@6.4.0, the packaged DarwinProcessRunTokenReader.js explicitly deletes CLANG_MODULE_CACHE_PATH before invoking xcrun swiftc, and does not pass -module-cache-path. The same behavior remains in main at bfa805d. I did not find a public CLI/environment setting for choosing the helper's module-cache directory in the inspected helper, configuration files, and installation/process-capture documentation. The configuration/documentation review was at 7ae3c50; the newer helper source was checked before submitting.

Observed evidence and limits

The latest limited test used:

npm rea-agents@6.4.0, release commit a56f7bb668ab23fec8d33f7af353ac7ed2fe1895;
Node.js 24.19.0 on macOS;
Apple Swift 6.1.2 and an explicit SDKROOT selecting the installed macOS 15.5 SDK, whose Swift compiler-build string matches;
an authored six-file Electron-like fixture, analyzed without executing the target;
a macOS sandbox denying network access and writes outside the working directory, with TMPDIR inside it.

The invocation below uses placeholders for the original local paths:

SDKROOT=<installed-macos-15.5-sdk> \
node <installed-rea>/scripts/rea.mjs analyze-javascript-application <fixture-directory> \
  --artifact-format directory --max-heap-mb 256 \
  --analysis-timeout-ms 10000 --format json

An earlier attempt had both cache-write denials and a Swift 6.1.2 / macOS 26.2 SDK mismatch. The latest test changed only the REA invocation's SDK selection and retained the same package, fixture, options, and sandbox restrictions.

It exited 1 after approximately 11.28 seconds, reporting execution_failure during source-analysis startup. The diagnostics referenced MacOSX15.5.sdk; the earlier 26.2/Swift 6.2 mismatch diagnostics did not recur. Both lookup-cache and module-cache writes were explicitly denied:

xcrun: error: couldn't create cache file '<system-temp>/xcrun_db-<suffix>' (errno=Operation not permitted)
<unknown>:0: error: unable to open output file '<system-cache>/clang/ModuleCache/<id>/SwiftShims-<id>.pcm': 'Operation not permitted'
<macos-15.5-sdk>/usr/lib/swift/Swift.swiftmodule/<target>.swiftinterface:5:8: error: could not build Objective-C module 'SwiftShims'

Paths and generated identifiers in these excerpts are redacted. The module-cache denial is now directly observed, rather than only inferred from source. No successful full source-analysis result was obtained.

This is one test under enforced write restrictions. There was no unrestricted control run, so it does not establish whether the same fixture succeeds in a normal unrestricted environment. Current main has been inspected, not executed. No global developer settings were changed, no cache-policy workaround or source patch was applied, and the target was not executed.

Desired outcome and success criteria

With a compatible compiler/SDK pair, a caller can use a supported CLI configuration to:

Select an explicit helper module-cache directory and use a documented way to handle xcrun's lookup-cache writes within the permitted boundary.
Run the normal static-analysis path without source patches, compiler shims, or disabling process-ownership checks.
Observe successful analysis of an inert fixture under an enforced write boundary, or an actionable failure identifying an unmet prerequisite.
Retain existing cancellation and helper-temporary-file cleanup behavior. Caller-managed persistent caches should have documented ownership and should not be deleted as incidental helper cleanup.

The default shared/reusable-cache path should remain unchanged. An opt-in private cache may incur a cold-compilation cost; that trade-off should be explicit.

#1139 intentionally adopted Swift's reusable cache to avoid repeated cold compilation. This request adds caller choice rather than reverting that default.
TMPDIR alone does not configure the module cache, and the helper removes CLANG_MODULE_CACHE_PATH. Swift supports a -module-cache-path argument, but this invocation does not forward one.
Apple's xcrun_nocache / --no-cache controls lookup behavior; I have not verified that it eliminates writes. It is not a demonstrated containment solution.
#1993 / #1999 address compiler-failure diagnostics; #1173 / #1194 address cancelled compilation residue. Those outcomes differ from caller-selected cache containment. Increasing the compile timeout would not supply that configuration.

Authority boundaries and non-goals

This request does not ask REA to repair SDK installations, change global developer settings, provide a sandbox, bypass ownership verification, or execute the analyzed application. It is limited to a supported configuration path for the macOS ownership helper. If an existing supported configuration already meets this requirement, documentation of that path would also resolve the request.

Originally created by @xthunder0 on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2218 ### Before submitting - [x] I searched open and closed issues and pull requests for related proposals. - [x] I removed credentials, private paths, and sensitive target data. ### Area CLI / macOS process ownership inspection ### Current limitation Please provide a documented, opt-in way to keep the caches used by macOS ownership-helper compilation within a caller-selected writable directory, while preserving the current reusable-cache behavior by default. The use case is static JavaScript analysis in a restricted environment where writes are allowed only within a dedicated working directory. Setting `TMPDIR` controls REA's helper root and compiler temporary objects, but does not establish containment of all compiler/tool-lookup caches. In `rea-agents@6.4.0`, the packaged `DarwinProcessRunTokenReader.js` explicitly deletes `CLANG_MODULE_CACHE_PATH` before invoking `xcrun swiftc`, and does not pass `-module-cache-path`. The same behavior remains in [main at `bfa805d`](https://github.com/morluto/rea/blob/bfa805decf83744bd3ecb0d1402667cf3ed5c3b2/src/process/DarwinProcessRunTokenReader.ts#L275-L295). I did not find a public CLI/environment setting for choosing the helper's module-cache directory in the inspected helper, configuration files, and installation/process-capture documentation. The configuration/documentation review was at `7ae3c50`; the newer helper source was checked before submitting. ### Observed evidence and limits The latest limited test used: - npm `rea-agents@6.4.0`, release commit `a56f7bb668ab23fec8d33f7af353ac7ed2fe1895`; - Node.js 24.19.0 on macOS; - Apple Swift 6.1.2 and an explicit `SDKROOT` selecting the installed macOS 15.5 SDK, whose Swift compiler-build string matches; - an authored six-file Electron-like fixture, analyzed without executing the target; - a macOS sandbox denying network access and writes outside the working directory, with `TMPDIR` inside it. The invocation below uses placeholders for the original local paths: ```sh SDKROOT=<installed-macos-15.5-sdk> \ node <installed-rea>/scripts/rea.mjs analyze-javascript-application <fixture-directory> \ --artifact-format directory --max-heap-mb 256 \ --analysis-timeout-ms 10000 --format json ``` An earlier attempt had both cache-write denials and a Swift 6.1.2 / macOS 26.2 SDK mismatch. The latest test changed only the REA invocation's SDK selection and retained the same package, fixture, options, and sandbox restrictions. It exited 1 after approximately 11.28 seconds, reporting `execution_failure` during source-analysis startup. The diagnostics referenced `MacOSX15.5.sdk`; the earlier 26.2/Swift 6.2 mismatch diagnostics did not recur. Both lookup-cache and module-cache writes were explicitly denied: ```text xcrun: error: couldn't create cache file '<system-temp>/xcrun_db-<suffix>' (errno=Operation not permitted) <unknown>:0: error: unable to open output file '<system-cache>/clang/ModuleCache/<id>/SwiftShims-<id>.pcm': 'Operation not permitted' <macos-15.5-sdk>/usr/lib/swift/Swift.swiftmodule/<target>.swiftinterface:5:8: error: could not build Objective-C module 'SwiftShims' ``` Paths and generated identifiers in these excerpts are redacted. The module-cache denial is now directly observed, rather than only inferred from source. No successful full source-analysis result was obtained. This is one test under enforced write restrictions. There was no unrestricted control run, so it does not establish whether the same fixture succeeds in a normal unrestricted environment. Current main has been inspected, not executed. No global developer settings were changed, no cache-policy workaround or source patch was applied, and the target was not executed. ### Desired outcome and success criteria With a compatible compiler/SDK pair, a caller can use a supported CLI configuration to: 1. Select an explicit helper module-cache directory and use a documented way to handle xcrun's lookup-cache writes within the permitted boundary. 2. Run the normal static-analysis path without source patches, compiler shims, or disabling process-ownership checks. 3. Observe successful analysis of an inert fixture under an enforced write boundary, or an actionable failure identifying an unmet prerequisite. 4. Retain existing cancellation and helper-temporary-file cleanup behavior. Caller-managed persistent caches should have documented ownership and should not be deleted as incidental helper cleanup. The default shared/reusable-cache path should remain unchanged. An opt-in private cache may incur a cold-compilation cost; that trade-off should be explicit. ### Alternatives and related work - [#1139](https://github.com/morluto/rea/pull/1139) intentionally adopted Swift's reusable cache to avoid repeated cold compilation. This request adds caller choice rather than reverting that default. - `TMPDIR` alone does not configure the module cache, and the helper removes `CLANG_MODULE_CACHE_PATH`. Swift supports a `-module-cache-path` argument, but this invocation does not forward one. - Apple's `xcrun_nocache` / `--no-cache` controls lookup behavior; I have not verified that it eliminates writes. It is not a demonstrated containment solution. - [#1993 / #1999](https://github.com/morluto/rea/pull/1999) address compiler-failure diagnostics; [#1173 / #1194](https://github.com/morluto/rea/pull/1194) address cancelled compilation residue. Those outcomes differ from caller-selected cache containment. Increasing the compile timeout would not supply that configuration. ### Authority boundaries and non-goals This request does not ask REA to repair SDK installations, change global developer settings, provide a sandbox, bypass ownership verification, or execute the analyzed application. It is limited to a supported configuration path for the macOS ownership helper. If an existing supported configuration already meets this requirement, documentation of that path would also resolve the request.

bsakaguchi

2026-10-15 18:27:51 +00:00

closed this issue
added the
area: runtime

enhancement
labels

bsakaguchi commented

2026-10-15 18:27:52 +00:00

Author

Owner

@charan-rathore commented on GitHub (Oct 12, 2026):

Taking this - PR coming today. I will preserve an explicitly supplied CLANG_MODULE_CACHE_PATH for swiftc and keep the default cache behavior when it is unset.

@charan-rathore commented on GitHub (Oct 12, 2026): Taking this - PR coming today. I will preserve an explicitly supplied CLANG_MODULE_CACHE_PATH for swiftc and keep the default cache behavior when it is unset.

bsakaguchi commented

2026-10-15 18:27:52 +00:00

Author

Owner

@charan-rathore commented on GitHub (Oct 12, 2026):

Refining the approach after a closer read of the existing tests: the PR will add an explicit REA_DARWIN_MODULE_CACHE_PATH opt-in (mapped to the compiler's CLANG_MODULE_CACHE_PATH) rather than honoring the ambient variable, so environments without the opt-in keep today's exact behavior, including the fallback for unusable ambient cache paths.

@charan-rathore commented on GitHub (Oct 12, 2026): Refining the approach after a closer read of the existing tests: the PR will add an explicit REA_DARWIN_MODULE_CACHE_PATH opt-in (mapped to the compiler's CLANG_MODULE_CACHE_PATH) rather than honoring the ambient variable, so environments without the opt-in keep today's exact behavior, including the fallback for unusable ambient cache paths.

bsakaguchi commented

2026-10-15 18:27:53 +00:00

Author

Owner

@morluto commented on GitHub (Oct 12, 2026):

Consolidation update: the earlier #2252 now folds #2260 with coauthor credit and covers the lookup-cache gap too. REA_DARWIN_MODULE_CACHE_PATH selects a persistent caller-owned Swift/Clang cache; REA_DARWIN_SWIFTC_PATH with a compatible absolute SDKROOT calls the installed compiler directly and bypasses xcrun lookup. xcrun --no-cache is not claimed as write-free. Missing direct prerequisites identify the selected setting; default shared-cache behavior and ownership checks remain.

Real public static JavaScript CLI analysis succeeded with a fresh private cache, network denied and writes restricted to the dedicated directory plus /dev/null. The policy explicitly excepts the read-only setuid /bin/ps process inspector. An outside-write probe was denied. Cold/warm runs took 2.39/1.24 s, CPU user+system 2.22/1.28 s; time reported maximum RSS 318,701,568/305,774,592 bytes. Each retained 53 cache entries and left no temporary entries. Peak simultaneous process-family memory and task token usage were unmeasured; equal 43,322-byte results do not prove token savings. No default-path performance comparison is claimed.

Without the process-inspection exception the blanket policy rejected ps with spawn EPERM and reported cleanup_incomplete, preserving ownership uncertainty. This is documented as a separate prerequisite, not a universal containment guarantee. All 29 ownership boundary tests (including real direct compiler, cancellation and cleanup), full source checks and docs checks pass. Keeping the issue open until #2252 merges; #2260 is consolidated there.

@morluto commented on GitHub (Oct 12, 2026): Consolidation update: the earlier #2252 now folds #2260 with coauthor credit and covers the lookup-cache gap too. REA_DARWIN_MODULE_CACHE_PATH selects a persistent caller-owned Swift/Clang cache; REA_DARWIN_SWIFTC_PATH with a compatible absolute SDKROOT calls the installed compiler directly and bypasses xcrun lookup. xcrun --no-cache is not claimed as write-free. Missing direct prerequisites identify the selected setting; default shared-cache behavior and ownership checks remain. Real public static JavaScript CLI analysis succeeded with a fresh private cache, network denied and writes restricted to the dedicated directory plus /dev/null. The policy explicitly excepts the read-only setuid /bin/ps process inspector. An outside-write probe was denied. Cold/warm runs took 2.39/1.24 s, CPU user+system 2.22/1.28 s; time reported maximum RSS 318,701,568/305,774,592 bytes. Each retained 53 cache entries and left no temporary entries. Peak simultaneous process-family memory and task token usage were unmeasured; equal 43,322-byte results do not prove token savings. No default-path performance comparison is claimed. Without the process-inspection exception the blanket policy rejected ps with spawn EPERM and reported cleanup_incomplete, preserving ownership uncertainty. This is documented as a separate prerequisite, not a universal containment guarantee. All 29 ownership boundary tests (including real direct compiler, cancellation and cleanup), full source checks and docs checks pass. Keeping the issue open until #2252 merges; #2260 is consolidated there.

bsakaguchi commented

2026-10-15 18:27:53 +00:00

Author

Owner

@morluto commented on GitHub (Oct 12, 2026):

The negative-policy lifecycle observation is now tracked separately in https://github.com/morluto/rea/issues/2290: denied process inspection can emit cleanup_incomplete while leaving the CLI alive. It is not attributed to the cache patch without a base comparison. #2252 documents process inspection as a separate prerequisite; the supported-path cold/warm control passes and cleans temporary files.

@morluto commented on GitHub (Oct 12, 2026): The negative-policy lifecycle observation is now tracked separately in https://github.com/morluto/rea/issues/2290: denied process inspection can emit cleanup_incomplete while leaving the CLI alive. It is not attributed to the cache patch without a base comparison. #2252 documents process inspection as a separate prerequisite; the supported-path cold/warm control passes and cleans temporary files.

Rows
Columns

[GH-ISSUE #2218] [Feature] Support caller-controlled caches for macOS ownership-helper compilation #740

Before submitting

Area

Current limitation

Observed evidence and limits

Desired outcome and success criteria

Alternatives and related work

Authority boundaries and non-goals