[PR #5384] [CLOSED] fix(verify): compare dsh lane evidence against the resolved target path #6914

Closed
opened 2026-11-01 21:44:01 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5384
Author: @kittimzhe
Created: 10/20/2026
Status: ❌ Closed

Base: main ← Head: fix/verify-client-macos-realpath


📝 Commits (1)

  • dffcb78 fix(verify): compare dsh lane evidence against the resolved target path

📊 Changes

1 file changed (+13 additions, -2 deletions)

View changed files

📝 scripts/verify-deepseek-client.mjs (+13 -2)

📄 Description

Summary

npm run verify:deepseek-client fails on macOS at the Evidence identity assertion: the lane asserts evidence.subject.local_path === target, but the analyzer resolves its input through realpath before recording Evidence (resolveSelectedInput in src/application/javascript/JavaScriptArtifactReconstruction.ts), and the macOS tmpdir() prefix is a symlink (/var/folders/... → /private/var/folders/...). The lane completes the entire lifecycle — setup, idempotent setup, doctor, headless chat, skill load, full tool catalog, real analyze_javascript_application execution — and then dies comparing two spellings of the same path. The lane's expectation was the only wrong side; the analyzer's resolved-path recording is correct and enforced upstream by assertApplicationSubject. The assertion now compares against the resolved path.

Problem and expected behavior

  • Smallest trigger: run the lane on macOS with a dsh CLI that supports --profile headless --json (verified with 0.2.1-rc.2).
  • Violated invariant: a verification lane must verify what the client actually records, not a spelling the caller used. Evidence records the resolved identity of what was analyzed; the lane compared it to the unresolved caller-facing path.
  • Expected: the lane passes on macOS. The same failure was reported in the #4402 discussion (unmodified main, macOS): "The lane still ends passed: false on a macOS-only /private/var versus /var mismatch in its local_path assertion". No dedicated issue exists; that comment is the closest report.

Change and scope

  • const resolvedTarget = await realpath(target) computed once after the fixture directories exist; the assertion compares against it.
  • The caller-facing input_path sent to the tool and the prompt text keep the original unresolved path, so the lane still covers the symlink-resolution step end to end.
  • The same pattern (assert.equal(evidence.subject.local_path, target) over a mkdtemp(tmpdir(), ...) target) exists in the other client lanes (claude, gemini, grok, copilot, qwen, omp, opencode, commandcode, native-agent). Only the deepseek lane is executable on this machine (the only client CLI installed here), so this PR fixes the lane it can verify and leaves the identical mechanical change in the others to maintainer direction.
  • Contribution scope: verification tooling fix within existing boundaries — no maintainer scope decision required ("not applicable").

Contract and boundary impact

  • Semantic owner and earliest changed stage: the verification lane's expectation in scripts/; no runtime code touched.
  • CLI and MCP/tool-catalog contract: none.
  • Provider, bridge, target-format, or platform compatibility: none.
  • Evidence, artifact, provenance contract: none — the analyzer's resolved-path recording is unchanged.
  • Generated metadata: none.

Expected effects:

  • Read existing files or captured data. (the lane reads its own lab fixtures; its process launches — local model fixture, dsh headless — are pre-existing behavior unchanged by this PR)
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • RED (before, same revision): lifecycle reached ["dry-run","setup","idempotent-setup","doctor"], the real analysis completed (analyze_javascript_application, ~1.1 s, status ok), Evidence was recorded, and the lane failed with AssertionError: '+ /private/var/folders/.../fixture α测试' - '/var/folders/.../fixture α测试' — the exact mismatch reported in the #4402 discussion.
  • GREEN (after): lifecycle ["dry-run","setup","idempotent-setup","doctor","native-tool-invocation","uninstall"], receipt passed: true, exit 0.
  • Tests: the lane is the verification artifact itself; scripts/ has no colocated unit tests to update.
  • Remaining proof gaps: the other nine client lanes were not executed (no CLIs installed on this machine); Windows is out of scope (the lane asserts non-win32).

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable. (RED/GREEN runs are executed; the other-lane pattern is source-derived and labeled as such)
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm ci — locked dependencies, Node 24.18.0 (.nvmrc), macOS 14 arm64.
  • npm run verify:deepseek-client (before the fix) — fails at the Evidence local_path assertion after completing setup, idempotent setup, doctor, headless chat, skill load, catalog validation, and the real analysis.
  • npm run verify:deepseek-client (after the fix) — passes: full lifecycle through native-tool-invocation and uninstall, receipt passed: true.
  • npm run check:fast — typecheck + lint pass (uncached).
  • Environment notes for other macOS contributors: the lane needs (1) a dsh CLI with headless JSON mode (REA_VERIFY_DEEPSEEK_COMMAND can point at a local install; the 0.1.5-rc.3 on this machine predates --profile headless --json), and (2) a working Swift toolchain for the Darwin ownership helper — stock CLT 16.2 on macOS 14 ships duplicate SwiftBridging definitions in usr/include/swift/module.modulemap and bridging.modulemap that break every import Foundation compile; removing the duplicate block from module.modulemap restores it. Both were local environment fixes, stated here because they are undeclared prerequisites of the lane.

Compatibility, safety, and release

  • Breaking changes or migration steps: none — verification script only.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable; the lane's local model fixture is unchanged.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: none.

Author readiness

Human owner: @kittimzhe

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above. (no dedicated issue; the failure report in the #4402 discussion is cited in Problem)
  • Tests cover changed observable behavior and meaningful failure paths. (the lane itself is the executable check; RED/GREEN receipts recorded above)
  • Owning docs, contracts, and generated metadata are updated where needed. (no docs describe the lane's assertion internals)
  • User-visible CLI/MCP changes include representative output. (verification tooling only; the RED/GREEN receipts are included above)
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5384 **Author:** [@kittimzhe](https://github.com/kittimzhe) **Created:** 10/20/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/verify-client-macos-realpath` --- ### 📝 Commits (1) - [`dffcb78`](https://github.com/morluto/rea/commit/dffcb788b6a745729dc60030d9648f2d10f76891) fix(verify): compare dsh lane evidence against the resolved target path ### 📊 Changes **1 file changed** (+13 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `scripts/verify-deepseek-client.mjs` (+13 -2) </details> ### 📄 Description ## Summary `npm run verify:deepseek-client` fails on macOS at the Evidence identity assertion: the lane asserts `evidence.subject.local_path === target`, but the analyzer resolves its input through `realpath` before recording Evidence (`resolveSelectedInput` in `src/application/javascript/JavaScriptArtifactReconstruction.ts`), and the macOS `tmpdir()` prefix is a symlink (`/var/folders/...` → `/private/var/folders/...`). The lane completes the entire lifecycle — setup, idempotent setup, doctor, headless chat, skill load, full tool catalog, real `analyze_javascript_application` execution — and then dies comparing two spellings of the same path. The lane's expectation was the only wrong side; the analyzer's resolved-path recording is correct and enforced upstream by `assertApplicationSubject`. The assertion now compares against the resolved path. ## Problem and expected behavior - Smallest trigger: run the lane on macOS with a dsh CLI that supports `--profile headless --json` (verified with 0.2.1-rc.2). - Violated invariant: a verification lane must verify what the client actually records, not a spelling the caller used. Evidence records the resolved identity of what was analyzed; the lane compared it to the unresolved caller-facing path. - Expected: the lane passes on macOS. The same failure was reported in the #4402 discussion (unmodified main, macOS): "The lane still ends passed: false on a macOS-only /private/var versus /var mismatch in its local_path assertion". No dedicated issue exists; that comment is the closest report. ## Change and scope - `const resolvedTarget = await realpath(target)` computed once after the fixture directories exist; the assertion compares against it. - The caller-facing `input_path` sent to the tool and the prompt text keep the original unresolved path, so the lane still covers the symlink-resolution step end to end. - The same pattern (`assert.equal(evidence.subject.local_path, target)` over a `mkdtemp(tmpdir(), ...)` target) exists in the other client lanes (claude, gemini, grok, copilot, qwen, omp, opencode, commandcode, native-agent). Only the deepseek lane is executable on this machine (the only client CLI installed here), so this PR fixes the lane it can verify and leaves the identical mechanical change in the others to maintainer direction. - Contribution scope: verification tooling fix within existing boundaries — no maintainer scope decision required ("not applicable"). ## Contract and boundary impact - Semantic owner and earliest changed stage: the verification lane's expectation in `scripts/`; no runtime code touched. - CLI and MCP/tool-catalog contract: none. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance contract: none — the analyzer's resolved-path recording is unchanged. - Generated metadata: none. Expected effects: - [x] Read existing files or captured data. *(the lane reads its own lab fixtures; its process launches — local model fixture, dsh headless — are pre-existing behavior unchanged by this PR)* - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - RED (before, same revision): lifecycle reached `["dry-run","setup","idempotent-setup","doctor"]`, the real analysis completed (`analyze_javascript_application`, ~1.1 s, status ok), Evidence was recorded, and the lane failed with `AssertionError: '+ /private/var/folders/.../fixture α测试' - '/var/folders/.../fixture α测试'` — the exact mismatch reported in the #4402 discussion. - GREEN (after): lifecycle `["dry-run","setup","idempotent-setup","doctor","native-tool-invocation","uninstall"]`, receipt `passed: true`, exit 0. - Tests: the lane is the verification artifact itself; `scripts/` has no colocated unit tests to update. - Remaining proof gaps: the other nine client lanes were not executed (no CLIs installed on this machine); Windows is out of scope (the lane asserts non-win32). For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. *(RED/GREEN runs are executed; the other-lane pattern is source-derived and labeled as such)* - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npm ci` — locked dependencies, Node 24.18.0 (`.nvmrc`), macOS 14 arm64. - `npm run verify:deepseek-client` (before the fix) — fails at the Evidence `local_path` assertion after completing setup, idempotent setup, doctor, headless chat, skill load, catalog validation, and the real analysis. - `npm run verify:deepseek-client` (after the fix) — passes: full lifecycle through `native-tool-invocation` and `uninstall`, receipt `passed: true`. - `npm run check:fast` — typecheck + lint pass (uncached). - Environment notes for other macOS contributors: the lane needs (1) a dsh CLI with headless JSON mode (`REA_VERIFY_DEEPSEEK_COMMAND` can point at a local install; the 0.1.5-rc.3 on this machine predates `--profile headless --json`), and (2) a working Swift toolchain for the Darwin ownership helper — stock CLT 16.2 on macOS 14 ships duplicate `SwiftBridging` definitions in `usr/include/swift/module.modulemap` and `bridging.modulemap` that break every `import Foundation` compile; removing the duplicate block from `module.modulemap` restores it. Both were local environment fixes, stated here because they are undeclared prerequisites of the lane. ## Compatibility, safety, and release - Breaking changes or migration steps: none — verification script only. - Real Hopper/Ghidra, browser, or OS coverage: not applicable; the lane's local model fixture is unchanged. - Package or release metadata impact: none. - Security, privacy, process, or containment review: none. ## Author readiness Human owner: @kittimzhe - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. *(no dedicated issue; the failure report in the #4402 discussion is cited in Problem)* - [x] Tests cover changed observable behavior and meaningful failure paths. *(the lane itself is the executable check; RED/GREEN receipts recorded above)* - [x] Owning docs, contracts, and generated metadata are updated where needed. *(no docs describe the lane's assertion internals)* - [ ] User-visible CLI/MCP changes include representative output. *(verification tooling only; the RED/GREEN receipts are included above)* - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:44:01 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#6914
No description provided.