[PR #3275] test(setup): sweep supported client lifecycle symmetry #5692

Open
opened 2026-10-31 01:08:42 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3275
Author: @IvenKooLab
Created: 10/14/2026
Status: 🔄 Open

Base: main ← Head: test/client-lifecycle-sweep


📝 Commits (10+)

  • 585f11c test(setup): sweep supported client lifecycle symmetry
  • 3e7e2ea test(setup): assert uninstall through post-run doctor state
  • 92dcbf9 test(setup): refresh the isolated client lifecycle sweep
  • 9c59b2b test(setup): isolate Gemini settings and retain failed client paths
  • e80f1c7 Merge remote-tracking branch 'upstream/main' into test/client-lifecycle-sweep
  • 2a2e9ad test(setup): forward the host SDK selection into the lifecycle sweep
  • 1c61dd8 Merge branch 'main' into test/client-lifecycle-sweep
  • b617a4c Merge branch 'main' into test/client-lifecycle-sweep
  • 3a03ef1 test(setup): preserve the selected Darwin compiler in lifecycle sweep
  • acd6497 Merge remote-tracking branch 'upstream/main' into test/client-lifecycle-sweep

📊 Changes

1 file changed (+122 additions, -0 deletions)

View changed files

➕ tests/acceptance/setup/supportedClientLifecycle.test.ts (+122 -0)

📄 Description

Summary

Add a compiled-CLI lifecycle test for every configurable client in the current platform catalog. Selected-profile tests cover only part of that catalog.

Problem and expected behavior

Each supported client should register correctly, survive repeated setup without changes, and lose its REA registration on uninstall. Uninstall must remove REA's skill manifest while preserving user-owned files.

This follows the maintainer's request; there is no separate issue.

Change and scope

The test registers cases from SUPPORTED_CLIENT_DEFINITIONS, then resolves paths inside each case. A path error fails that case instead of removing it from the sweep. It uses the existing cliTest fixture and its worker-scoped compile cache. Each client gets a separate temporary home so shared configuration paths cannot hide another client's failure. Gemini's system settings and defaults also point inside that home.

For each client, it runs setup, scoped doctor, repeated setup, uninstall, and scoped doctor again. It checks registration and skill alignment, unchanged configuration bytes, no repeated setup actions, and an exact missing registration state after uninstall. User-owned notes must survive.

The PR adds one acceptance-test file. Existing malformed-input tests supply parser coverage. This fits the existing contribution scope.

Contract and boundary impact

  • Semantic owner and earliest changed stage: setup acceptance tests; no production behavior changes.
  • Contribution scope fit and linked maintainer decision: existing lifecycle coverage, requested in the comment above. No new capability or authority.
  • CLI and MCP/tool-catalog contract: exercises existing setup, doctor, and uninstall commands; no contract changes.
  • Provider, bridge, target-format, or platform compatibility: uses the current platform's client catalog; no provider changes.
  • Evidence, artifact, provenance, or reconstruction contract: no contract changes; executed results and measurement limits are below.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): isolated temporary homes; verifies REA cleanup and preservation of user-owned notes.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none; no generated files committed.

Expected effects during tests:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

These effects are CLI subprocesses and test code reading and writing disposable configuration files. No instrumentation is injected and no process memory or registers are changed.

Evidence and regression coverage

  • Tests added or updated: 24 client workflows passed on macOS arm64, with five CLI calls per client.
  • Base reproduction or other evidence: a temporary invalid OMP_PROFILE made the previous sweep omit OMP (23 cases, targeted run skipped with exit 0). The updated sweep retained all 24 cases and failed OMP with the producer's path error. The fault was removed before the passing run. An earlier Hermes fault check restored its registration after uninstall and failed on aligned versus expected missing; that fault was also removed.
  • User-visible CLI/MCP output (if applicable): no output changes. Assertions check setup ready, uninstall complete, doctor registration aligned then missing, and exit codes 0 then 1.
  • Remaining proof gaps: Windows/Linux, actual client applications, model backends, and Hopper/Ghidra were not exercised.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

October 20 verification at 3a03ef1e5 (macOS arm64, pinned Node 24.18.0/npm 11.16.0, fresh locked npm ci):

  • npm run test:focused -- tests/acceptance/setup/supportedClientLifecycle.test.ts — 24/24 passed, 206.02 seconds, with command-scoped SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX26.5.sdk and REA_DARWIN_SWIFTC_PATH=/Library/Developer/CommandLineTools/usr/bin/swiftc.
  • An additional filtered compiled Codex lifecycle case with a local 120-second case deadline — 1 passed, 23 filtered/skipped; this is extra toolchain evidence, not a second complete sweep.
  • npm run check, git diff --check, formatting and pre-commit/pre-push hooks — passed.
  • Both caller-selected toolchain values now reach each child CLI. No compiler path or SDK is hardcoded in the test. The current registration-absence assertion remains a present doctor row with state: "missing", separate from skill removal and user-note retention.
  • No actual third-party client execution or additional Windows/Linux local host run is claimed. New-head GitHub CI is pending; the previous head's required CI and author declaration passed.

Node 24.18.0, npm 11.16.0, locked dependencies, macOS arm64:

  • npm run build:cached — passed.
  • npm run test:focused -- tests/acceptance/setup/supportedClientLifecycle.test.ts tests/acceptance/setup/geminiWorkspaceTrust.test.ts — 29 passed (24 lifecycle cases and 5 Gemini workspace/trust cases).
  • Earlier validation on the same production runtime: npm run test:focused -- tests/boundary/cli/clientProfileLifecycle.test.ts tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts — 18 passed.
  • npm run check — passed.
  • npm run verify:test-discovery — all 1,327 test files assigned exactly once.
  • git diff --check — passed.

Before this review follow-up, one fresh-worker measurement per version took 100.72 s before and 96.07 s after (updated sweep at 92dcbf9a7). Child-tree CPU was 142.56/136.53 s; sampled aggregate peak RSS was 654176/660384 KiB. Both ran 120 CLI calls (source-derived). Build/install were excluded, OS caches were not reset, and RSS was sampled every 100 ms with shared mappings included. Separate cold/warm costs and model tokens were not measured. These runs do not establish a reliable performance improvement. Performance was not remeasured for this isolation/catalog follow-up.

Compatibility, safety, and release

  • Breaking changes or migration steps: none; tests only.
  • Real Hopper/Ghidra, browser, or OS coverage: compiled REA CLI on macOS arm64; other environments remain unverified.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: tests use temporary homes and local CLI subprocesses; no credentials or production configuration are required.

Author readiness

Human owner: @IvenKooLab

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.
    Head 9c59b2b70 remains conflict-free against current main (verified 2026-10-19); readiness re-check requested after the head-association race.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3275 **Author:** [@IvenKooLab](https://github.com/IvenKooLab) **Created:** 10/14/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `test/client-lifecycle-sweep` --- ### 📝 Commits (10+) - [`585f11c`](https://github.com/morluto/rea/commit/585f11cae7ebf86c9f7d75e720686afd375e9cf6) test(setup): sweep supported client lifecycle symmetry - [`3e7e2ea`](https://github.com/morluto/rea/commit/3e7e2ea8936c4efe4fd4511157bc21bc434bc5ae) test(setup): assert uninstall through post-run doctor state - [`92dcbf9`](https://github.com/morluto/rea/commit/92dcbf9a7fb8a350e8624058d942befa6873eb07) test(setup): refresh the isolated client lifecycle sweep - [`9c59b2b`](https://github.com/morluto/rea/commit/9c59b2b704d6d423bee8a97acedd0b19aad77ed1) test(setup): isolate Gemini settings and retain failed client paths - [`e80f1c7`](https://github.com/morluto/rea/commit/e80f1c76d2a2a6195eaca72d4ec68222d0154ae4) Merge remote-tracking branch 'upstream/main' into test/client-lifecycle-sweep - [`2a2e9ad`](https://github.com/morluto/rea/commit/2a2e9adc5e4b5dc8da089fbe690df8b8f42209e6) test(setup): forward the host SDK selection into the lifecycle sweep - [`1c61dd8`](https://github.com/morluto/rea/commit/1c61dd854ed0f0d233abe1b0123e21fd7238ba84) Merge branch 'main' into test/client-lifecycle-sweep - [`b617a4c`](https://github.com/morluto/rea/commit/b617a4c1849ea02d43d1c7673986c23a900a9322) Merge branch 'main' into test/client-lifecycle-sweep - [`3a03ef1`](https://github.com/morluto/rea/commit/3a03ef1e5d5bee3eb9d096805225ed509ce41f13) test(setup): preserve the selected Darwin compiler in lifecycle sweep - [`acd6497`](https://github.com/morluto/rea/commit/acd64972d80e4aff5201e1bdcbc705f089c9edcd) Merge remote-tracking branch 'upstream/main' into test/client-lifecycle-sweep ### 📊 Changes **1 file changed** (+122 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `tests/acceptance/setup/supportedClientLifecycle.test.ts` (+122 -0) </details> ### 📄 Description ## Summary Add a compiled-CLI lifecycle test for every configurable client in the current platform catalog. Selected-profile tests cover only part of that catalog. ## Problem and expected behavior Each supported client should register correctly, survive repeated setup without changes, and lose its REA registration on uninstall. Uninstall must remove REA's skill manifest while preserving user-owned files. This follows the [maintainer's request](https://github.com/morluto/rea/pull/3275#issuecomment-6186258999); there is no separate issue. ## Change and scope The test registers cases from `SUPPORTED_CLIENT_DEFINITIONS`, then resolves paths inside each case. A path error fails that case instead of removing it from the sweep. It uses the existing `cliTest` fixture and its worker-scoped compile cache. Each client gets a separate temporary home so shared configuration paths cannot hide another client's failure. Gemini's system settings and defaults also point inside that home. For each client, it runs setup, scoped doctor, repeated setup, uninstall, and scoped doctor again. It checks registration and skill alignment, unchanged configuration bytes, no repeated setup actions, and an exact `missing` registration state after uninstall. User-owned notes must survive. The PR adds one acceptance-test file. Existing malformed-input tests supply parser coverage. This fits the existing [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope). ## Contract and boundary impact - Semantic owner and earliest changed stage: setup acceptance tests; no production behavior changes. - Contribution scope fit and linked maintainer decision: existing lifecycle coverage, requested in the comment above. No new capability or authority. - CLI and MCP/tool-catalog contract: exercises existing setup, doctor, and uninstall commands; no contract changes. - Provider, bridge, target-format, or platform compatibility: uses the current platform's client catalog; no provider changes. - Evidence, artifact, provenance, or reconstruction contract: no contract changes; executed results and measurement limits are below. - Target selection, authority, upstream behavior, and cleanup details (if applicable): isolated temporary homes; verifies REA cleanup and preservation of user-owned notes. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none; no generated files committed. Expected effects during tests: - [x] Read existing files or captured data. - [x] Launch or attach to a process. - [x] Inject instrumentation or run scripts/code. - [x] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. These effects are CLI subprocesses and test code reading and writing disposable configuration files. No instrumentation is injected and no process memory or registers are changed. ## Evidence and regression coverage - Tests added or updated: 24 client workflows passed on macOS arm64, with five CLI calls per client. - Base reproduction or other evidence: a temporary invalid `OMP_PROFILE` made the previous sweep omit OMP (23 cases, targeted run skipped with exit 0). The updated sweep retained all 24 cases and failed OMP with the producer's path error. The fault was removed before the passing run. An earlier Hermes fault check restored its registration after uninstall and failed on `aligned` versus expected `missing`; that fault was also removed. - User-visible CLI/MCP output (if applicable): no output changes. Assertions check setup `ready`, uninstall `complete`, doctor registration `aligned` then `missing`, and exit codes 0 then 1. - Remaining proof gaps: Windows/Linux, actual client applications, model backends, and Hopper/Ghidra were not exercised. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed October 20 verification at `3a03ef1e5` (macOS arm64, pinned Node 24.18.0/npm 11.16.0, fresh locked `npm ci`): - `npm run test:focused -- tests/acceptance/setup/supportedClientLifecycle.test.ts` — **24/24 passed**, 206.02 seconds, with command-scoped `SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX26.5.sdk` and `REA_DARWIN_SWIFTC_PATH=/Library/Developer/CommandLineTools/usr/bin/swiftc`. - An additional filtered compiled Codex lifecycle case with a local 120-second case deadline — 1 passed, 23 filtered/skipped; this is extra toolchain evidence, not a second complete sweep. - `npm run check`, `git diff --check`, formatting and pre-commit/pre-push hooks — passed. - Both caller-selected toolchain values now reach each child CLI. No compiler path or SDK is hardcoded in the test. The current registration-absence assertion remains a present doctor row with `state: "missing"`, separate from skill removal and user-note retention. - No actual third-party client execution or additional Windows/Linux local host run is claimed. New-head GitHub CI is pending; the previous head's required CI and author declaration passed. Node 24.18.0, npm 11.16.0, locked dependencies, macOS arm64: - `npm run build:cached` — passed. - `npm run test:focused -- tests/acceptance/setup/supportedClientLifecycle.test.ts tests/acceptance/setup/geminiWorkspaceTrust.test.ts` — 29 passed (24 lifecycle cases and 5 Gemini workspace/trust cases). - Earlier validation on the same production runtime: `npm run test:focused -- tests/boundary/cli/clientProfileLifecycle.test.ts tests/acceptance/setup/clientConfigurationDiagnosticsCli.test.ts` — 18 passed. - `npm run check` — passed. - `npm run verify:test-discovery` — all 1,327 test files assigned exactly once. - `git diff --check` — passed. Before this review follow-up, one fresh-worker measurement per version took 100.72 s before and 96.07 s after (updated sweep at `92dcbf9a7`). Child-tree CPU was 142.56/136.53 s; sampled aggregate peak RSS was 654176/660384 KiB. Both ran 120 CLI calls (source-derived). Build/install were excluded, OS caches were not reset, and RSS was sampled every 100 ms with shared mappings included. Separate cold/warm costs and model tokens were not measured. These runs do not establish a reliable performance improvement. Performance was not remeasured for this isolation/catalog follow-up. ## Compatibility, safety, and release - Breaking changes or migration steps: none; tests only. - Real Hopper/Ghidra, browser, or OS coverage: compiled REA CLI on macOS arm64; other environments remain unverified. - Package or release metadata impact: none. - Security, privacy, process, or containment review: tests use temporary homes and local CLI subprocesses; no credentials or production configuration are required. ## Author readiness Human owner: @IvenKooLab - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. _Head 9c59b2b70 remains conflict-free against current `main` (verified 2026-10-19); readiness re-check requested after the head-association race._ --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5692
No description provided.