[GH-ISSUE #4989] [Bug] Windows update command surface test intermittently exceeds its deadline #3084

Closed
opened 2026-10-31 00:36:16 +00:00 by bsakaguchi · 3 comments
Owner

Originally created by @morluto on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4989

Before submitting

  • Searched open and closed issues for this test path and timeout; no matching report found.
  • Evidence contains no credentials or sensitive target data.

Area

Other (Windows CI)

Observed behavior

At 6fd7e7334a18223bf50008553a3557440fd37999, Windows qualification attempt 2 fails only tests/process-global/updateMaintenance.test.ts > update subprocess boundaries > exposes update as the sole CLI command: Test timed out in 30000ms (30,009 ms). 515 other curated cases pass; 31 are skipped by their platform prerequisites. Failed job.

Reproduction

Observed once on a GitHub Windows x64 runner using Node 24.18.0 and the curated workflow invocation. The same test passed in 1,676 ms on attempt 1 at the exact same source; that earlier job failed only afterward when GitHub ArtifactService/CreateArtifact timed out. The identical candidate tree also passed Windows CI in run 39104592429.

This test executes node scripts/rea.mjs --help, then node scripts/rea.mjs upgrade --json, through runUpdateCommand. NOT_ENOUGH_INFO: the log does not identify which invocation was pending at timeout; no local Windows reproduction or CPU trace is available.

Expected behavior

The command surface should expose update, omit upgrade from help, and reject the removed upgrade command within the existing test bound. Preserve stream-close collection and actual command assertions.

Impact and scope

Intermittently blocks qualification. A production command hang, deterministic timeout, and underlying cause are not established. No timeout increase, retry policy, assertion removal, or runtime change has been made.

REA version

6.7.0 candidate at 6fd7e7334a18223bf50008553a3557440fd37999.

Runtime and platform

GitHub Windows x64 hosted runner, Node 24.18.0, Vitest 5.0.3, coverage disabled.

Target and artifact context

Compiled REA CLI in an isolated temporary home; no analysis engine or target.

Interface

Other

Earliest failure stage

Unknown

Relevant output or evidence

Error: Test timed out in 30000ms. at updateMaintenance.test.ts:384. Other subprocess tests in the same file complete (68–97 ms for small fixtures); real setup dry-run completed in 3,320 ms. Source boundary: runUpdateCommand in src/application/UpdateRuntime.ts collects until physical output close. The test combines two public CLI invocations under one deadline.

Analysis or suspected cause

NOT_ENOUGH_INFO: startup contention, a retained handle/pipe, and aggregate test budgeting have not been distinguished. A fresh-runner control is being run at identical source with unchanged assertions and deadline. A successful control would not establish a root cause or a fix. Proposed follow-up: capture each invocation's stage, child exit/output-close facts and cleanup ownership before selecting a structural correction.

Originally created by @morluto on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4989 ### Before submitting - [x] Searched open and closed issues for this test path and timeout; no matching report found. - [x] Evidence contains no credentials or sensitive target data. ### Area Other (Windows CI) ### Observed behavior At `6fd7e7334a18223bf50008553a3557440fd37999`, Windows qualification attempt 2 fails only `tests/process-global/updateMaintenance.test.ts > update subprocess boundaries > exposes update as the sole CLI command`: `Test timed out in 30000ms` (30,009 ms). 515 other curated cases pass; 31 are skipped by their platform prerequisites. [Failed job](https://github.com/morluto/rea/actions/runs/39105741829/job/117615291628). ### Reproduction Observed once on a GitHub Windows x64 runner using Node 24.18.0 and the curated workflow invocation. The same test passed in 1,676 ms on attempt 1 at the exact same source; that earlier job failed only afterward when GitHub ArtifactService/CreateArtifact timed out. The identical candidate tree also passed Windows CI in run 39104592429. This test executes `node scripts/rea.mjs --help`, then `node scripts/rea.mjs upgrade --json`, through `runUpdateCommand`. NOT_ENOUGH_INFO: the log does not identify which invocation was pending at timeout; no local Windows reproduction or CPU trace is available. ### Expected behavior The command surface should expose update, omit upgrade from help, and reject the removed upgrade command within the existing test bound. Preserve stream-close collection and actual command assertions. ### Impact and scope Intermittently blocks qualification. A production command hang, deterministic timeout, and underlying cause are not established. No timeout increase, retry policy, assertion removal, or runtime change has been made. ### REA version 6.7.0 candidate at `6fd7e7334a18223bf50008553a3557440fd37999`. ### Runtime and platform GitHub Windows x64 hosted runner, Node 24.18.0, Vitest 5.0.3, coverage disabled. ### Target and artifact context Compiled REA CLI in an isolated temporary home; no analysis engine or target. ### Interface Other ### Earliest failure stage Unknown ### Relevant output or evidence `Error: Test timed out in 30000ms.` at updateMaintenance.test.ts:384. Other subprocess tests in the same file complete (68–97 ms for small fixtures); real setup dry-run completed in 3,320 ms. Source boundary: runUpdateCommand in src/application/UpdateRuntime.ts collects until physical output close. The test combines two public CLI invocations under one deadline. ### Analysis or suspected cause NOT_ENOUGH_INFO: startup contention, a retained handle/pipe, and aggregate test budgeting have not been distinguished. A fresh-runner control is being run at identical source with unchanged assertions and deadline. A successful control would not establish a root cause or a fix. Proposed follow-up: capture each invocation's stage, child exit/output-close facts and cleanup ownership before selecting a structural correction.
bsakaguchi 2026-10-31 00:36:16 +00:00
  • closed this issue
  • added the
    triage
    bug
    labels
Author
Owner

@XinwangTao commented on GitHub (Oct 18, 2026):

The evidence leans toward GitHub-hosted runner environment noise rather than a product hang: the identical tree passed in 1,676 ms on attempt 1, and that earlier job only failed later at CreateArtifact — runner VM/artifact stalls can eat the shared 30s budget of a test that normally completes in under 100 ms. Agreed that a fresh-runner control alone cannot establish root cause. Two concrete suggestions: (1) split the single deadline so each invocation (--help and upgrade --json) gets its own stage budget with start/end timestamps — the current log can't even tell which command was pending at timeout, and stage timing would settle the startup-contention vs pipe-hold question directly; (2) if staging later shows the upgrade child's stdout pipe never closing, that points at runUpdateCommand waiting for physical output close while a spawned child inherited extra handles on Windows — worth checking the child's handle inheritance when spawning. If the control stays green for a while, marking the test flaky with a retry or a modest Windows-only deadline bump would be pragmatic, but only after staging proves it isn't the child genuinely hanging.

<!-- gh-comment-id:6247493439 --> @XinwangTao commented on GitHub (Oct 18, 2026): The evidence leans toward GitHub-hosted runner environment noise rather than a product hang: the identical tree passed in 1,676 ms on attempt 1, and that earlier job only failed later at CreateArtifact — runner VM/artifact stalls can eat the shared 30s budget of a test that normally completes in under 100 ms. Agreed that a fresh-runner control alone cannot establish root cause. Two concrete suggestions: (1) split the single deadline so each invocation (--help and upgrade --json) gets its own stage budget with start/end timestamps — the current log can't even tell which command was pending at timeout, and stage timing would settle the startup-contention vs pipe-hold question directly; (2) if staging later shows the upgrade child's stdout pipe never closing, that points at runUpdateCommand waiting for physical output close while a spawned child inherited extra handles on Windows — worth checking the child's handle inheritance when spawning. If the control stays green for a while, marking the test flaky with a retry or a modest Windows-only deadline bump would be pragmatic, but only after staging proves it isn't the child genuinely hanging.
Author
Owner

@morluto commented on GitHub (Oct 18, 2026):

The fresh-runner control passed on attempt 3 at the unchanged candidate source (run 39105741829), and the subsequent exact-release Windows qualification also passed at 27dc42d10249f0410904a75c0997842eb03b931c in release run 39108764422. No timeout, assertion, retry-policy or runtime code change was made. The intermittent timeout remains unexplained; keeping this report open.

<!-- gh-comment-id:6247593084 --> @morluto commented on GitHub (Oct 18, 2026): The fresh-runner control passed on attempt 3 at the unchanged candidate source (run 39105741829), and the subsequent exact-release Windows qualification also passed at `27dc42d10249f0410904a75c0997842eb03b931c` in [release run 39108764422](https://github.com/morluto/rea/actions/runs/39108764422/job/117621783454). No timeout, assertion, retry-policy or runtime code change was made. The intermittent timeout remains unexplained; keeping this report open.
Author
Owner

@XY-code1 commented on GitHub (Oct 18, 2026):

I opened draft PR #5019 to isolate the two CLI subprocess checks into separate Vitest cases. This keeps the existing assertions, timeout values, retry policy, and runtime code unchanged, while preventing one invocation from consuming the other invocation's test budget and making any future timeout identify the exact command in the test title.

On Windows x64, the focused file passes all 28 tests; the isolated --help and removed upgrade --json checks completed in about 1.1s and 1.0s respectively. This does not claim to explain the original hosted-runner stall; that uncertainty is recorded in the PR.

<!-- gh-comment-id:6249658550 --> @XY-code1 commented on GitHub (Oct 18, 2026): I opened draft PR #5019 to isolate the two CLI subprocess checks into separate Vitest cases. This keeps the existing assertions, timeout values, retry policy, and runtime code unchanged, while preventing one invocation from consuming the other invocation's test budget and making any future timeout identify the exact command in the test title. On Windows x64, the focused file passes all 28 tests; the isolated `--help` and removed `upgrade --json` checks completed in about 1.1s and 1.0s respectively. This does not claim to explain the original hosted-runner stall; that uncertainty is recorded in the PR.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#3084
No description provided.