[GH-ISSUE #6275] [Bug] Windows native verifier's private-runtime ACL observer times out in powershell.exe #3800

Closed
opened 2026-10-31 00:44:11 +00:00 by bsakaguchi · 0 comments
Owner

Originally created by @kaoru0822-kitauji on GitHub (Oct 22, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6275

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

  • I searched the open and closed issues for a related report.
  • I removed credentials, tokens, private paths, and sensitive target data.

Area

CI / Windows native verification

Observed behavior

npm run verify:windows-native in the Windows capability and package lane intermittently fails with Native conformance failed ETIMEDOUT / spawnSync powershell.exe ETIMEDOUT. The child is the private-runtime ACL observer in scripts/verify-windows-native.mjs (execFileSync("powershell.exe", …, { timeout: 15_000 })).

Two failure shapes were observed (from job logs):

  1. Scheduled main run 39723321808 (bd5d189): phases started 475ms, acl-read 949ms, owner-sid-read 954ms, rule-sids-read 10485ms, readonly-set 10500ms, then no complete before the 15s kill.
  2. PR #6259 run 39731089547 (JavaScript-only change): stdout: '', stderr: '', signal: 'SIGTERM' — powershell.exe did not reach its first script statement within 15s.

For comparison, 15 passing Windows lane jobs from 2026-10-22 03:41–05:41Z reported in-script complete at 1235–2883ms (median ~1.45s). Typical per-phase cost: Get-Acl 90–400ms, Update-TypeData guard + GetAccessRules 330–900ms, final @{…} | ConvertTo-Json / Where-Object 680–1520ms. Step start to observer output was 2.3–8.0s.

Reproduction

Not reproduced locally (no Windows host). Observed in hosted-runner CI only, 2 of 17 Windows lane jobs that reached this step in the sampled window.

Expected behavior

The independent SID-only ACL observer completes deterministically, with a deadline that bounds a hung observer rather than ordinary cold-start or runner contention.

Impact and scope

Turns scheduled main CI and unrelated PRs red. No product behavior affected; CI-only verifier.

Earliest failure stage

Unknown (between powershell.exe process start and script completion; varies per run).

Relevant output or evidence

Observed: the phase lines and empty-stderr result quoted above, from the linked job logs.

Source-derived: most of the in-script time is spent in PowerShell machinery rather than the ACL read: Get-Acl (module autoload), Update-TypeData (added in #5941 to poison the Access property; it updates the session type table), and the pipeline/ConvertTo-Json output.

Hypothesis (unconfirmed): the failing runs were CPU/disk-starved — the first failure ran every phase ~20x slower than typical, and the second never reached the first statement. The 15s deadline then covers both powershell.exe cold start and ~1.5s of avoidable cmdlet work.

Proposed check: make the observer call .NET members directly (SID-typed GetOwner/GetAccessRules, string-built JSON), keep the SID-only guarantee without a runtime type-table update, record process uptime at script start, and size the deadline for cold start.

Originally created by @kaoru0822-kitauji on GitHub (Oct 22, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6275 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CI / Windows native verification ### Observed behavior `npm run verify:windows-native` in the **Windows capability and package lane** intermittently fails with `Native conformance failed ETIMEDOUT` / `spawnSync powershell.exe ETIMEDOUT`. The child is the private-runtime ACL observer in `scripts/verify-windows-native.mjs` (`execFileSync("powershell.exe", …, { timeout: 15_000 })`). Two failure shapes were observed (from job logs): 1. Scheduled main run [39723321808](https://github.com/morluto/rea/actions/runs/39723321808/job/119648114325) (`bd5d189`): phases `started 475ms, acl-read 949ms, owner-sid-read 954ms, rule-sids-read 10485ms, readonly-set 10500ms`, then no `complete` before the 15s kill. 2. PR #6259 run [39731089547](https://github.com/morluto/rea/actions/runs/39731089547/job/119666774462) (JavaScript-only change): `stdout: ''`, `stderr: ''`, `signal: 'SIGTERM'` — powershell.exe did not reach its first script statement within 15s. For comparison, 15 passing Windows lane jobs from 2026-10-22 03:41–05:41Z reported in-script `complete` at 1235–2883ms (median ~1.45s). Typical per-phase cost: `Get-Acl` 90–400ms, `Update-TypeData` guard + `GetAccessRules` 330–900ms, final `@{…} | ConvertTo-Json` / `Where-Object` 680–1520ms. Step start to observer output was 2.3–8.0s. ### Reproduction Not reproduced locally (no Windows host). Observed in hosted-runner CI only, 2 of 17 Windows lane jobs that reached this step in the sampled window. ### Expected behavior The independent SID-only ACL observer completes deterministically, with a deadline that bounds a hung observer rather than ordinary cold-start or runner contention. ### Impact and scope Turns scheduled main CI and unrelated PRs red. No product behavior affected; CI-only verifier. ### Earliest failure stage Unknown (between powershell.exe process start and script completion; varies per run). ### Relevant output or evidence Observed: the phase lines and empty-stderr result quoted above, from the linked job logs. Source-derived: most of the in-script time is spent in PowerShell machinery rather than the ACL read: `Get-Acl` (module autoload), `Update-TypeData` (added in #5941 to poison the `Access` property; it updates the session type table), and the pipeline/`ConvertTo-Json` output. Hypothesis (unconfirmed): the failing runs were CPU/disk-starved — the first failure ran every phase ~20x slower than typical, and the second never reached the first statement. The 15s deadline then covers both powershell.exe cold start and ~1.5s of avoidable cmdlet work. Proposed check: make the observer call .NET members directly (SID-typed `GetOwner`/`GetAccessRules`, string-built JSON), keep the SID-only guarantee without a runtime type-table update, record process uptime at script start, and size the deadline for cold start.
bsakaguchi 2026-10-31 00:44:11 +00:00
  • closed this issue
  • added the
    bug
    label
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#3800
No description provided.