[PR #3013] [MERGED] perf(ci): start source feedback early and diagnose stalled runs #5556

New issue

Closed

opened 2026-10-31 01:07:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:07:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3013
Author: @morluto
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @morluto

Base: main ← Head: perf/ci-feedback-planning

📝 Commits (9)

a610b19 perf(ci): start source feedback early and diagnose stalled runs
b078411 fix(ci): verify workflow identities before main comparison limits
52211f7 fix(ci): preserve diagnostics across passing checks and install failures
41db1a1 fix(ci): preserve ambiguous workflow run ordering
33cb51b chore(ci): reconcile main with CI feedback changes
d387501 fix(ci): preserve failure-report candidate and request budgets
0b40fe5 fix(ci): retain candidate errors while checking later main evidence
2b9efb6 fix(ci): retain partial reports and require complete health scans
8ef9ad6 test(ci): prune stale fixtures and duplicated assumptions

📊 Changes

43 files changed (+4661 additions, -192 deletions)

View changed files

📝 .github/workflows/ci-timings.yml (+39 -1)
📝 .github/workflows/ci.yml (+62 -11)
📝 .github/workflows/real-cutter.yml (+7 -5)
📝 docs/testing.md (+33 -2)
📝 package.json (+5 -2)
➕ scripts/ci/artifacts.d.mts (+67 -0)
➕ scripts/ci/artifacts.mjs (+436 -0)
➕ scripts/ci/failure-report.d.mts (+159 -0)
➕ scripts/ci/failure-report.mjs (+529 -0)
➕ scripts/ci/github-api.d.mts (+16 -0)
➕ scripts/ci/github-api.mjs (+33 -0)
📝 scripts/ci/job-timings-batch.mjs (+3 -0)
➕ scripts/ci/pr-ci-health.d.mts (+95 -0)
➕ scripts/ci/pr-ci-health.mjs (+718 -0)
📝 scripts/ci/qualification.mjs (+4 -3)
📝 scripts/ci/required.mjs (+5 -1)
📝 scripts/ci/static-check-report.mjs (+2 -0)
➕ scripts/ci/static-task-context.d.mts (+18 -0)
➕ scripts/ci/static-task-context.mjs (+104 -0)
➕ scripts/ci/test-group.mjs (+46 -0)

...and 23 more files

📄 Description

Source tests currently wait for the package build even when they import only source. Missing PR workflows, runner waits, and approval waits are also hard to distinguish. This follows merged #3007 with earlier portable feedback and bounded read-only CI diagnostics on GitHub-hosted runners.

Fast CI uses one source job after classification and five built-dependent shards after build. Full qualification retains six complete coverage shards. Both tiers retain every portable test, exact inventory checks, and failure gates.
Vitest, CI group execution, and development preparation consume one canonical project definition. The Hopper conformance test that imports a compiled verifier now belongs to the built group. Independent source/built blob labels preserve both reports during merging.
The existing hourly/manual CI timings workflow diagnoses missing/startup-failed, queued/running, approval-required, canceled, failed, and unknown PR CI. It binds evidence to the current head and repository, counts older open PRs for ambiguous branch identities, and preserves unknown/deferred outcomes for incomplete scans. Collection uses GET-only authority and writes artifacts/step summaries.
Static reports retain the producing Turbo task hashes, exact checkout and PR head identities, toolchain, and clean-source state. The diagnostic reporter compares supported lint/duplication evidence against failed main ancestors, verifying PR merge parents and source blobs. matching-main-diagnostics describes supported diagnostics only; other causes remain unclassified. Historical missing receipts and rerun artifacts without attempt proof stay unknown. Receipt collection is best effort and cannot skip the actual static checks.
Correct the existing message-budget regression assertion to expect the production owner's canonical sorting while still requiring all 100 messages.

Adaptive shard-count reduction remains deferred. The available successful hosted timing artifact covers an older 1,061-file inventory; 80 current files lack usable durations. Phase weights do not predict job wall time. This split keeps six test jobs, and the selector remains observational. Smaller matrices need current complete timings and completed hosted comparisons.

Validation on a610b19d897793f5baeeab693e91dc3be2af5599, based on main 0bf01dfa595c6d2bf7012c049a253895d89de8b8, Node 24.18.0 / npm 11.16.0:

All 460 source files: 6,420 tests passed, 9 skipped, in a separate checkout without dist/ or generated artifacts.
238 focused CI/cache/gate/report/development tests passed, including real independent blob production and merge; the final receipt-step regression also passed after making collection best effort.
npm run check:ci, npm run docs:check, exact discovery of all 1,132 test files, actionlint, and diff checks passed. A clean checkout receipt records the exact final SHA and eight task identities.
Live read-only health collection returned a partial snapshot at its limits instead of claiming absent CI from incomplete evidence.

The completed source-only run took 72.04s elapsed, 119.71s sampled-family CPU, and 2.41 GiB peak summed process-family RSS. Dependencies/OS caches were warm; no build artifacts were present. CPU includes the ps sampler; RSS samples are approximately 100ms and can double-count shared pages. Host interference was uncontrolled. These are task measurements, not a before/after or hosted CI speedup; runner queue time, cold install, full built-suite/coverage performance, real-provider execution, and agent tokens remain unmeasured.

Research: checked OpenClaw source pinned at 41a5b474f7f463e8d1109dd9b23029bdb334adb0, Context7 for installed Vitest 5.0.3/GitHub Actions, and DeepWiki for orientation; verified conclusions against primary source. Relevant examples are OpenClaw manifest planning, source/build job prerequisites, missing-CI sweeper, and Vitest blob reports. REA's detector only reports observations; no close/reopen recovery or main-red waivers are introduced.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3013 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `perf/ci-feedback-planning` --- ### 📝 Commits (9) - [`a610b19`](https://github.com/morluto/rea/commit/a610b19d897793f5baeeab693e91dc3be2af5599) perf(ci): start source feedback early and diagnose stalled runs - [`b078411`](https://github.com/morluto/rea/commit/b0784116ddca3c4035f12f21dcd0549835402f73) fix(ci): verify workflow identities before main comparison limits - [`52211f7`](https://github.com/morluto/rea/commit/52211f741ed0f1a95515687c47dc35a86dbacab7) fix(ci): preserve diagnostics across passing checks and install failures - [`41db1a1`](https://github.com/morluto/rea/commit/41db1a1bdab9a042aab603e85246964134d2f4ef) fix(ci): preserve ambiguous workflow run ordering - [`33cb51b`](https://github.com/morluto/rea/commit/33cb51b33629c5f75c549339fca33f879256ed1f) chore(ci): reconcile main with CI feedback changes - [`d387501`](https://github.com/morluto/rea/commit/d387501100260f64886a034d687ed8b794669f65) fix(ci): preserve failure-report candidate and request budgets - [`0b40fe5`](https://github.com/morluto/rea/commit/0b40fe569e57b972682cebfca5f8b03868670ac4) fix(ci): retain candidate errors while checking later main evidence - [`2b9efb6`](https://github.com/morluto/rea/commit/2b9efb601cd66f8ab3c5e4bf61f741e1ab5df503) fix(ci): retain partial reports and require complete health scans - [`8ef9ad6`](https://github.com/morluto/rea/commit/8ef9ad66ada5333b80fc01289820338761251e2b) test(ci): prune stale fixtures and duplicated assumptions ### 📊 Changes **43 files changed** (+4661 additions, -192 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/ci-timings.yml` (+39 -1) 📝 `.github/workflows/ci.yml` (+62 -11) 📝 `.github/workflows/real-cutter.yml` (+7 -5) 📝 `docs/testing.md` (+33 -2) 📝 `package.json` (+5 -2) ➕ `scripts/ci/artifacts.d.mts` (+67 -0) ➕ `scripts/ci/artifacts.mjs` (+436 -0) ➕ `scripts/ci/failure-report.d.mts` (+159 -0) ➕ `scripts/ci/failure-report.mjs` (+529 -0) ➕ `scripts/ci/github-api.d.mts` (+16 -0) ➕ `scripts/ci/github-api.mjs` (+33 -0) 📝 `scripts/ci/job-timings-batch.mjs` (+3 -0) ➕ `scripts/ci/pr-ci-health.d.mts` (+95 -0) ➕ `scripts/ci/pr-ci-health.mjs` (+718 -0) 📝 `scripts/ci/qualification.mjs` (+4 -3) 📝 `scripts/ci/required.mjs` (+5 -1) 📝 `scripts/ci/static-check-report.mjs` (+2 -0) ➕ `scripts/ci/static-task-context.d.mts` (+18 -0) ➕ `scripts/ci/static-task-context.mjs` (+104 -0) ➕ `scripts/ci/test-group.mjs` (+46 -0) _...and 23 more files_ </details> ### 📄 Description Source tests currently wait for the package build even when they import only source. Missing PR workflows, runner waits, and approval waits are also hard to distinguish. This follows merged #3007 with earlier portable feedback and bounded read-only CI diagnostics on GitHub-hosted runners. - Fast CI uses one source job after classification and five built-dependent shards after build. Full qualification retains six complete coverage shards. Both tiers retain every portable test, exact inventory checks, and failure gates. - Vitest, CI group execution, and development preparation consume one canonical project definition. The Hopper conformance test that imports a compiled verifier now belongs to the built group. Independent source/built blob labels preserve both reports during merging. - The existing hourly/manual CI timings workflow diagnoses missing/startup-failed, queued/running, approval-required, canceled, failed, and unknown PR CI. It binds evidence to the current head and repository, counts older open PRs for ambiguous branch identities, and preserves unknown/deferred outcomes for incomplete scans. Collection uses GET-only authority and writes artifacts/step summaries. - Static reports retain the producing Turbo task hashes, exact checkout and PR head identities, toolchain, and clean-source state. The diagnostic reporter compares supported lint/duplication evidence against failed main ancestors, verifying PR merge parents and source blobs. `matching-main-diagnostics` describes supported diagnostics only; other causes remain unclassified. Historical missing receipts and rerun artifacts without attempt proof stay unknown. Receipt collection is best effort and cannot skip the actual static checks. - Correct the existing message-budget regression assertion to expect the production owner's canonical sorting while still requiring all 100 messages. Adaptive shard-count reduction remains deferred. The available successful hosted timing artifact covers an older 1,061-file inventory; 80 current files lack usable durations. Phase weights do not predict job wall time. This split keeps six test jobs, and the selector remains observational. Smaller matrices need current complete timings and completed hosted comparisons. Validation on `a610b19d897793f5baeeab693e91dc3be2af5599`, based on main `0bf01dfa595c6d2bf7012c049a253895d89de8b8`, Node 24.18.0 / npm 11.16.0: - All 460 source files: 6,420 tests passed, 9 skipped, in a separate checkout without `dist/` or generated artifacts. - 238 focused CI/cache/gate/report/development tests passed, including real independent blob production and merge; the final receipt-step regression also passed after making collection best effort. - `npm run check:ci`, `npm run docs:check`, exact discovery of all 1,132 test files, actionlint, and diff checks passed. A clean checkout receipt records the exact final SHA and eight task identities. - Live read-only health collection returned a partial snapshot at its limits instead of claiming absent CI from incomplete evidence. The completed source-only run took 72.04s elapsed, 119.71s sampled-family CPU, and 2.41 GiB peak summed process-family RSS. Dependencies/OS caches were warm; no build artifacts were present. CPU includes the `ps` sampler; RSS samples are approximately 100ms and can double-count shared pages. Host interference was uncontrolled. These are task measurements, not a before/after or hosted CI speedup; runner queue time, cold install, full built-suite/coverage performance, real-provider execution, and agent tokens remain unmeasured. Research: checked OpenClaw source pinned at `41a5b474f7f463e8d1109dd9b23029bdb334adb0`, Context7 for installed Vitest 5.0.3/GitHub Actions, and DeepWiki for orientation; verified conclusions against primary source. Relevant examples are [OpenClaw manifest planning](https://github.com/openclaw/openclaw/blob/41a5b474f7f463e8d1109dd9b23029bdb334adb0/scripts/ci-build-manifest.mjs), [source/build job prerequisites](https://github.com/openclaw/openclaw/blob/41a5b474f7f463e8d1109dd9b23029bdb334adb0/.github/workflows/ci.yml), [missing-CI sweeper](https://github.com/openclaw/openclaw/blob/41a5b474f7f463e8d1109dd9b23029bdb334adb0/.github/workflows/pr-ci-sweeper.yml), and [Vitest blob reports](https://vitest.dev/guide/reporters.html#blob-reporter). REA's detector only reports observations; no close/reopen recovery or main-red waivers are introduced. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:07:34 +00:00