[GH-ISSUE #2238] ci: make static-check feedback concise without weakening diagnostics

bsakaguchi commented

2026-10-15 18:27:53 +00:00

Owner

Originally created by @N0zoM1z0 on GitHub (Oct 12, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2238

Originally assigned to: @N0zoM1z0 on GitHub.

Problem

Successful static checks drown actionable feedback in thousands of lines. This is a presentation problem; the checks still detect bugs and drift and should remain enabled. Follow-up to #2190 and merged #2210.

Baseline: main 428eee00fdd0406cdf2b734c9ea14a2bb02da279, Node 24.18.0 / npm 11.16.0, oxlint 1.87.0 and jscpd 5.0.12 from the lockfile.

Observed and reproduced evidence

Main CI run 38272242883 passed. Its static log was 529,411 bytes / 6,426 lines, including 1,288 clone blocks and 911 lint warnings. The actual check:ci step took 18 seconds; changing reporting is not a claim of large execution-time savings.

On an isolated checkout of this exact baseline:

jscpd src/ tests/ --reporters consoleFull --no-colors exited 0 and printed 208,031 bytes / 5,175 lines.
jscpd src/ tests/ --reporters json --output /tmp/rea-ci-efficiency-duplicates --no-colors exited 0 and printed 325 bytes / 6 lines. Its full report retained 1,288 clones, 2,189 sources, and 13,553 duplicated lines (2.8191602218219187%), below the existing 3% threshold.
oxlint --format json exited 0 and retained all 911 diagnostics in a 359,118-byte report. Individual rule, filename, severity and source location remain available.
A separate lint fixture with an unused variable and debugger exited 1 and retained both warning and error diagnostics.
A separate duplicate fixture exited 1 with 48.64864864864865% duplication using JSON reporting and a 3% threshold. Thus the compact reporter can preserve the actual failure contract.

The jscpd config also emits an unknown-field warning for _comment; move that explanation into the testing guide while retaining the actual scan policy.

Proposed bounded change

Execute the same installed checkers once, with machine-readable complete reports. Emit concise success statistics/warning aggregates and actionable error diagnostics, while preserving their failure exit status and stderr.
Keep full reports in ignored .cache/static-checks/, expose explicit detailed local commands, and upload these reports from the static CI job even when checks fail.
Declare report outputs and producer inputs in Turbo so a cached static pass restores the corresponding report instead of losing diagnostic evidence.

Do not suppress rules, change severity, omit tests from duplicate scanning, raise the 3% threshold, delete checks, or alter coverage policy.

Acceptance and failure controls

A warning-only run stays successful with a short console summary and the complete warning report.
A real lint error remains nonzero and its filename, location, rule and message are visible.
A duplication-threshold violation remains nonzero; the report preserves every clone and aggregate counts.
Tool/config failure or invalid/missing output cannot reuse an earlier successful report or produce a green result.
Cached tasks restore the report output; changing the reporter producer invalidates the relevant task.
Full static checks, workflow lint, focused behavioral regressions and applicable CI pass before merging.

Scope and remaining investigation

Three Actions samples show created-to-start scheduling delays varying from about 3 seconds to 199 seconds for test shards. The latest run also waited 207 seconds before classification-job creation. These timestamps do not prove an account quota, runner-capacity root cause, or that changing shard count would help. Retain current scheduling and coverage guarantees pending stronger evidence. Slow CLI/MCP families are being measured separately, with unchanged assertions and case inventory.

Local investigation and raw measurements are recorded in /tmp/rea-ci-efficiency-investigation-20261012.md; the GitHub links and reproduction commands above are the portable evidence.

Implementation and adversarial verification

Ready-for-review PR: #2242, based on newer main 0baad58e6. Its relevant
checker configuration and CI ownership were unchanged from the audit baseline.

The six real-checker regressions pass. A lost-exit-status mutation causes
three failures; a stale-report cleanup mutation causes two; restoration passes
all six, repeated under LC_ALL=C. Actual Turbo restores both reports
byte-for-byte after deletion; a reporter-only source mutation invalidates both
lint and jscpd task hashes, which return after restoring the source.

A clone-location control uses the same basename under src/ and tests/.
jscpd's default paths strip both scan roots, so the report producer requests
absolute paths and normalizes them once into unambiguous checkout-relative
locations, preserving all other fields. Both files remain distinguishable in
the retained artifact.

Complete static checks, architecture guards, docs/README checks and actionlint
passed locally. The complete static log is about 100 lines / 4 KB. This remains
a console-size improvement rather than a same-runner execution-speed claim.

Cached lint source inventory correction

Manual review found a real stale-success reproduction. After a successful
check:fast, append const REA_CI_CACHE_PROBE = ; to
website/public/assets/main.js. Cached check:fast exits 0, while fresh lint
exits 1 and identifies that actual file and location. Restore the original
bytes before continuing.

The installed checker also discovers docs/.vitepress/config.ts and six
experiment scripts, absent from both cached lint task inputs. The correction
adds these observed roots plus .gitignore, and excludes generated site/docs
and experiment outputs. It does not shrink lint scan scope.

The new maintained regression independently reads oxlint --debug=files and
turbo run lint lint:built --dry=json, requiring every actual checker source
to be present in both resolved input sets. The old task configuration fails
this guard. The corrected configuration passes all seven regressions; the
actual website mutation now makes check:fast fail and restoration passes.

Commit 99de74380 contains this correction and has been pushed to #2242
for a fresh full CI run. Both branch commits use the GitHub anonymous email
for author and committer.

Originally created by @N0zoM1z0 on GitHub (Oct 12, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2238 Originally assigned to: @N0zoM1z0 on GitHub. ## Problem Successful static checks drown actionable feedback in thousands of lines. This is a presentation problem; the checks still detect bugs and drift and should remain enabled. Follow-up to #2190 and merged #2210. Baseline: main `428eee00fdd0406cdf2b734c9ea14a2bb02da279`, Node 24.18.0 / npm 11.16.0, oxlint 1.87.0 and jscpd 5.0.12 from the lockfile. ## Observed and reproduced evidence [Main CI run 38272242883](https://github.com/morluto/rea/actions/runs/38272242883) passed. Its static log was 529,411 bytes / 6,426 lines, including 1,288 clone blocks and 911 lint warnings. The actual check:ci step took 18 seconds; changing reporting is not a claim of large execution-time savings. On an isolated checkout of this exact baseline: - `jscpd src/ tests/ --reporters consoleFull --no-colors` exited 0 and printed 208,031 bytes / 5,175 lines. - `jscpd src/ tests/ --reporters json --output /tmp/rea-ci-efficiency-duplicates --no-colors` exited 0 and printed 325 bytes / 6 lines. Its full report retained 1,288 clones, 2,189 sources, and 13,553 duplicated lines (2.8191602218219187%), below the existing 3% threshold. - `oxlint --format json` exited 0 and retained all 911 diagnostics in a 359,118-byte report. Individual rule, filename, severity and source location remain available. - A separate lint fixture with an unused variable and `debugger` exited 1 and retained both warning and error diagnostics. - A separate duplicate fixture exited 1 with 48.64864864864865% duplication using JSON reporting and a 3% threshold. Thus the compact reporter can preserve the actual failure contract. The jscpd config also emits an unknown-field warning for `_comment`; move that explanation into the testing guide while retaining the actual scan policy. ## Proposed bounded change 1. Execute the same installed checkers once, with machine-readable complete reports. Emit concise success statistics/warning aggregates and actionable error diagnostics, while preserving their failure exit status and stderr. 2. Keep full reports in ignored `.cache/static-checks/`, expose explicit detailed local commands, and upload these reports from the static CI job even when checks fail. 3. Declare report outputs and producer inputs in Turbo so a cached static pass restores the corresponding report instead of losing diagnostic evidence. Do not suppress rules, change severity, omit tests from duplicate scanning, raise the 3% threshold, delete checks, or alter coverage policy. ## Acceptance and failure controls - A warning-only run stays successful with a short console summary and the complete warning report. - A real lint error remains nonzero and its filename, location, rule and message are visible. - A duplication-threshold violation remains nonzero; the report preserves every clone and aggregate counts. - Tool/config failure or invalid/missing output cannot reuse an earlier successful report or produce a green result. - Cached tasks restore the report output; changing the reporter producer invalidates the relevant task. - Full static checks, workflow lint, focused behavioral regressions and applicable CI pass before merging. ## Scope and remaining investigation Three Actions samples show created-to-start scheduling delays varying from about 3 seconds to 199 seconds for test shards. The latest run also waited 207 seconds before classification-job creation. These timestamps do not prove an account quota, runner-capacity root cause, or that changing shard count would help. Retain current scheduling and coverage guarantees pending stronger evidence. Slow CLI/MCP families are being measured separately, with unchanged assertions and case inventory. Local investigation and raw measurements are recorded in `/tmp/rea-ci-efficiency-investigation-20261012.md`; the GitHub links and reproduction commands above are the portable evidence. ## Implementation and adversarial verification Ready-for-review PR: #2242, based on newer main `0baad58e6`. Its relevant checker configuration and CI ownership were unchanged from the audit baseline. The six real-checker regressions pass. A lost-exit-status mutation causes three failures; a stale-report cleanup mutation causes two; restoration passes all six, repeated under `LC_ALL=C`. Actual Turbo restores both reports byte-for-byte after deletion; a reporter-only source mutation invalidates both lint and jscpd task hashes, which return after restoring the source. A clone-location control uses the same basename under `src/` and `tests/`. jscpd's default paths strip both scan roots, so the report producer requests absolute paths and normalizes them once into unambiguous checkout-relative locations, preserving all other fields. Both files remain distinguishable in the retained artifact. Complete static checks, architecture guards, docs/README checks and actionlint passed locally. The complete static log is about 100 lines / 4 KB. This remains a console-size improvement rather than a same-runner execution-speed claim. ## Cached lint source inventory correction Manual review found a real stale-success reproduction. After a successful `check:fast`, append `const REA_CI_CACHE_PROBE = ;` to `website/public/assets/main.js`. Cached `check:fast` exits 0, while fresh lint exits 1 and identifies that actual file and location. Restore the original bytes before continuing. The installed checker also discovers `docs/.vitepress/config.ts` and six experiment scripts, absent from both cached lint task inputs. The correction adds these observed roots plus `.gitignore`, and excludes generated site/docs and experiment outputs. It does not shrink lint scan scope. The new maintained regression independently reads `oxlint --debug=files` and `turbo run lint lint:built --dry=json`, requiring every actual checker source to be present in both resolved input sets. The old task configuration fails this guard. The corrected configuration passes all seven regressions; the actual website mutation now makes `check:fast` fail and restoration passes. Commit `99de74380` contains this correction and has been pushed to #2242 for a fresh full CI run. Both branch commits use the GitHub anonymous email for author and committer.

bsakaguchi

2026-10-15 18:27:53 +00:00

closed this issue
added the
area: build

enhancement
labels

bsakaguchi commented

2026-10-15 18:27:54 +00:00

Author

Owner

@sa71044298-create commented on GitHub (Oct 12, 2026):

@sa71044298-create commented on GitHub (Oct 12, 2026): WhatsApp

bsakaguchi commented

2026-10-15 18:27:54 +00:00

Author

Owner

@sa71044298-create commented on GitHub (Oct 12, 2026):

@sa71044298-create commented on GitHub (Oct 12, 2026): ![IMG-20261011-WA0013.jpg](https://github.com/user-attachments/assets/3c04ada2-6f2a-44ae-ab9b-b66d860e9e26)

Rows
Columns

[GH-ISSUE #2238] ci: make static-check feedback concise without weakening diagnostics #743

Problem

Observed and reproduced evidence

Proposed bounded change

Acceptance and failure controls

Scope and remaining investigation

Implementation and adversarial verification

Cached lint source inventory correction