[PR #13] [MERGED] Explain regressions on the commit page and offer the re-bless flow #12

Closed
opened 2026-10-31 22:06:00 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfyci-dashboard/pull/13
Author: @marawan206
Created: 10/5/2026
Status: ✅ Merged
Merged: 10/8/2026
Merged by: @alexisrolland

Base: regression-history ← Head: regression-triage


📝 Commits (9)

  • 20958d8 feat(regression): triage components for the commit page
  • 62c62b9 feat(regression): lane-aware commit page with triage summary
  • 5997910 fix(regression): tolerate commit metadata without parents
  • 67ce8bf fix(regression): take inherited drift from the index only
  • 2c46d38 fix(regression): keep the run's golden apart from the current golden
  • 58c50c1 fix(regression): report failed fetches on the commit page
  • b74f54c fix(regression): explain execution errors in the triage summary
  • 32f9d8a fix(regression): use the shared overall rule and first-bad line
  • cdc94a6 fix(regression): say which lane the commit page shows for an unknown ?lane=

📊 Changes

8 files changed (+553 additions, -36 deletions)

View changed files

➕ components/CommitMetaChip.tsx (+43 -0)
📝 components/LaneSelect.tsx (+30 -0)
📝 components/MetricTable.tsx (+24 -2)
➕ components/ReblessCallout.tsx (+60 -0)
📝 components/RegressionBadge.tsx (+1 -1)
➕ components/RegressionSummaryHeader.tsx (+199 -0)
📝 pages/regression/[branch]/[commit].tsx (+194 -32)
📝 src/regression/types.ts (+2 -1)

📄 Description

Why

A failing commit page showed metrics and diff images but did not say what changed, by how much, against which baseline, or since when, and gave no way forward when the change was intentional (the current FLUX case: a kernel optimisation in #16488 moved every pixel by a rounding error and the v0.33.2 golden is simply stale).

What

On pages/regression/[branch]/[commit].tsx:

  • RegressionSummaryHeader: pass/fail counts; for each failing workflow one sentence against the golden it was judged by ("vs golden v0.33.2 (blessed 2026-08-20 by …): MSE 3.44 > 2, PSNR 42.8 dB < 45, 70.0% px > 1%; outputs unchanged vs previous run "), an "Inherited drift" label when the index says so, an amber "Accepted drift" badge when the output sha matches the lane's current golden, and a "failing since" line (commit, PR, author, N runs, compare link) from the index head. Execution errors are explained with the worker's status.
  • CommitMetaChip: subject, author, committed date, PR and parent links from summary.commit_meta (falls back to the index entry, hidden when neither exists).
  • Lane tabs (?lane=) when more than one lane is published; every URL goes through the lane-aware path resolver.
  • ReblessCallout on failing cards: explains the re-bless flow with a copyable gh workflow run golden-baseline.yml … command.
  • MetricTable: optional noise-floor row ("golden re-run: identical / metrics").
  • Env strip gains driver, CUDA and runtime backends when the run record carries them. Failed fetches are reported with a Retry button.
  • The run's own golden tag is kept apart from the lane's current golden, so a page for a pre-bless run keeps reading correctly after a re-bless.

Stacked on #12 (shares types.ts, client.ts, RegressionBadge, LaneSelect).

Verification

  • pnpm lint: no warnings; pnpm build: all routes compile (checked from a clean worktree).
  • Vercel preview: /regression/master/2139131e880cf07116f0d0238de8cfbb125d2114 reads "flux_dev_t2i vs golden v0.33.2: MSE 3.44 > 2, PSNR 42.8 dB < 45, 70.0% px > 1%; outputs unchanged vs previous run 7fbcfa8b" with "Inherited drift, failing since b5cc8830 (#16488, comfyanonymous)"; /regression/master/b5cc8830279eae909a59de030af1e50761c36751 shows the same commit as the first bad one with its diff figures; the sdxl card stays green.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfyci-dashboard/pull/13 **Author:** [@marawan206](https://github.com/marawan206) **Created:** 10/5/2026 **Status:** ✅ Merged **Merged:** 10/8/2026 **Merged by:** [@alexisrolland](https://github.com/alexisrolland) **Base:** `regression-history` ← **Head:** `regression-triage` --- ### 📝 Commits (9) - [`20958d8`](https://github.com/Comfy-Org/comfyci-dashboard/commit/20958d8390191243e6468b2867002e528c7961db) feat(regression): triage components for the commit page - [`62c62b9`](https://github.com/Comfy-Org/comfyci-dashboard/commit/62c62b9e49ef6bba76b6fba6ab9e0af9ce9abc66) feat(regression): lane-aware commit page with triage summary - [`5997910`](https://github.com/Comfy-Org/comfyci-dashboard/commit/5997910138b36995a04964c9a082d2959fcdcd46) fix(regression): tolerate commit metadata without parents - [`67ce8bf`](https://github.com/Comfy-Org/comfyci-dashboard/commit/67ce8bf6785084441d72a9c4b74e7cf6d547dd9e) fix(regression): take inherited drift from the index only - [`2c46d38`](https://github.com/Comfy-Org/comfyci-dashboard/commit/2c46d3886b007e63efdd3f790d485718478a7ec3) fix(regression): keep the run's golden apart from the current golden - [`58c50c1`](https://github.com/Comfy-Org/comfyci-dashboard/commit/58c50c16403fc8c52bbee2a48f89911bf71ee1ed) fix(regression): report failed fetches on the commit page - [`b74f54c`](https://github.com/Comfy-Org/comfyci-dashboard/commit/b74f54ce0bdc9c5b9b70013320286c90084c75c8) fix(regression): explain execution errors in the triage summary - [`32f9d8a`](https://github.com/Comfy-Org/comfyci-dashboard/commit/32f9d8ac3201f2105d9b12de2be8df756b80b37a) fix(regression): use the shared overall rule and first-bad line - [`cdc94a6`](https://github.com/Comfy-Org/comfyci-dashboard/commit/cdc94a60836c283925ece064a17f021d09dbe058) fix(regression): say which lane the commit page shows for an unknown ?lane= ### 📊 Changes **8 files changed** (+553 additions, -36 deletions) <details> <summary>View changed files</summary> ➕ `components/CommitMetaChip.tsx` (+43 -0) 📝 `components/LaneSelect.tsx` (+30 -0) 📝 `components/MetricTable.tsx` (+24 -2) ➕ `components/ReblessCallout.tsx` (+60 -0) 📝 `components/RegressionBadge.tsx` (+1 -1) ➕ `components/RegressionSummaryHeader.tsx` (+199 -0) 📝 `pages/regression/[branch]/[commit].tsx` (+194 -32) 📝 `src/regression/types.ts` (+2 -1) </details> ### 📄 Description ## Why A failing commit page showed metrics and diff images but did not say what changed, by how much, against which baseline, or since when, and gave no way forward when the change was intentional (the current FLUX case: a kernel optimisation in #16488 moved every pixel by a rounding error and the v0.33.2 golden is simply stale). ## What On `pages/regression/[branch]/[commit].tsx`: - `RegressionSummaryHeader`: pass/fail counts; for each failing workflow one sentence against the golden it was judged by ("vs golden v0.33.2 (blessed 2026-08-20 by …): MSE 3.44 > 2, PSNR 42.8 dB < 45, 70.0% px > 1%; outputs unchanged vs previous run <sha>"), an "Inherited drift" label when the index says so, an amber "Accepted drift" badge when the output sha matches the lane's current golden, and a "failing since" line (commit, PR, author, N runs, compare link) from the index head. Execution errors are explained with the worker's status. - `CommitMetaChip`: subject, author, committed date, PR and parent links from `summary.commit_meta` (falls back to the index entry, hidden when neither exists). - Lane tabs (`?lane=`) when more than one lane is published; every URL goes through the lane-aware path resolver. - `ReblessCallout` on failing cards: explains the re-bless flow with a copyable `gh workflow run golden-baseline.yml …` command. - `MetricTable`: optional noise-floor row ("golden re-run: identical / metrics"). - Env strip gains driver, CUDA and runtime backends when the run record carries them. Failed fetches are reported with a Retry button. - The run's own golden tag is kept apart from the lane's current golden, so a page for a pre-bless run keeps reading correctly after a re-bless. Stacked on #12 (shares `types.ts`, `client.ts`, `RegressionBadge`, `LaneSelect`). ## Verification - `pnpm lint`: no warnings; `pnpm build`: all routes compile (checked from a clean worktree). - Vercel preview: `/regression/master/2139131e880cf07116f0d0238de8cfbb125d2114` reads "flux_dev_t2i vs golden v0.33.2: MSE 3.44 > 2, PSNR 42.8 dB < 45, 70.0% px > 1%; outputs unchanged vs previous run 7fbcfa8b" with "Inherited drift, failing since b5cc8830 (#16488, comfyanonymous)"; `/regression/master/b5cc8830279eae909a59de030af1e50761c36751` shows the same commit as the first bad one with its diff figures; the sdxl card stays green. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 22:06:00 +00:00
Sign in to join this conversation.
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Comfy-Org/comfyci-dashboard#12
No description provided.