[PR #4] [MERGED] Record why a golden was blessed and allow promoting an existing run #4

New issue

Closed

opened 2026-10-31 22:06:10 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 22:06:10 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfyci-runpod-worker/pull/4
Author: @marawan206
Created: 10/5/2026
Status: ✅ Merged
Merged: 10/8/2026
Merged by: @alexisrolland

Base: run-index ← Head: goldens-v2

📝 Commits (1)

37aa1cc Record why a golden was blessed and allow promoting an existing run

📊 Changes

5 files changed (+455 additions, -42 deletions)

View changed files

📝 .github/workflows/golden-baseline.yml (+47 -10)
📝 README.md (+25 -1)
📝 scripts/golden_baseline.py (+192 -23)
📝 scripts/index_builder.py (+11 -8)
➕ scripts/tests/test_golden_baseline.py (+180 -0)

📄 Description

Why

Accepting an intentional numerics change (the #16488 adaln rounding-order change that moved every FLUX pixel by a rounding error) means re-blessing the golden. Nothing recorded who accepted what and why, and generating a golden always cost two GPU runs even when the accepted output already existed on the results branch.

What

golden_baseline.py: --reason, --lane (selects the lane's storage prefix), --from-run BRANCH/<sha>, --force, and a full commit sha accepted as --ref. blessed.json v2 records lane, python and cuda (driver and image identity once the worker reports them), the golden's output sha, the superseded tag and the reason; current.json carries reason, supersedes, output_sha256; golden/<wf>/history.json keeps every bless. --from-run promotes an existing run's outputs without GPU time, names the golden after that commit (a --ref must resolve to it), and inherits the previous golden's noise floor marked measured: false. An existing golden directory is never overwritten without --force; bless-only checks every workflow before blessing any. Blessing and the run-index refresh run as derived writers, so a publish that retries behind the regression cron re-applies them on the fresh tree, and the dashboard stops reporting accepted drift as an open regression in the same publish.
golden-baseline.yml: from_run, reason, lane, force inputs; every input reaches the script through the environment instead of being interpolated into the shell; own concurrency group; timeout.
README: re-bless flow.

Verification

python -m pytest scripts/tests -q (helpers, promote + bless + history on a real git remote, the from-run command-line flow end to end including a reason with quotes and $, refusal to overwrite without --force, ref/shape validation, bless-only validation before any bless).
The FLUX golden candidate at v0.38.0 was generated with the existing workflow (run 37243296184): both runs produced 3fa6e2ad…, identical noise floor. Blessing it with a reason after the maintainer ack is the first use of this flow.

Stacked on #3.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfyci-runpod-worker/pull/4 **Author:** [@marawan206](https://github.com/marawan206) **Created:** 10/5/2026 **Status:** ✅ Merged **Merged:** 10/8/2026 **Merged by:** [@alexisrolland](https://github.com/alexisrolland) **Base:** `run-index` ← **Head:** `goldens-v2` --- ### 📝 Commits (1) - [`37aa1cc`](https://github.com/Comfy-Org/comfyci-runpod-worker/commit/37aa1ccb4724e07314599cbef552c7ada05f85c9) Record why a golden was blessed and allow promoting an existing run ### 📊 Changes **5 files changed** (+455 additions, -42 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/golden-baseline.yml` (+47 -10) 📝 `README.md` (+25 -1) 📝 `scripts/golden_baseline.py` (+192 -23) 📝 `scripts/index_builder.py` (+11 -8) ➕ `scripts/tests/test_golden_baseline.py` (+180 -0) </details> ### 📄 Description ## Why Accepting an intentional numerics change (the #16488 adaln rounding-order change that moved every FLUX pixel by a rounding error) means re-blessing the golden. Nothing recorded who accepted what and why, and generating a golden always cost two GPU runs even when the accepted output already existed on the results branch. ## What - `golden_baseline.py`: `--reason`, `--lane` (selects the lane's storage prefix), `--from-run BRANCH/<sha>`, `--force`, and a full commit sha accepted as `--ref`. `blessed.json` v2 records lane, python and cuda (driver and image identity once the worker reports them), the golden's output sha, the superseded tag and the reason; `current.json` carries `reason`, `supersedes`, `output_sha256`; `golden/<wf>/history.json` keeps every bless. `--from-run` promotes an existing run's outputs without GPU time, names the golden after that commit (a `--ref` must resolve to it), and inherits the previous golden's noise floor marked `measured: false`. An existing golden directory is never overwritten without `--force`; bless-only checks every workflow before blessing any. Blessing and the run-index refresh run as derived writers, so a publish that retries behind the regression cron re-applies them on the fresh tree, and the dashboard stops reporting accepted drift as an open regression in the same publish. - `golden-baseline.yml`: `from_run`, `reason`, `lane`, `force` inputs; every input reaches the script through the environment instead of being interpolated into the shell; own concurrency group; timeout. - README: re-bless flow. ## Verification - `python -m pytest scripts/tests -q` (helpers, promote + bless + history on a real git remote, the from-run command-line flow end to end including a reason with quotes and `$`, refusal to overwrite without `--force`, ref/shape validation, bless-only validation before any bless). - The FLUX golden candidate at v0.38.0 was generated with the existing workflow (run 37243296184): both runs produced `3fa6e2ad…`, identical noise floor. Blessing it with a reason after the maintainer ack is the first use of this flow. Stacked on #3. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 22:06:10 +00:00