mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-05-04 21:22:07 +00:00
a7218b2922
## Summary Fixes three critical issues with the CI performance reporting pipeline that made perf reports useless on PRs (demonstrated by PR #9248 — deep watcher removal merged without useful perf signal). ## Changes ### 1. Fix z-score baseline variance collection (`0/5 runs`) **Root cause:** PR #9305 added z-score statistical analysis code to `perf-report.ts`, but the historical data download step was placed in the wrong workflow file. The report is generated in `pr-perf-report.yaml` (a `workflow_run`-triggered job), but the historical download was in `ci-perf-report.yaml` (the test runner) — different runners, different filesystems. **Fix:** Implement `perf-data` orphan branch storage: - On push to main: save `perf-metrics.json` to `perf-data` branch with timestamped filename - On PR report: fetch last 5 baselines from `perf-data` branch into `temp/perf-history/` - Rolling window of 20 baselines, oldest pruned automatically - Same pattern used by `github-action-benchmark` (33.7k repos) ### 2. Fix force-push comment staleness **Root cause:** `cancel-in-progress: true` kills the perf test run before it uploads artifacts. The downstream report workflow only triggers on `conclusion == 'success'` — cancelled runs are ignored, so the comment from the first successful run goes stale. **Fix:** - Change `cancel-in-progress: false` — with GitHub's queue depth of 1, rapid pushes (A,B,C,D) run A and D, skipping B and C - Add SHA validation in `pr-perf-report.yaml` — before posting, check if the workflow_run's head SHA still matches the PR's current head. Skip posting stale results. ### 3. Add permissions for baseline operations - `contents: write` on CI job (needed for pushing to perf-data branch) - `actions: read` on both workflows (needed for artifact/baseline access) ## One-time setup required After merging, create the `perf-data` orphan branch: ```bash git checkout --orphan perf-data git rm -rf . echo '# Performance Baselines' > README.md mkdir -p baselines git add README.md baselines git commit -m 'Initialize perf-data branch' git push origin perf-data ``` The first 2 pushes to main after setup will build up variance data, and z-scores will start appearing in PR reports (threshold is `historical.length >= 2`). ## Testing - YAML validated with `yaml.safe_load()` - `perf-report.ts` `loadHistoricalReports()` already reads from `temp/perf-history/<index>/perf-metrics.json` — no code changes needed - All new steps use `continue-on-error: true` for graceful degradation ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-9886-fix-fix-perf-CI-pipeline-z-score-baselines-force-push-staleness-baseline-storage-3226d73d365081538424c7945e71f308) by [Unito](https://www.unito.io)
GitHub Workflows
Naming Convention
Workflow files follow a consistent naming pattern: <prefix>-<descriptive-name>.yaml
Category Prefixes
| Prefix | Purpose | Example |
|---|---|---|
ci- |
Testing, linting, validation | ci-tests-e2e.yaml |
release- |
Version management, publishing | release-version-bump.yaml |
pr- |
PR automation (triggered by labels) | pr-claude-review.yaml |
api- |
External Api type generation | api-update-registry-api-types.yaml |
i18n- |
Internationalization updates | i18n-update-core.yaml |
Documentation
Each workflow file contains comments explaining its purpose, triggers, and behavior. For specific details about what each workflow does, refer to the comments at the top of each .yaml file.
For GitHub Actions documentation, see Events that trigger workflows.