mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-03-07 22:20:03 +00:00
7b316eb9a2
## Summary Replace fixed 10%/20% perf delta thresholds with dynamic σ-based classification using z-scores, eliminating false alarms from naturally noisy duration metrics (10-17% CV). ## Changes - **What**: - Run each perf test 3× (`--repeat-each=3`) and report the mean, reducing single-run noise - Download last 5 successful main branch perf artifacts to compute historical μ/σ per metric - Replace fixed threshold flags with z-score significance: `⚠️ regression` (z>2), `✅ neutral/improvement`, `🔇 noisy` (CV>50%) - Add collapsible historical variance table (μ, σ, CV) to PR comment - Graceful cold start: falls back to simple delta table until ≥2 historical runs exist - New `scripts/perf-stats.ts` module with `computeStats`, `zScore`, `classifyChange` - 18 unit tests for stats functions - **CI time impact**: ~3 min → ~5-6 min (repeat-each adds ~2 min, historical download <10s) ## Review Focus - The `gh api` call in the new "Download historical perf baselines" step: it queries the last 5 successful push runs on the base branch. The `gh` CLI is available natively on `ubuntu-latest` runners and auto-authenticates with `GITHUB_TOKEN`. - `getHistoricalStats` averages per-run measurements before computing cross-run σ — this is intentional since historical artifacts may also contain repeated measurements after this change lands. - The `noisy` classification (CV>50%) suppresses metrics like `layouts` that hover near 0 and have meaningless percentage swings. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-9305-feat-add-statistical-significance-to-perf-report-with-z-score-thresholds-3156d73d3650818d9360eeafd9ae7dc1) by [Unito](https://www.unito.io)
GitHub Workflows
Naming Convention
Workflow files follow a consistent naming pattern: <prefix>-<descriptive-name>.yaml
Category Prefixes
| Prefix | Purpose | Example |
|---|---|---|
ci- |
Testing, linting, validation | ci-tests-e2e.yaml |
release- |
Version management, publishing | release-version-bump.yaml |
pr- |
PR automation (triggered by labels) | pr-claude-review.yaml |
api- |
External Api type generation | api-update-registry-api-types.yaml |
i18n- |
Internationalization updates | i18n-update-core.yaml |
Documentation
Each workflow file contains comments explaining its purpose, triggers, and behavior. For specific details about what each workflow does, refer to the comments at the top of each .yaml file.
For GitHub Actions documentation, see Events that trigger workflows.