mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-05-04 05:02:17 +00:00
perf: add layout/GC metrics + reduce false positives in regression detection (#10477)
## Summary Add layout duration, style recalc duration, and heap usage metrics to CI perf reports, while improving statistical reliability to reduce false positive regressions. ## Changes - **What**: - Collect `layoutDurationMs`, `styleRecalcDurationMs`, `heapUsedBytes` (absolute snapshot) alongside existing metrics - Add effect size gate (`minAbsDelta`) for integer-quantized count metrics (style recalcs, layouts, DOM nodes, event listeners) — prevents z=7.2 false positives from e.g. 11→12 style recalcs - Switch from mean to **median** for PR metric aggregation — robust to outlier CI runs that dominate n=3 mean - Increase historical baseline window from **5 to 15 runs** for more stable σ estimates - Reorder reported metrics: layout/style duration first (actionable), counts and heap after (informational) ## Review Focus The effect size gate in `classifyChange()` — it now requires both z > 2 AND absolute delta ≥ `minAbsDelta` (when configured) to flag a regression. This addresses the core false positive issue where integer metrics with near-zero historical variance produce extreme z-scores for trivial changes. Median vs mean tradeoff: median is more robust to outliers but less sensitive to real shifts — acceptable given n=3 and CI noise levels. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-10477-perf-add-layout-GC-metrics-reduce-false-positives-in-regression-detection-32d6d73d365081daa72cec96d8a07b90) by [Unito](https://www.unito.io)
This commit is contained in:
2
.github/workflows/pr-report.yaml
vendored
2
.github/workflows/pr-report.yaml
vendored
@@ -180,7 +180,7 @@ jobs:
|
||||
if git ls-remote --exit-code origin perf-data >/dev/null 2>&1; then
|
||||
git fetch origin perf-data --depth=1
|
||||
mkdir -p temp/perf-history
|
||||
for file in $(git ls-tree --name-only origin/perf-data baselines/ 2>/dev/null | sort -r | head -10); do
|
||||
for file in $(git ls-tree --name-only origin/perf-data baselines/ 2>/dev/null | sort -r | head -15); do
|
||||
git show "origin/perf-data:${file}" > "temp/perf-history/$(basename "$file")" 2>/dev/null || true
|
||||
done
|
||||
echo "Loaded $(ls temp/perf-history/*.json 2>/dev/null | wc -l) historical baselines"
|
||||
|
||||
Reference in New Issue
Block a user