Commit Graph

4 Commits

Author SHA1 Message Date
Christian Byrne
033b3dad3a feat: add Slack notification workflow for coverage improvements (#10977)
## Summary

Adds a GitHub Actions workflow + TypeScript script that posts to Slack
when a merged PR improves unit or E2E test coverage.

## Changes

- **What**: New `coverage-slack-notify.yaml` workflow triggered on push
to main. Compares current coverage against previous baselines, generates
Slack Block Kit payload with progress bars and milestone celebrations,
posts to `#p-frontend-automated-testing`.
- **Script**: `scripts/coverage-slack-notify.ts` — parses lcov files,
computes deltas, detects milestone crossings (every 5%), builds Slack
payload. Pure functions exported for testability.
- **Tests**: 26 unit tests in `scripts/coverage-slack-notify.test.ts`
covering all pure functions including edge cases (malformed lcov, exact
boundaries, zero coverage).

### Security hardening
- All `${{ }}` expressions moved from `run:` blocks to `env:` variables
- `SLACK_BOT_TOKEN` passed via env var, not inline
- Unique heredoc delimiter (timestamp-based) prevents payload injection
- `parseInt` fallback (`|| 0`) guards against malformed lcov
- PR regex anchored to first line of commit message

### Robustness
- `continue-on-error: true` on Slack post step (outage does not fail the
job)
- Baseline save guarded by `steps.unit-tests.outcome == success`
(prevents corrupt baselines on test failure)
- Channel ID commented for maintainability
- Top-level `text` field added for Slack mobile push notifications
- Author linked to GitHub profile instead of bare `@username`

## Review Focus

- Workflow step ordering and conditional logic
- Security of expression handling and secret management
- Slack payload structure and Block Kit formatting

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10977-feat-add-Slack-notification-workflow-for-coverage-improvements-33d6d73d3650819c8950f483c83f297c)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-04-14 20:58:47 -07:00
Christian Byrne
437f41c553 perf: add layout/GC metrics + reduce false positives in regression detection (#10477)
## Summary

Add layout duration, style recalc duration, and heap usage metrics to CI
perf reports, while improving statistical reliability to reduce false
positive regressions.

## Changes

- **What**:
- Collect `layoutDurationMs`, `styleRecalcDurationMs`, `heapUsedBytes`
(absolute snapshot) alongside existing metrics
- Add effect size gate (`minAbsDelta`) for integer-quantized count
metrics (style recalcs, layouts, DOM nodes, event listeners) — prevents
z=7.2 false positives from e.g. 11→12 style recalcs
- Switch from mean to **median** for PR metric aggregation — robust to
outlier CI runs that dominate n=3 mean
- Increase historical baseline window from **5 to 15 runs** for more
stable σ estimates
- Reorder reported metrics: layout/style duration first (actionable),
counts and heap after (informational)

## Review Focus

The effect size gate in `classifyChange()` — it now requires both z > 2
AND absolute delta ≥ `minAbsDelta` (when configured) to flag a
regression. This addresses the core false positive issue where integer
metrics with near-zero historical variance produce extreme z-scores for
trivial changes.

Median vs mean tradeoff: median is more robust to outliers but less
sensitive to real shifts — acceptable given n=3 and CI noise levels.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10477-perf-add-layout-GC-metrics-reduce-false-positives-in-regression-detection-32d6d73d365081daa72cec96d8a07b90)
by [Unito](https://www.unito.io)
2026-03-25 10:16:56 -07:00
Christian Byrne
2af3940867 feat: add trend visualization with sparklines to perf report (#9939)
## Summary

Add historical trend visualization (ASCII sparklines + directional
arrows) to the performance PR report, showing how each metric has moved
over recent commits on main.

## Changes

- **What**: New `sparkline()`, `trendDirection()`, `trendArrow()`
functions in `perf-stats.ts`. New collapsible "Trend" section in the
perf report showing per-metric sparklines, direction indicators, and
latest values. CI workflow updated to download historical data from the
`perf-data` orphan branch and switched to `setup-frontend` action with
`pnpm exec tsx`.

## Review Focus

- The trend section only renders when ≥3 historical data points exist
(gracefully absent otherwise)
- `trendDirection()` uses a split-half mean comparison with ±10%
threshold — review whether this sensitivity is appropriate
- The `git archive` step in `pr-perf-report.yaml` is idempotent and
fails silently if no perf-history data exists yet on the perf-data
branch

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9939-feat-add-trend-visualization-with-sparklines-to-perf-report-3246d73d36508125a6fcc39612f850fe)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-17 07:10:30 -07:00
Christian Byrne
54fe02bdf1 feat: unified PR report combining bundle size and runtime perf (#9911)
## Summary

Replaces two separate PR comment workflows (bundle size + performance)
with a single unified report that posts one combined comment per PR.

## Changes

- **What**: New `pr-report.yaml` aggregator workflow triggers on both
`CI: Size Data` and `CI: Performance Report` completions. Finds sibling
workflow runs by PR head SHA. Renders combined report via
`unified-report.js` (shells out to existing `size-report.js` and
`perf-report.ts`). Sections show "pending" or "failed" placeholders when
data is unavailable.
- **Breaking**: Removes `pr-size-report.yaml` and `pr-perf-report.yaml`.
Legacy `<!-- COMFYUI_FRONTEND_SIZE -->` and `<!-- COMFYUI_FRONTEND_PERF
-->` comments are auto-cleaned on first run.
- **Dependencies**: None

## Review Focus

- Concurrency key uses `head_sha` so the later-completing workflow
cancels the earlier report run, ensuring the final comment always has
both sections.
- Stale-run guard: verifies workflow_run SHA matches the live PR head
before posting.
- The `workflow_dispatch` re-trigger path from `pr-size-report.yaml` is
not carried forward — the unified workflow handles re-trigger naturally
via its dual-trigger design.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9911-feat-unified-PR-report-combining-bundle-size-and-runtime-perf-3236d73d365081baac1cce6f0d9244ac)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-17 03:02:58 -07:00