feat: add performance testing infrastructure with CDP metrics (#9170)

## Summary

Add a permanent, non-failing performance regression detection system
using Chrome DevTools Protocol metrics, with automatic PR commenting.

## Changes

- **What**: Performance testing infrastructure — `PerformanceHelper`
fixture class using CDP `Performance.getMetrics` to collect
`RecalcStyleCount`, `LayoutCount`, `LayoutDuration`, `TaskDuration`,
`JSHeapUsedSize`. Adds `@perf` Playwright project (Chromium-only,
single-threaded, 60s timeout), 4 baseline perf tests, CI workflow with
sticky PR comment reporting, and `perf-report.js` script for generating
markdown comparison tables.

## Review Focus

- `PerformanceHelper` uses `page.context().newCDPSession(page)` — CDP is
Chromium-only, so perf metrics are not collected on Firefox. This is
intentional since CDP gives us browser-level style recalc/layout counts
that `performance.mark/measure` cannot capture.
- The CI workflow uses `continue-on-error: true` so perf tests never
block merging.
- Baseline comparison uses `dawidd6/action-download-artifact` to
download metrics from the target branch, following the same pattern as
`pr-size-report.yaml`.

## Stack

This is the foundation PR for the Firefox performance fix stack:
1. **→ This PR: perf testing infrastructure**
2. `perf/fix-cursor-cache` — cursor style caching (depends on this)
3. `perf/fix-subgraph-svg` — SVG pre-rasterization (depends on this)
4. `perf/fix-clippath-raf` — RAF batching for clip-path (depends on
this)

PRs 2-4 are independent of each other.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9170-feat-add-performance-testing-infrastructure-with-CDP-metrics-3116d73d3650817cb43def6f8e9917f8)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
This commit is contained in:
Christian Byrne
2026-02-25 20:09:57 -08:00
committed by GitHub
parent c957841862
commit 8e215b3174
8 changed files with 474 additions and 1 deletions

110
.github/workflows/ci-perf-report.yaml vendored Normal file
View File

@@ -0,0 +1,110 @@
name: 'CI: Performance Report'
on:
push:
branches: [main, core/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
concurrency:
group: perf-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
jobs:
perf-tests:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
timeout-minutes: 30
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.12
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
permissions:
contents: read
packages: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
with:
include_build_step: true
- name: Start ComfyUI server
uses: ./.github/actions/start-comfyui-server
- name: Run performance tests
id: perf
continue-on-error: true
run: pnpm exec playwright test --project=performance --workers=1
- name: Upload perf metrics
if: always()
uses: actions/upload-artifact@v6
with:
name: perf-metrics
path: test-results/perf-metrics.json
retention-days: 30
if-no-files-found: warn
report:
needs: perf-tests
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 22
- name: Download PR perf metrics
continue-on-error: true
uses: actions/download-artifact@v7
with:
name: perf-metrics
path: test-results/
- name: Download baseline perf metrics
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
branch: ${{ github.event.pull_request.base.ref }}
workflow: ci-perf-report.yaml
event: push
name: perf-metrics
path: temp/perf-baseline/
if_no_artifact_found: warn
- name: Generate perf report
run: npx --yes tsx scripts/perf-report.ts > perf-report.md
- name: Read perf report
id: perf-report
uses: juliangruber/read-file-action@b549046febe0fe86f8cb4f93c24e284433f9ab58 # v1.1.7
with:
path: ./perf-report.md
- name: Create or update PR comment
uses: actions-cool/maintain-one-comment@4b2dbf086015f892dcb5e8c1106f5fccd6c1476b # v3.2.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
number: ${{ github.event.pull_request.number }}
body: |
${{ steps.perf-report.outputs.content }}
<!-- COMFYUI_FRONTEND_PERF -->
body-include: '<!-- COMFYUI_FRONTEND_PERF -->'