Commit Graph

323 Commits

Author SHA1 Message Date
snomiao
224c845b6c fix: tighten focused QA prompt to only test PR-specific behavior
The Codex agent was spending time on login flow, template browsing,
and general smoke testing instead of testing the PR's actual changes.

Changes:
- Add 30-second time budget for video recording
- Move video-start AFTER login and editor verification
- Explicitly prohibit template browsing and sidebar exploration
- Reduce test steps to 3-6 targeted actions
- Restructure prompt with clear Instructions/Rules sections
2026-03-19 14:34:01 +00:00
snomiao
d4f22467c0 fix: render markdown in QA reports with marked.js
Replace crude sed-based markdown conversion with client-side
rendering via marked.js CDN. Adds proper table, list, and
code styling for the report section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:11:10 +00:00
snomiao
b764c39492 fix: run report job on workflow_dispatch events
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 10:08:46 +00:00
snomiao
8b2db18a1b refactor: replace GPT frame extraction with Gemini native video analysis
Replace the OpenAI GPT-based frame extraction approach (ffmpeg + screenshots)
with Gemini 2.5 Flash's native video understanding. This eliminates false
positives from frame-based analysis (e.g. "black screen = critical bug" during
page transitions) and produces dramatically better QA reviews.

Changes:
- Remove ffmpeg frame extraction, ffprobe duration detection, and all related
  logic (~365 lines removed)
- Add @google/generative-ai SDK for native video/mp4 upload to Gemini
- Update CLI: remove --max-frames, --min-interval-seconds, --keep-frames flags
- Update env: OPENAI_API_KEY → GEMINI_API_KEY
- Update workflow: swap API key secret and model in pr-qa.yaml
- Update report: replace "Frames analyzed" with "Video size"
- Add note in prompt that brief black frames during transitions are normal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 08:33:07 +00:00
snomiao
0b3f6c9bdc fix: use fill+click for quick login before video recording
playwright-cli doesn't support 'evaluate' command. Instead, instruct
Codex to quickly fill the username input and click Next on user-select
page BEFORE starting video recording, so the video only shows actual
QA testing.
2026-03-18 23:21:41 +00:00
snomiao
2f88a07c75 fix: use evaluate to set localStorage before video recording
storageState config doesn't work with playwright-cli. Instead, use
evaluate to set Comfy.userId/userName after opening the page, then
navigate back. This skips user-select before video-start so the
recording only shows actual QA testing.
2026-03-18 23:08:05 +00:00
snomiao
ac7158f25e fix: pre-seed localStorage to skip user-select in QA runs
Write a Playwright storageState JSON with Comfy.userId/userName pre-set
so the app loads directly to the graph editor. Saves ~40s per QA run
that was wasted on navigating the user-select page.
2026-03-18 22:54:41 +00:00
snomiao
3b4402c2d1 fix: prefer explicit qa-session.webm over corrupt auto-recorded videos
The convert step was using find which picked up a 0-byte file from
playwright's videos/ directory instead of the valid qa-session.webm.
Now prefers qa-session.webm explicitly and skips empty files.
2026-03-18 22:31:54 +00:00
snomiao
8d50345193 fix: improve focused QA prompt to test PR-specific behavior, not random walk 2026-03-18 22:10:17 +00:00
snomiao
b00e58e513 fix: re-add push trigger for sno-skills and sno-qa-* branches 2026-03-18 21:27:36 +00:00
snomiao
f4148d13f1 fix: also install ffprobe for GPT video review frame extraction 2026-03-18 21:13:48 +00:00
snomiao
438af97b3d fix: use sudo for ffmpeg static binary extraction to /usr/local/bin 2026-03-18 20:58:55 +00:00
snomiao
7e756ec673 fix: use static ffmpeg binary instead of apt-get (avoids dpkg lock hang) 2026-03-18 20:33:50 +00:00
snomiao
87a4316125 fix: add ffmpeg install back (not pre-installed on GH runners) 2026-03-18 20:16:33 +00:00
snomiao
52a825f631 fix: normalize flat artifact download into expected subdirectory 2026-03-18 20:07:58 +00:00
snomiao
7bde727c7b fix: pre-install chromium and clarify prompt for codex
Codex was using pnpm dlx instead of the global playwright-cli.
Pre-install chromium in setup step and make prompt explicit about
using the global command directly without pnpm/npx.
2026-03-18 19:39:52 +00:00
snomiao
dea1c2c421 fix: add debug output to video convert step 2026-03-18 19:10:47 +00:00
snomiao
7549ab4807 fix: use danger-full-access sandbox for codex on GH Actions 2026-03-18 18:58:46 +00:00
snomiao
3ca4a144ba fix: use correct codex model name gpt-5.4-mini 2026-03-18 18:50:41 +00:00
snomiao
34c6e15678 feat: switch QA from Claude Code to OpenAI Codex CLI
Replace claude --print with codex exec for cheaper QA runs.
Uses codex-mini-latest model ($1.50/$6 vs Sonnet $3/$15).
Uses existing OPENAI_API_KEY secret (no new secrets needed).
2026-03-18 18:45:05 +00:00
snomiao
799fc9e0bc fix: default to linux-only QA, full 3-OS only via qa-full label
Reduces per-run cost from ~$10-16 to ~$2.50 by defaulting to
Linux-only. Use qa-full label or workflow_dispatch for 3-OS runs.
2026-03-18 18:27:57 +00:00
snomiao
c7844f3c74 fix: use explicit video-start/stop, remove ffmpeg install, use gpt-4.1-mini
- Replace saveVideo config (didn't produce video) with explicit
  playwright-cli video-start/video-stop commands in QA prompt
- Remove apt-get install ffmpeg step (pre-installed on GH runners)
- Switch video review model from gpt-4o to gpt-4.1-mini
2026-03-18 18:18:18 +00:00
snomiao
eec387ea49 fix: use auto video recording and show GPT reports on QA site
- Enable saveVideo in playwright-cli config for real video recording
- Replace screenshot stitching with webm→mp4 conversion
- Move video review step before deploy so reports are included
- Add GPT video review reports inline on the Cloudflare Pages site
- Each video card now has expandable "GPT Video Review" section
2026-03-18 17:20:55 +00:00
snomiao
9cbbc05042 fix: configure playwright-cli outputDir and improve artifact collection
- Set .playwright/cli.config.json with outputDir pointing to screenshots/
- This way bare 'playwright-cli screenshot' auto-saves to the right place
- Create screenshot directory before Claude runs (don't rely on Claude)
- Collect step now searches working directory for stray PNGs
- Simplified prompt: no --filename needed, just 'playwright-cli screenshot'
2026-03-18 16:42:30 +00:00
snomiao
cd491b19d3 fix: stitch screenshots from correct directory and simplify prompt
Screenshots were saved to artifact root but stitch looked in frames/.
Now: prompt tells Claude to save to screenshots/ dir with numbered names,
collect step consolidates PNGs there, stitch step globs from screenshots/.
Removed video-start/video-stop (Claude doesn't use them).
2026-03-18 15:50:34 +00:00
snomiao
11575306cc fix: use playwright-cli video recording and collect default output
- Add playwright-cli config with outputDir and saveVideo
- Use video-start/video-stop instead of relying on screenshot frames
- Add fallback artifact collection from .playwright-cli/ default dir
- Simplify prompts to focus on video recording workflow
2026-03-18 15:14:09 +00:00
snomiao
a7d600242b fix: resolve QA_ARTIFACTS path in prompt so Claude gets the literal path
The escaped \$QA_ARTIFACTS in the heredoc produced literal text
'$QA_ARTIFACTS' in the prompt. Claude's Bash tool didn't reliably
expand this env var, so no screenshots or reports were saved.
Remove the escapes so the heredoc expands the variable to the actual
path (e.g. /home/runner/work/_temp/qa-artifacts).
2026-03-18 14:22:29 +00:00
snomiao
f8bdda585a fix: escape backticks in QA prompt heredoc to prevent command substitution
Backtick-wrapped playwright-cli examples in the unquoted heredoc were
being interpreted as bash command substitution, producing empty prompts.
Replace backtick syntax with plain "Run:" prefixed commands.
2026-03-18 13:48:06 +00:00
snomiao
31675b62cc fix: reorganize QA CI — remove screen recording, merge video-review into report
- Remove all Xvfb/ffmpeg screen recording infrastructure from qa job
  (captured blank display since playwright-cli runs headless)
- Add screenshot instructions to QA prompts: Claude saves sequential
  frames to $QA_ARTIFACTS/frames/ after every interaction
- Stitch screenshots into video via ffmpeg in report job (2fps)
- Merge video-review job into report job (4 jobs → 3 jobs)
- Unified PR comment with video links + video review in <details> collapse
- Clean up stale QA_VIDEO_REVIEW_COMMENT markers from prior runs
2026-03-18 09:51:53 +00:00
snomiao
7417aee87d fix: harden setup-comfyui-server against shell injection
Move extra_server_params input to env var to prevent shell injection
from untrusted input. Replace wait-for-it pip dependency with a
cross-platform curl polling loop.
2026-03-17 06:27:51 +00:00
snomiao
f14252accb feat: add comfy-qa skill and automated QA CI pipeline
Add Claude Code skills and a label-triggered QA workflow:

- .claude/skills/comfy-qa/SKILL.md: 12-category QA test plan using
  playwright-cli for browser automation
- .github/workflows/pr-qa.yaml: CI workflow triggered by qa-changes
  (focused, Linux) or qa-full (3-OS matrix) labels. Records screen via
  ffmpeg, runs Claude CLI with playwright-cli, deploys video gallery to
  Cloudflare Pages, posts PR comment with GIF thumbnails, and runs
  OpenAI vision-based video review
- scripts/qa-video-review.ts: frame extraction + GPT-4o analysis
- scripts/qa-video-review.test.ts: unit tests for video review
- knip.config.ts: resolve knip errors for ingest-types package
2026-03-17 06:25:44 +00:00
Christian Byrne
46dad2e077 ops: restrict PyPI publishing to bi-weekly ComfyUI releases (#9948)
## Summary

Restrict PyPI publishing of `comfyui-frontend-package` to bi-weekly
ComfyUI release cycles only, instead of every nightly version bump.

## Changes

- **What**: Move `publish_pypi` job from `release-draft-create.yaml` to
`release-biweekly-comfyui.yaml`
1. Removed `publish_pypi` job from `release-draft-create.yaml` (no
longer publishes on every merged Release PR)
2. Added `publish-pypi` job to `release-biweekly-comfyui.yaml` with tag
polling, build, publish, and PyPI availability confirmation
3. Gated `create-comfyui-pr` on `publish-pypi` success so the ComfyUI
requirements bump PR is only created after the package is confirmed
available
4. Updated ComfyUI PR body to confirm PyPI availability instead of
warning about a pending release PR
- **Breaking**: None — nightly releases still create GitHub releases and
publish npm types; only PyPI publishing timing changes
- **Dependencies**: None

## Review Focus

- The `publish-pypi` job uses `if: always() &&
needs.resolve-version.result == 'success'` to run even when
`trigger-release-if-needed` is skipped (tag already exists)
- Tag polling (30min timeout) waits for the version bump PR to be merged
before building from the tagged commit
- PyPI propagation polling (15min timeout) confirms the package is
installable before creating the ComfyUI PR

Fixes COM-16778

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9948-ops-restrict-PyPI-publishing-to-bi-weekly-ComfyUI-releases-3246d73d36508198b00fcc247ac5b58c)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-16 13:11:15 -07:00
Johnpaul Chiwetelu
f68d8365a6 chore: enable auto-merge on backport PRs (#10108)
## Summary
- Adds `gh pr merge --auto --squash` after backport PR creation in the
backport workflow, so backport PRs merge automatically once checks pass
- Uses `|| echo "::warning::..."` fallback to avoid failing the workflow
if auto-merge can't be enabled (e.g. repo setting not configured)

## Test plan
- [ ] Trigger backport workflow on a test PR with `needs-backport` label
- [ ] Verify auto-merge is enabled on the created backport PR

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10108-chore-enable-auto-merge-on-backport-PRs-3256d73d3650814eb6e5fb2bdf3c5ec7)
by [Unito](https://www.unito.io)
2026-03-16 12:57:48 -07:00
Johnpaul Chiwetelu
ed5e0a0b51 chore: replace team CODEOWNERS with external PR review workflow (#10104)
## Summary

Remove team assignments from CODEOWNERS to reduce notification noise for
internal PRs. Add a workflow that requests team review only when
external contributors open PRs.

## Changes

- **What**: Strip `@Comfy-org/comfy_frontend_devs` and
`@Comfy-Org/comfy_maintainer` from all CODEOWNERS entries (keep
individual user assignments). Add `pr-request-team-review.yaml` workflow
that uses `pull_request_target` to request team review for
non-collaborator PRs.
- **Dependencies**: None

## Review Focus

- The workflow uses `pull_request_target` but does not check out or
execute any untrusted code — it only runs `gh pr edit --add-reviewer`.
- The `author_association` check excludes OWNER, MEMBER, and
COLLABORATOR — internal PRs will not trigger team review requests.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10104-chore-replace-team-CODEOWNERS-with-external-PR-review-workflow-3256d73d3650813b887ac16b5e97b4c4)
by [Unito](https://www.unito.io)
2026-03-16 15:49:58 +01:00
Christian Byrne
a7218b2922 fix: fix perf CI pipeline — z-score baselines, force-push staleness, baseline storage (#9886)
## Summary

Fixes three critical issues with the CI performance reporting pipeline
that made perf reports useless on PRs (demonstrated by PR #9248 — deep
watcher removal merged without useful perf signal).

## Changes

### 1. Fix z-score baseline variance collection (`0/5 runs`)
**Root cause:** PR #9305 added z-score statistical analysis code to
`perf-report.ts`, but the historical data download step was placed in
the wrong workflow file. The report is generated in
`pr-perf-report.yaml` (a `workflow_run`-triggered job), but the
historical download was in `ci-perf-report.yaml` (the test runner) —
different runners, different filesystems.

**Fix:** Implement `perf-data` orphan branch storage:
- On push to main: save `perf-metrics.json` to `perf-data` branch with
timestamped filename
- On PR report: fetch last 5 baselines from `perf-data` branch into
`temp/perf-history/`
- Rolling window of 20 baselines, oldest pruned automatically
- Same pattern used by `github-action-benchmark` (33.7k repos)

### 2. Fix force-push comment staleness
**Root cause:** `cancel-in-progress: true` kills the perf test run
before it uploads artifacts. The downstream report workflow only
triggers on `conclusion == 'success'` — cancelled runs are ignored, so
the comment from the first successful run goes stale.

**Fix:**
- Change `cancel-in-progress: false` — with GitHub's queue depth of 1,
rapid pushes (A,B,C,D) run A and D, skipping B and C
- Add SHA validation in `pr-perf-report.yaml` — before posting, check if
the workflow_run's head SHA still matches the PR's current head. Skip
posting stale results.

### 3. Add permissions for baseline operations
- `contents: write` on CI job (needed for pushing to perf-data branch)
- `actions: read` on both workflows (needed for artifact/baseline
access)

## One-time setup required
After merging, create the `perf-data` orphan branch:
```bash
git checkout --orphan perf-data
git rm -rf .
echo '# Performance Baselines' > README.md
mkdir -p baselines
git add README.md baselines
git commit -m 'Initialize perf-data branch'
git push origin perf-data
```

The first 2 pushes to main after setup will build up variance data, and
z-scores will start appearing in PR reports (threshold is
`historical.length >= 2`).

## Testing
- YAML validated with `yaml.safe_load()`
- `perf-report.ts` `loadHistoricalReports()` already reads from
`temp/perf-history/<index>/perf-metrics.json` — no code changes needed
- All new steps use `continue-on-error: true` for graceful degradation

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9886-fix-fix-perf-CI-pipeline-z-score-baselines-force-push-staleness-baseline-storage-3226d73d365081538424c7945e71f308)
by [Unito](https://www.unito.io)
2026-03-15 02:46:10 -07:00
Christian Byrne
4bdf67ca21 fix: restore fork PR lint/format CI workflow (#9846)
## Problem

The lint/format CI workflow was broken for fork PRs in two ways:

### 1. Node version mismatch in setup-frontend action
The `setup-frontend` shared action (created in #8377) was missed when
Node version was standardized to `.nvmrc` in #9521. It still used
`node-version: 'lts/*'` instead of `node-version-file: '.nvmrc'`.

### 2. Fork PRs with lint issues silently passed CI
Fork PRs with auto-fixable lint/format issues got a **green checkmark**
despite having unfixed issues:
1. Auto-fix steps (`lint:fix`, `format`) fix issues in the workspace
2. `Commit changes` is correctly skipped for forks (can't push to fork
branches)
3. `Final validation` passes because it runs on the already-fixed
workspace
4. The `Comment on PR about manual fix needed` step tries to post a
comment via `actions/github-script`, but fork PRs have a read-only
`GITHUB_TOKEN` — the comment silently fails (`continue-on-error: true`)
5. **Result**: workflow reports success, contributor thinks their code
is clean

## Fix

- **setup-frontend**: Use `node-version-file: '.nvmrc'` instead of
`node-version: 'lts/*'`
- **ci-lint-format**: Replace the broken fork comment step with an
explicit `exit 1` that fails CI and prints clear fix instructions in the
log. This follows the principle from `.github/AGENTS.md`: fork PRs can't
post comments, so don't try.

## Testing
- [ ] Verify fork PRs with clean code still pass
- [ ] Verify fork PRs with lint issues now properly fail (instead of
silently passing)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9846-fix-restore-fork-PR-lint-format-CI-workflow-3226d73d3650811cb5bfe9f1f989cc0c)
by [Unito](https://www.unito.io)
2026-03-13 06:14:16 -07:00
Dante
08a2f8ae15 ci: deploy Storybook to fixed production URL on main merge (#9373)
## Summary
- Add `deploy-production` job to Storybook CI workflow
- Triggers on push to `main` branch
- Deploys to fixed Cloudflare Pages URL:
`https://comfy-storybook.pages.dev`
- PR preview deployments remain unchanged

## Linked Issues
- Notion:
[COM-15826](https://www.notion.so/Deploy-Storybook-to-fixed-production-URL-on-main-branch-merge-3196d73d36508159a875d42694a619d1)

## Test Plan
- [ ] Merge to main and verify `deploy-production` job runs in GitHub
Actions
- [ ] Confirm `https://comfy-storybook.pages.dev` serves latest
Storybook
- [ ] Verify existing PR preview deployments still work
- [ ] Verify other jobs (comment, chromatic) are not affected by the new
trigger

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9373-ci-deploy-Storybook-to-fixed-production-URL-on-main-merge-3196d73d3650816795ead1a5b839a571)
by [Unito](https://www.unito.io)
2026-03-11 16:38:29 +01:00
Hunter
c084605e4d fix: default frontend preview variant to cpu (#9718)
Frontend previews don't need GPU resources. Default to cpu variant and
only use gpu when the `preview-gpu` label is explicitly added.

The plain `preview` label now deploys a cpu-only ephemeral env.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9718-fix-default-frontend-preview-variant-to-cpu-31f6d73d3650811f878cd5dd5ad3881c)
by [Unito](https://www.unito.io)
2026-03-10 15:31:03 -04:00
Hunter
b368a865cf feat: dispatch frontend PR preview environments to cloud (#9715)
## Summary

Add support for deploying full ephemeral preview environments from
frontend PRs. This is the frontend-side half — it sends `pr_number` and
`variant` (cpu/gpu) in the dispatch payload, and adds a cleanup dispatch
on PR close/unlabel.

### Changes

- **`cloud-dispatch-build.yaml`** — Add `pr_number` and `variant` to the
`frontend-asset-build` dispatch payload. Variant is derived from which
preview label triggered the event (`preview-cpu` → cpu, else gpu).

- **`cloud-dispatch-cleanup.yaml`** (new) — Fire-and-forget dispatch of
`frontend-preview-cleanup` to the cloud repo when a frontend PR is
closed or has its preview label removed. Enables synchronized teardown.

### Companion PR

Cloud-side: Comfy-Org/cloud (creates the `deploy-frontend-preview` job,
extends the reconciler)

### How it works

1. Label a frontend PR with `preview`, `preview-cpu`, or `preview-gpu`
2. Assets build and upload to GCS (existing flow)
3. Cloud deploys a full ephemeral env at `fe-pr-{N}.testenvs.comfy.org`
using all `:main` service tags
4. Subsequent pushes update the frontend SHA via AppSet upsert
5. On close/unlabel, cleanup dispatch triggers immediate teardown

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9715-feat-dispatch-frontend-PR-preview-environments-to-cloud-31f6d73d3650819da1b5ca5ce419e06e)
by [Unito](https://www.unito.io)
2026-03-10 13:16:37 -04:00
Hunter
4e5bb3e540 fix: dispatch cloud build on synchronize for preview-labeled PRs (#9636)
## Summary

Cloud build dispatch was only triggering on the `labeled` event, not on
subsequent pushes to PRs that already had a preview label.

## Changes

- **What**: Add `synchronize` to `pull_request` event types and update
the `if` condition to support all three preview labels (`preview`,
`preview-cpu`, `preview-gpu`). For `labeled` events, check the added
label name; for `synchronize` events, check existing PR labels.

## Review Focus

The `if` condition now branches on `github.event.action` to use the
correct label-checking mechanism for each event type.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9636-fix-dispatch-cloud-build-on-synchronize-for-preview-labeled-PRs-31e6d73d3650814e9069e37d6199ffc9)
by [Unito](https://www.unito.io)
2026-03-09 15:55:53 -07:00
Benjamin Lu
750a2d23e0 chore: standardize on Node 24 (#9521)
## Summary

Standardize the repo's Node contract on 24 while centralizing workflow
resolution through `.nvmrc` so local setup, CI, and package metadata
stay aligned from one version file.

## Changes

- **What**: Add `package.json` `engines.node = 24.x`, switch every
`actions/setup-node` workflow in the repo to `node-version-file:
'.nvmrc'`, and update contributor and Playwright docs to point to
`.nvmrc` as the Node source of truth.

## Review Focus

The workflow behavior should be unchanged apart from sourcing the Node
version from `.nvmrc` instead of repeating literals like `20`, `22`,
`24.x`, or `lts/*`. GitHub's formatter also moved the new `engines`
block to the package metadata section near the end of `package.json`.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-07 09:06:10 +00:00
Hunter
f02adf84eb feat: dispatch cloud build when preview label is added to PR (#9518)
## Summary

Dispatch a `frontend-asset-build` event to the cloud repo when the
`preview` label is added to a PR, so cloud can build preview assets.

## Changes

- **What**: Extended `cloud-dispatch-build.yaml` to trigger on
`pull_request` `labeled` events filtered to the `preview` label. The
payload sends the PR head SHA and branch.

## Review Focus

- The `pull_request` trigger gives a read-only `GITHUB_TOKEN`, but the
dispatch step uses `CLOUD_DISPATCH_TOKEN` so this is fine.
- Fork PRs are blocked by the existing `github.repository` guard.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9518-feat-dispatch-cloud-build-when-preview-label-is-added-to-PR-31c6d73d365081a8aab6f585960977f6)
by [Unito](https://www.unito.io)
2026-03-07 03:57:08 +00:00
Robin Huang
6c2680f0ba feat: Add PostHog telemetry provider (#9409)
Add PostHog as a telemetry provider for cloud builds so custom events
can be correlated with session recordings. Follows the same pattern as
MixpanelTelemetryProvider with dynamic import, event queuing, and
disabled events from remote config. Tree-shaken away in OSS builds.

The posthog-js package uses Apache-2.0 (verified from its LICENSE file)
but declares it as "SEE LICENSE IN LICENSE" in package.json, which
  the license checker can't parse.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9409-feat-Add-PostHog-telemetry-provider-31a6d73d3650818b8e86c772c6551099)
by [Unito](https://www.unito.io)
2026-03-05 16:19:35 -08:00
Christian Byrne
6689a1b14e fix: split perf report workflow for fork PR support (#9382)
## Summary

Perf report workflow fails on fork PRs because `GITHUB_TOKEN` is
read-only for forks, causing "Resource not accessible by integration" on
the PR comment step.

## Changes

- **What**: Split `ci-perf-report.yaml` into a data-collection workflow
+ a `workflow_run`-triggered reporter (`pr-perf-report.yaml`), matching
the existing `ci-size-data`/`pr-size-report` pattern. Added fork PR
permissions guidance to `.github/AGENTS.md`.
- **ci-perf-report.yaml**: Removed the `report` job and `pull-requests:
write` permission. Added PR metadata (number + base branch) artifact
upload.
- **pr-perf-report.yaml** (new): Triggered by `workflow_run` on the perf
workflow. Downloads metrics + metadata artifacts, generates report,
posts PR comment with write permissions from the default-branch context.

## Review Focus

- The two-workflow split follows the same pattern as `ci-size-data.yaml`
→ `pr-size-report.yaml`, which already works for fork PRs.
- The `workflow_run` trigger runs in the base repo context per [GitHub
Security Lab
guidance](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/),
so it safely has write permissions even for fork PRs.
- AGENTS.md guidance documents this pattern to prevent recurrence.

Fixes the failure seen in
https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/22684230751/job/65763595989?pr=9380

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9382-fix-split-perf-report-workflow-for-fork-PR-support-3196d73d365081b29b35ed354e7789e2)
by [Unito](https://www.unito.io)
2026-03-04 22:09:31 -08:00
Johnpaul Chiwetelu
a2cb864828 chore: bump CI container to 0.0.13 (#9394)
Updates CI container from `0.0.12` to `0.0.13`

**Triggered by:** [Tag
0.0.13](https://github.com/Comfy-Org/comfyui-ci-container/tree/0.0.13)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9394-chore-bump-CI-container-to-0-0-13-3196d73d365081279c5bca477025cb5a)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: github-actions <github-actions@github.com>
2026-03-05 05:49:10 +01:00
Christian Byrne
7b316eb9a2 feat: add statistical significance to perf report with z-score thresholds (#9305)
## Summary

Replace fixed 10%/20% perf delta thresholds with dynamic σ-based
classification using z-scores, eliminating false alarms from naturally
noisy duration metrics (10-17% CV).

## Changes

- **What**:
- Run each perf test 3× (`--repeat-each=3`) and report the mean,
reducing single-run noise
- Download last 5 successful main branch perf artifacts to compute
historical μ/σ per metric
- Replace fixed threshold flags with z-score significance: `⚠️
regression` (z>2), ` neutral/improvement`, `🔇 noisy` (CV>50%)
  - Add collapsible historical variance table (μ, σ, CV) to PR comment
- Graceful cold start: falls back to simple delta table until ≥2
historical runs exist
- New `scripts/perf-stats.ts` module with `computeStats`, `zScore`,
`classifyChange`
  - 18 unit tests for stats functions

- **CI time impact**: ~3 min → ~5-6 min (repeat-each adds ~2 min,
historical download <10s)

## Review Focus

- The `gh api` call in the new "Download historical perf baselines"
step: it queries the last 5 successful push runs on the base branch. The
`gh` CLI is available natively on `ubuntu-latest` runners and
auto-authenticates with `GITHUB_TOKEN`.
- `getHistoricalStats` averages per-run measurements before computing
cross-run σ — this is intentional since historical artifacts may also
contain repeated measurements after this change lands.
- The `noisy` classification (CV>50%) suppresses metrics like `layouts`
that hover near 0 and have meaningless percentage swings.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9305-feat-add-statistical-significance-to-perf-report-with-z-score-thresholds-3156d73d3650818d9360eeafd9ae7dc1)
by [Unito](https://www.unito.io)
2026-03-04 16:16:53 -08:00
Hunter
7c8a548798 feat: add cloud frontend build dispatch workflow (#9308)
## Summary

Adds `.github/workflows/cloud-dispatch-build.yaml` — fires a
`repository_dispatch` event (`frontend-asset-build`) to
`Comfy-Org/cloud` on push to `cloud/*` branches and `main`.

The cloud repo handles the actual build, GCS upload, and secret
management (Sentry, Algolia, GCS creds). This is fire-and-forget.

## Changes

- New workflow: `cloud-dispatch-build.yaml`
- Trigger: `push` to `cloud/*` and `main` only
- Payload: `ref` (commit SHA) + `branch` (branch name), built with `jq`
to prevent injection
- SHA-pinned `peter-evans/repository-dispatch@v4.0.1`
- Hardened: `permissions: {}`, fork guard (`if: github.repository ==
'Comfy-Org/ComfyUI_frontend'`), concurrency to avoid dispatch storms
- `cloud-deploy-frontend.yaml` left unchanged (still needed during
migration)

## Setup Required

A repository secret `CLOUD_DISPATCH_TOKEN` must be configured — see PR
description comments.

## Part of

Frontend separate deploy prep (Task 1.3)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9308-feat-add-cloud-frontend-build-dispatch-workflow-3156d73d36508164a515eb968f6c5d79)
by [Unito](https://www.unito.io)
2026-02-28 17:59:19 -05:00
Christian Byrne
8e215b3174 feat: add performance testing infrastructure with CDP metrics (#9170)
## Summary

Add a permanent, non-failing performance regression detection system
using Chrome DevTools Protocol metrics, with automatic PR commenting.

## Changes

- **What**: Performance testing infrastructure — `PerformanceHelper`
fixture class using CDP `Performance.getMetrics` to collect
`RecalcStyleCount`, `LayoutCount`, `LayoutDuration`, `TaskDuration`,
`JSHeapUsedSize`. Adds `@perf` Playwright project (Chromium-only,
single-threaded, 60s timeout), 4 baseline perf tests, CI workflow with
sticky PR comment reporting, and `perf-report.js` script for generating
markdown comparison tables.

## Review Focus

- `PerformanceHelper` uses `page.context().newCDPSession(page)` — CDP is
Chromium-only, so perf metrics are not collected on Firefox. This is
intentional since CDP gives us browser-level style recalc/layout counts
that `performance.mark/measure` cannot capture.
- The CI workflow uses `continue-on-error: true` so perf tests never
block merging.
- Baseline comparison uses `dawidd6/action-download-artifact` to
download metrics from the target branch, following the same pattern as
`pr-size-report.yaml`.

## Stack

This is the foundation PR for the Firefox performance fix stack:
1. **→ This PR: perf testing infrastructure**
2. `perf/fix-cursor-cache` — cursor style caching (depends on this)
3. `perf/fix-subgraph-svg` — SVG pre-rasterization (depends on this)
4. `perf/fix-clippath-raf` — RAF batching for clip-path (depends on
this)

PRs 2-4 are independent of each other.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9170-feat-add-performance-testing-infrastructure-with-CDP-metrics-3116d73d3650817cb43def6f8e9917f8)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-02-25 20:09:57 -08:00
Benjamin Lu
e4b456bb2c fix: publish desktop-specific frontend release artifact (#9206)
## Summary
- add a desktop-specific frontend release artifact (`dist-desktop.zip`)
in release draft creation
- build `dist-desktop.zip` with `DISTRIBUTION=desktop`
- keep existing `dist.zip` behavior for core/PyPI consumers
- extend `scripts/zipdist.js` to support custom source and output paths

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9206-fix-publish-desktop-specific-frontend-release-artifact-3126d73d3650812495cdf6e9ad2ac280)
by [Unito](https://www.unito.io)
2026-02-25 03:35:41 -08:00
Alexander Brown
c25f9a0e93 feat: synthetic widgets getter for SubgraphNode (proxy-widget-v2) (#8856)
## Summary

Replace the Proxy-based proxy widget system with a store-driven
architecture where `promotionStore` and `widgetValueStore` are the
single sources of truth for subgraph widget promotion and widget values,
and `SubgraphNode.widgets` is a synthetic getter composing lightweight
`PromotedWidgetView` objects from store state.

## Motivation

The subgraph widget promotion system previously scattered state across
multiple unsynchronized layers:

- **Persistence**: `node.properties.proxyWidgets` (tuples on the
LiteGraph node)
- **Runtime**: Proxy-based `proxyWidget.ts` with `Overlay` objects,
`DisconnectedWidget` singleton, and `isProxyWidget` type guards
- **UI**: Each Vue component independently calling `parseProxyWidgets()`
via `customRef` hacks
- **Mutation flags**: Imperative `widget.promoted = true/false` set on
`subgraph-opened` events

This led to 4+ independent parsings of the same data, complex cache
invalidation, and no reactive contract between the promotion state and
the rendering layer. Widget values were similarly owned by LiteGraph
with no Vue-reactive backing.

The core principle driving these changes: **Vue owns truth**. Pinia
stores are the canonical source; LiteGraph objects delegate to stores
via getters/setters; Vue components react to store state directly.

## Changes

### New stores (single sources of truth)

- **`promotionStore`** — Reactive `Map<NodeId, PromotionEntry[]>`
tracking which interior widgets are promoted on which SubgraphNode
instances. Graph-scoped by root graph ID to prevent cross-workflow state
collision. Replaces `properties.proxyWidgets` parsing, `customRef`
hacks, `widget.promoted` mutation, and the `subgraph-opened` event
listener.
- **`widgetValueStore`** — Graph-scoped `Map<WidgetKey, WidgetState>`
that is the canonical owner of widget values. `BaseWidget.value`
delegates to this store via getter/setter when a node ID is assigned.
Eliminates the need for Proxy-based value forwarding.

### Synthetic widgets getter (SubgraphNode)

`SubgraphNode.widgets` is now a getter that reads
`promotionStore.getPromotions(rootGraphId, nodeId)` and returns cached
`PromotedWidgetView` objects. No stubs, no Proxies, no fake widgets
persisted in the array. The setter is a no-op — mutations go through
`promotionStore`.

### PromotedWidgetView

A class behind a `createPromotedWidgetView` factory, implementing the
`PromotedWidgetView` interface. Delegates value/type/options/drawing to
the resolved interior widget and stores. Owns positional state (`y`,
`computedHeight`) for canvas layout. Cached by
`PromotedWidgetViewManager` for object-identity stability across frames.

### DOM widget promotion

Promoted DOM widgets (textarea, image upload, etc.) render on the
SubgraphNode surface via `positionOverride` in `domWidgetStore`.
`DomWidgets.vue` checks for overrides and uses the SubgraphNode's
coordinates instead of the interior node's.

### Promoted previews

New `usePromotedPreviews` composable resolves image/audio/video preview
widgets from promoted entries, enabling SubgraphNodes to display
previews of interior preview nodes.

### Deleted

- `proxyWidget.ts` (257 lines) — Proxy handler, `Overlay`,
`newProxyWidget`, `isProxyWidget`
- `DisconnectedWidget.ts` (39 lines) — Singleton Proxy target
- `useValueTransform.ts` (32 lines) — Replaced by store delegation

### Key architectural changes

- `BaseWidget.value` getter/setter delegates to `widgetValueStore` when
node ID is set
- `LGraph.add()` reordered: `node.graph` assigned before widget
`setNodeId` (enables store registration)
- `LGraph.clear()` cleans up graph-scoped stores to prevent stale
entries across workflow switches
- `promotionStore` and `widgetValueStore` state nested under root graph
UUID for multi-workflow isolation
- `SubgraphNode.serialize()` writes promotions back to
`properties.proxyWidgets` for persistence compatibility
- Legacy `-1` promotion entries resolved and migrated on first load with
dev warning

## Test coverage

- **3,700+ lines of new/updated tests** across 36 test files
- **Unit**: `promotionStore.test.ts`, `widgetValueStore.test.ts`,
`promotedWidgetView.test.ts` (921 lines),
`subgraphNodePromotion.test.ts`, `proxyWidgetUtils.test.ts`,
`DomWidgets.test.ts`, `PromotedWidgetViewManager.test.ts`,
`usePromotedPreviews.test.ts`, `resolvePromotedWidget.test.ts`,
`subgraphPseudoWidgetCache.test.ts`
- **E2E**: `subgraphPromotion.spec.ts` (622 lines) — promote/demote,
manual/auto promotion, paste preservation, seed control augmentation,
image preview promotion; `imagePreview.spec.ts` extended with
multi-promoted-preview coverage
- **Fixtures**: 2 new subgraph workflow fixtures for preview promotion
scenarios

## Review focus

- Graph-scoped store keying (`rootGraphId`) — verify isolation across
workflows/tabs and cleanup on `LGraph.clear()`
- `PromotedWidgetView` positional stability — `_arrangeWidgets` writes
to `y`/`computedHeight` on cached objects; getter returns fresh array
but stable object references
- DOM widget position override lifecycle — overrides set on promote,
cleared on demote/removal/subgraph navigation
- Legacy `-1` entry migration — resolved and written back on first load;
unresolvable entries dropped with dev warning
- Serialization round-trip — `promotionStore` state →
`properties.proxyWidgets` on serialize, hydrated back on configure

## Diff breakdown (excluding lockfile)

- 153 files changed, ~7,500 insertions, ~1,900 deletions (excluding
pnpm-lock.yaml churn)
- ~3,700 lines are tests
- ~300 lines deleted (proxyWidget.ts, DisconnectedWidget.ts,
useValueTransform.ts)

<!-- Fixes #ISSUE_NUMBER -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-8856-feat-synthetic-widgets-getter-for-SubgraphNode-proxy-widget-v2-3076d73d365081c7b517f5ec7cb514f3)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: GitHub Action <action@github.com>
2026-02-23 13:33:41 -08:00