Pass the comprehensive test plan from .claude/skills/comfy-qa/SKILL.md
to Gemini when generating test steps. This gives Gemini knowledge of all
12 QA categories (canvas, menus, sidebar, settings, etc.) so it picks
the most relevant tests for each PR.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nx build runs typecheck as a prerequisite (via @nx/vite/plugin config).
Use vite build directly for the main branch comparison build.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Main branch may have transient TS errors when built with the PR
branch's lockfile. Since we only need the dist for visual comparison,
run nx build directly instead of pnpm build (which includes typecheck).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the unreliable codex exec approach with a Playwright script
(qa-record.ts) that uses Gemini to generate targeted test steps from
the PR diff, then executes them deterministically via Playwright's API.
Key changes:
- New scripts/qa-record.ts: Gemini generates JSON test actions, Playwright
executes them with reliable helper functions (menu nav, dialog fill, etc.)
- Remove codex CLI and playwright-cli dependencies
- Remove 150+ lines of prompt templates from pr-qa.yaml
- Firefox headless with video recording (same approach proven locally)
- Fallback steps if Gemini fails
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Tighten BEFORE prompt to 15s snapshot (show old state only)
- Add qa-generate-test.ts: Gemini-powered Playwright test generator
- New workflow step: generate .spec.ts and push to {branch}-add-qa-test
- Tests assert UIUX behavior (tab names, dirty state, visibility)
- Build both main (dist-before/) and PR (dist/) frontends in focused mode
- Run QA twice: BEFORE on main branch frontend, AFTER on PR branch
- Send both videos to Gemini in one request for comparative analysis
- Side-by-side dashboard layout with Before (main) / After (PR) panels
- Comparative prompt evaluates whether before confirms old behavior
and after proves the fix works
- Falls back to single-video mode when no before video available
moov atom was at end of file (8.6MB offset) — browser had to download
the entire video before seeking. Keyframes were only every 10 seconds.
Add -movflags +faststart (moov before mdat) and -g 60 (keyframe every
2.4s at 25fps) to ffmpeg conversion.
- Remove autoplay/loop so video timeline is seekable
- Skip card generation for platforms without recordings
- Add --pr-context flag to qa-video-review.ts so Gemini evaluates
against PR purpose instead of just describing what happened
- Workflow now builds pr-context.txt from PR title/body/diff
The Codex agent was spending 35s browsing the "Getting Started" template
gallery instead of testing the PR's changes. Pre-seeding this setting
via the ComfyUI API ensures the agent lands directly in the graph editor.
The Codex agent was spending time on login flow, template browsing,
and general smoke testing instead of testing the PR's actual changes.
Changes:
- Add 30-second time budget for video recording
- Move video-start AFTER login and editor verification
- Explicitly prohibit template browsing and sidebar exploration
- Reduce test steps to 3-6 targeted actions
- Restructure prompt with clear Instructions/Rules sections
Replace crude sed-based markdown conversion with client-side
rendering via marked.js CDN. Adds proper table, list, and
code styling for the report section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the OpenAI GPT-based frame extraction approach (ffmpeg + screenshots)
with Gemini 2.5 Flash's native video understanding. This eliminates false
positives from frame-based analysis (e.g. "black screen = critical bug" during
page transitions) and produces dramatically better QA reviews.
Changes:
- Remove ffmpeg frame extraction, ffprobe duration detection, and all related
logic (~365 lines removed)
- Add @google/generative-ai SDK for native video/mp4 upload to Gemini
- Update CLI: remove --max-frames, --min-interval-seconds, --keep-frames flags
- Update env: OPENAI_API_KEY → GEMINI_API_KEY
- Update workflow: swap API key secret and model in pr-qa.yaml
- Update report: replace "Frames analyzed" with "Video size"
- Add note in prompt that brief black frames during transitions are normal
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
playwright-cli doesn't support 'evaluate' command. Instead, instruct
Codex to quickly fill the username input and click Next on user-select
page BEFORE starting video recording, so the video only shows actual
QA testing.
storageState config doesn't work with playwright-cli. Instead, use
evaluate to set Comfy.userId/userName after opening the page, then
navigate back. This skips user-select before video-start so the
recording only shows actual QA testing.
Write a Playwright storageState JSON with Comfy.userId/userName pre-set
so the app loads directly to the graph editor. Saves ~40s per QA run
that was wasted on navigating the user-select page.
The convert step was using find which picked up a 0-byte file from
playwright's videos/ directory instead of the valid qa-session.webm.
Now prefers qa-session.webm explicitly and skips empty files.
Codex was using pnpm dlx instead of the global playwright-cli.
Pre-install chromium in setup step and make prompt explicit about
using the global command directly without pnpm/npx.
Replace claude --print with codex exec for cheaper QA runs.
Uses codex-mini-latest model ($1.50/$6 vs Sonnet $3/$15).
Uses existing OPENAI_API_KEY secret (no new secrets needed).
- Replace saveVideo config (didn't produce video) with explicit
playwright-cli video-start/video-stop commands in QA prompt
- Remove apt-get install ffmpeg step (pre-installed on GH runners)
- Switch video review model from gpt-4o to gpt-4.1-mini
- Enable saveVideo in playwright-cli config for real video recording
- Replace screenshot stitching with webm→mp4 conversion
- Move video review step before deploy so reports are included
- Add GPT video review reports inline on the Cloudflare Pages site
- Each video card now has expandable "GPT Video Review" section
- Set .playwright/cli.config.json with outputDir pointing to screenshots/
- This way bare 'playwright-cli screenshot' auto-saves to the right place
- Create screenshot directory before Claude runs (don't rely on Claude)
- Collect step now searches working directory for stray PNGs
- Simplified prompt: no --filename needed, just 'playwright-cli screenshot'
Screenshots were saved to artifact root but stitch looked in frames/.
Now: prompt tells Claude to save to screenshots/ dir with numbered names,
collect step consolidates PNGs there, stitch step globs from screenshots/.
Removed video-start/video-stop (Claude doesn't use them).
- Add playwright-cli config with outputDir and saveVideo
- Use video-start/video-stop instead of relying on screenshot frames
- Add fallback artifact collection from .playwright-cli/ default dir
- Simplify prompts to focus on video recording workflow
The escaped \$QA_ARTIFACTS in the heredoc produced literal text
'$QA_ARTIFACTS' in the prompt. Claude's Bash tool didn't reliably
expand this env var, so no screenshots or reports were saved.
Remove the escapes so the heredoc expands the variable to the actual
path (e.g. /home/runner/work/_temp/qa-artifacts).
Backtick-wrapped playwright-cli examples in the unquoted heredoc were
being interpreted as bash command substitution, producing empty prompts.
Replace backtick syntax with plain "Run:" prefixed commands.
- Remove all Xvfb/ffmpeg screen recording infrastructure from qa job
(captured blank display since playwright-cli runs headless)
- Add screenshot instructions to QA prompts: Claude saves sequential
frames to $QA_ARTIFACTS/frames/ after every interaction
- Stitch screenshots into video via ffmpeg in report job (2fps)
- Merge video-review job into report job (4 jobs → 3 jobs)
- Unified PR comment with video links + video review in <details> collapse
- Clean up stale QA_VIDEO_REVIEW_COMMENT markers from prior runs
Move extra_server_params input to env var to prevent shell injection
from untrusted input. Replace wait-for-it pip dependency with a
cross-platform curl polling loop.
Add Claude Code skills and a label-triggered QA workflow:
- .claude/skills/comfy-qa/SKILL.md: 12-category QA test plan using
playwright-cli for browser automation
- .github/workflows/pr-qa.yaml: CI workflow triggered by qa-changes
(focused, Linux) or qa-full (3-OS matrix) labels. Records screen via
ffmpeg, runs Claude CLI with playwright-cli, deploys video gallery to
Cloudflare Pages, posts PR comment with GIF thumbnails, and runs
OpenAI vision-based video review
- scripts/qa-video-review.ts: frame extraction + GPT-4o analysis
- scripts/qa-video-review.test.ts: unit tests for video review
- knip.config.ts: resolve knip errors for ingest-types package
## Summary
- Move `contain-layout contain-style` from the outer node container to
the inner wrapper to prevent CSS containment from interfering with
inter-node compositing
- Add `isolation: isolate` to the outer container for explicit stacking
context isolation
- Fixes#9988
## Root Cause
Originally, PR
[#5169](https://github.com/Comfy-Org/ComfyUI_frontend/pull/5169)
(`57db10f`, 2025-08-23) applied all three containment properties to the
outer node container:
```
contain-layout contain-style contain-paint
```
`contain-paint` ensured each node's painting was clipped to its own
boundaries, guaranteeing render isolation between sibling nodes.
Later, PR
[#6289](https://github.com/Comfy-Org/ComfyUI_frontend/pull/6289)
(`6e4471ad`, 2025-10-28) removed `contain-paint` to fix a node sizing
issue — likely because `contain-paint` clips content to the element's
padding box, which would clip selection overlays and footer tabs that
extend beyond the node boundary via negative insets.
Removing `contain-paint` while keeping `contain-layout contain-style` on
the outer container broke paint isolation between nodes. Without paint
containment, the browser's compositor no longer treated each node as an
isolated painting unit, allowing text from lower z-index nodes to
visually bleed through higher z-index nodes.
## Fix
1. **Move containment inward**: `contain-layout contain-style` is moved
from the outer container to the inner wrapper (`node-inner-wrapper`).
This preserves layout/style containment performance benefits while
keeping them scoped to the node body — away from the stacking context
boundary between sibling nodes.
2. **Explicit isolation**: `isolation: isolate` is added to the outer
container. While `transform` + `z-index` already create a stacking
context, `isolation: isolate` provides an unambiguous signal to the
browser's compositor to treat each node as an isolated compositing unit
— replacing the paint isolation role that `contain-paint` previously
served, without clipping overlays.
## Red-Green Verification
| Commit | Status | Purpose |
|--------|--------|---------|
|
[`feb8555`](https://github.com/Comfy-Org/ComfyUI_frontend/commit/feb855501)
`test:` | 🔴 Red (verified locally) | Test asserts `isolate`
on outer container and `contain-layout contain-style` on inner wrapper
only — fails against current main |
|
[`c47f89e`](https://github.com/Comfy-Org/ComfyUI_frontend/commit/c47f89ed2)
`fix:` | 🟢 Green (CI: Tests Unit passing) | Moves
containment to inner wrapper, adds `isolation: isolate` — test passes |
**Local red verification:**
```
AssertionError: expected [ 'group/node', 'lg-node', …(9) ] to include 'isolate'
❯ src/renderer/extensions/vueNodes/components/LGraphNode.test.ts:264:26
```
## Test Plan
- [x] CI red on test-only commit (verified locally — test fails without
fix)
- [x] CI green on fix commit (CI: Tests Unit passes)
- [ ] Place a String Literal node with long text on the canvas
- [ ] Place a Load Image node (with an image loaded) overlapping the
String Literal
- [ ] Verify text from the String Literal does NOT bleed through the
Load Image node
- [ ] Verify node selection outlines, error borders, and footer tabs
render correctly
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
## Summary
- Read `sentry_dsn` from remote config at runtime for cloud builds,
following the same pattern as `firebase.ts` and `comfyApi.ts`
- Upload sourcemaps to both staging (`SENTRY_PROJECT`) and prod
(`SENTRY_PROJECT_PROD`) Sentry projects at build time
- Remove `SENTRY_DSN_PROD` / `USE_PROD_CONFIG`-based DSN selection — DSN
is now delivered per-environment via dynamic config
## Companion PR
- https://github.com/Comfy-Org/cloud/pull/2883 (cloud repo: dynamic
config + workflow)
## Setup required
- Add `SENTRY_PROJECT_PROD` variable (`cloud-frontend-prod`) to GitHub
STAGING environment in cloud repo
## Test plan
- [ ] Verify staging build reads staging DSN from remote config
- [ ] Verify prod build reads prod DSN from remote config
- [ ] Confirm sourcemaps are uploaded to both Sentry projects
- [ ] Validate fallback to `__SENTRY_DSN__` when remote config is
unavailable
## Summary
Fix high-resolution image preview expanding to full screen width, hiding
the close button.
## Changes
- **What**: Override scoped `ComfyImage` styles (`width/height: 100%`,
`object-fit: cover`) in the gallery context with `auto` sizing and
`max-width: 100vw; max-height: 100vh` constraints. The scoped
`[data-v-xxx]` attribute selector gave `comfy-image-main` higher
specificity than the existing `img.galleria-image` rule, so the new
selector `img.comfy-image-main.galleria-image` is used to win
specificity.
## Review Focus
- CSS specificity approach: using `.comfy-image-main.galleria-image` to
override Vue scoped styles without `!important`.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10129-fix-high-res-image-preview-overflowing-screen-and-hiding-close-button-3266d73d36508191a03bcc75f9912421)
by [Unito](https://www.unito.io)
---------
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: GitHub Action <action@github.com>
## Summary
- Remove the legacy missing nodes modal dialog and migrate all
functionality to the existing Error Overlay / TabErrors system
- Migrate core node version warning from `MissingCoreNodesMessage.vue`
to `MissingNodeCard` in the errors tab
- Remove `Comfy.Workflow.ShowMissingNodesWarning` setting (errors tab
always surfaces missing nodes)
- Delete 6 legacy files: `useMissingNodesDialog.ts`,
`MissingNodesContent.vue`, `MissingNodesFooter.vue`,
`MissingNodesHeader.vue`, `MissingCoreNodesMessage.vue` and its test
- Rename `showMissingNodesDialog`/`showMissingModelsDialog` params to
`showMissingNodes`/`showMissingModels`
- Add `errorOverlay` and `missingNodeCard` to centralized `TestIds`
- Migrate all E2E tests from legacy dialog selectors to error overlay
testIds
- Add new E2E test: MissingNodeCard visible via "See Errors" button flow
- Add new E2E test: subgraph missing node type verified by expanding
pack row
- Add `surfaceMissingNodes` unit tests to `executionErrorStore`
- Guard `semver.compare` against invalid version strings
- Add `role="alert"`, `aria-hidden` for accessibility
- Use reactive props destructuring in `MissingNodeCard` and
`MissingPackGroupRow`
**Net change: -669 lines** (19 files, +323 / -992)
<img width="733" height="579" alt="image"
src="https://github.com/user-attachments/assets/c497809d-b176-43bf-9872-34bd74c6ea0d"
/>
## Test plan
- [x] Unit tests: MissingNodeCard core node warning (7 tests)
- [x] Unit tests: surfaceMissingNodes (4 tests)
- [x] Unit tests: workflowService showPendingWarnings (updated)
- [x] E2E: Error overlay visible on missing nodes workflow
- [x] E2E: Error overlay visible on subgraph missing nodes
- [x] E2E: MissingNodeCard visible via See Errors button
- [x] E2E: Subgraph node type visible after expanding pack row
- [x] E2E: Error overlay does not resurface on undo/redo
- [x] E2E: Error overlay does not reappear on workflow tab switch
- [x] Typecheck, lint, knip all passing
## Related issues
- Closes#9923 (partially — `errorOverlay` and `missingNodeCard` added
to TestIds)
- References #10027 (mock hoisting inconsistency)
- References #10033 (i18n-based test selectors)
- References #10085 (DDD layer violation + focusedErrorNodeId)
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10102-refactor-remove-legacy-missing-nodes-dialog-3256d73d365081c194d2e90bc6401846)
by [Unito](https://www.unito.io)
