Compare commits

...

202 Commits

Author SHA1 Message Date
snomiao
05c6e1c0ff fix: use comfyPage.page.waitForTimeout for delay injection
The test uses comfyPageFixture, not bare page. Also match
firstNode await calls for node interaction pauses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 11:42:41 +00:00
snomiao
15fb037a55 feat: inject 800ms pauses between test actions for readable videos
Regex inserts await page.waitForTimeout(800) before every
comfyPage/topbar/page/canvas/expect await call in the Phase 2
test code. Adds ~5-8s to a 10-step test (negligible vs 10min research).

Default playback changed to 0.5x (was 0.25x) since pauses provide
natural breathing room. A 15s video at 0.5x = 30s viewing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 11:27:02 +00:00
snomiao
dbc12a9d6a feat: default playback 0.25x + cursor overlay in E2E test videos
- Report player defaults to 0.25x speed (was 0.5x) — 5s test videos
  play in 20s, much more watchable
- Phase 2 injects cursor overlay via addInitScript into the test code
  before running — white SVG arrow follows mousemove events

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 10:35:23 +00:00
snomiao
909b75b9d1 fix: set PLAYWRIGHT_LOCAL=1 for Phase 2 to enable video recording
Playwright config only records video when PLAYWRIGHT_LOCAL is set.
In CI, this env var was missing so Phase 2 produced no video.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 13:09:51 +00:00
snomiao
868d774007 fix: don't overwrite Phase 2 test video with idle research video
After context.close(), renameLatestWebm would overwrite the Phase 2
test execution video with the idle research browser recording.
Now skips the rename if qa-session.webm already exists from Phase 2.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 12:18:42 +00:00
snomiao
69cd6a7628 fix: Phase 2 records test execution video, copies to qa-session.webm
The old video showed an idle screen (research browser doing nothing).
Now Phase 2 runs the test with --video=on from browser_tests/tests/,
finds the recorded .webm, and copies it to qa-session.webm where
the deploy script expects it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 07:32:22 +00:00
snomiao
9c39635f16 fix: runTest uses project Playwright config + fixtures
- Copy test to browser_tests/tests/ where Playwright config expects it
- System prompt teaches Claude the project's test fixtures:
  comfyPageFixture, comfyPage.menu.topbar, comfyPage.workflow, etc.
- Increased time budget to 10 min for write→run→fix iterations
- Increased max turns to 50

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 06:40:11 +00:00
snomiao
3e957213c8 fix: broaden research-log.json search paths in deploy script
Also search qa-artifacts/before/*/research/ for the research log
since artifacts are downloaded with that nested structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 05:13:32 +00:00
snomiao
f5d99c9c22 feat: Claude writes E2E tests to reproduce bugs instead of driving browser
Phase 1: Claude reads issue + a11y tree, writes a Playwright .spec.ts
test that asserts the bug exists. Runs the test, reads errors, iterates
until the test passes (proving the bug) or determines NOT_REPRODUCIBLE.

Phase 2: Run the passing test with --video=on for clean recording.

This replaces interactive browser driving with deterministic test code.
Claude Sonnet 4.6 excels at writing Playwright tests — much more
reliable than real-time browser interaction.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 21:02:36 +00:00
snomiao
09bbd89172 fix: use ariaSnapshot() instead of removed page.accessibility API
page.accessibility.snapshot() was removed in Playwright 1.49+.
Use page.locator('body').ariaSnapshot() which returns a text
representation of the accessibility tree.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 20:50:25 +00:00
snomiao
0b5613246e feat: deploy research-log.json + use it as primary verdict source
- Copy research-log.json to deploy dir (accessible at /research-log.json)
- Read verdict from research log first (a11y-verified ground truth)
- Fall back to video review verdict only if no research log exists
- Research log is uploaded as part of QA artifacts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 19:38:19 +00:00
snomiao
3a27263ca6 feat: three-phase QA pipeline — Research → Reproduce → Report
Phase 1 (qa-agent.ts): Claude investigates via a11y API only.
  - No video, no Gemini vision — only page.accessibility.snapshot()
  - Every action logged with a11y before/after state
  - done() requires evidence citing inspect() results
  - Outputs reproduction plan for Phase 2

Phase 2 (qa-reproduce.ts): Deterministic replay of research plan.
  - Executes each step with a11y assertions
  - Gemini describes visual changes (narration for humans)
  - Clean focused video with subtitles

Phase 3: Report job reads research-log.json for verdict (ground truth),
  narration-log.json for descriptions, video for visuals.
  Gemini formats logs into report — never determines verdict.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 18:17:31 +00:00
snomiao
6044452b8f fix: prevent AI lies — assertion-based verdicts + blind reviewer
Agent: MUST use inspect() after every action, verdict based on DOM
state not opinions. "NEVER claim REPRODUCED unless inspect() confirms."

Reviewer: Two-phase prompt — Phase 1 describes what it SEES (blind,
no context). Phase 2 compares observations against issue/PR context.
Anti-hallucination rules: "describe ONLY what you observe, NEVER infer."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 17:26:48 +00:00
snomiao
c4a243060b feat: Agent SDK auto-detects Claude Code session — no API key needed locally
ANTHROPIC_API_KEY is optional: Agent SDK uses Claude Code OAuth
session when running locally (detects CLAUDE_CODE_SSE_PORT).
In CI, ANTHROPIC_API_KEY from secrets is used.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 15:07:57 +00:00
snomiao
3690e98c79 refactor: require ANTHROPIC_API_KEY, remove Gemini-only fallback
The Gemini-only agentic loop had ~47% success rate — too low to be
useful as a fallback. Now ANTHROPIC_API_KEY is required for issue
reproduction. Fails clearly if missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 15:01:51 +00:00
snomiao
1c40893cfa fix: inject cursor overlay via addScriptTag after login, not addInitScript
addInitScript runs before page load — Vue's app mount destroys the
cursor div when it takes over the DOM. Using addScriptTag after login
ensures the cursor persists in the stable DOM.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 13:30:37 +00:00
snomiao
72a28a1e76 fix: cursor overlay on locator clicks (clickByText, menu items)
Locator.click/hover bypasses our page.mouse monkey-patch. Now
clickByText, hoverMenuItem, clickSubmenuItem get the element
bounding box and update cursor overlay manually.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 11:58:01 +00:00
snomiao
49c95248e5 fix: verdict JSON grep pattern — capture value without closing quote
The grep \{"verdict":\s*"[^"]+ captures up to but not including the
closing quote. The second grep for "[A-Z_]+"$ then fails because
there's no closing quote. Fixed: match "verdict":\s*"[A-Z_]+ then
extract [A-Z_]+$ (no quotes needed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 10:51:35 +00:00
snomiao
6bb9d18ca6 fix: grep pipefail crash + add QA troubleshooting doc
- Add || true to all grep pipelines in deploy script (grep returns 1
  on no match, pipefail kills script)
- Add docs/qa/TROUBLESHOOTING.md covering all failures encountered:
  __name errors, zod/v4 imports, model IDs, badge mismatches, cursor,
  loadDefaultWorkflow, pressKey timing, agent behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 10:17:36 +00:00
snomiao
ca49e9cb1b feat: structured JSON verdict from AI reviewer, light-first theme
- Video review prompt now requests a ## Verdict JSON block:
  {"verdict": "REPRODUCED|NOT_REPRODUCIBLE|INCONCLUSIVE", "risk": "low|medium|high"}
- Deploy script reads JSON verdict first, falls back to grep
- Eliminates all regex-matching false positives permanently
- Theme: light mode is default, dark via prefers-color-scheme:dark
- Cards use solid backgrounds, grain overlay only in dark mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 09:11:09 +00:00
snomiao
db538c9d76 feat: report site follows system light/dark theme
Add prefers-color-scheme:light media query with light palette.
Replace hardcoded dark oklch values with CSS variables.
Light mode: white surfaces, dark text, subtle borders, no grain overlay.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 08:53:08 +00:00
snomiao
c04b31a0f1 fix: loadDefaultWorkflow uses API instead of menu, pressKey uses instant press
- loadDefaultWorkflow now calls app.resetToDefaultWorkflow() via JS API
  instead of navigating File → Load Default menu (menu item name varies)
- pressKey reverted to instant press() — the 400ms hold via down/up
  prevented Escape from propagating to parent dialog (#10397 BEFORE video
  showed wrong behavior because hold intercepted the event)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 08:45:14 +00:00
snomiao
5938fdef8d fix: monkey-patch page.mouse for universal cursor overlay
Instead of manually calling moveCursorOverlay in each action,
patch page.mouse.move/click/dblclick/down/up globally. Now EVERY
mouse operation shows the cursor — text clicks, menu hovers, etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:47:56 +00:00
snomiao
27d55f093b fix: use gemini-3-flash-preview in hybrid agent (not 2.5 preview)
Gemini 2.5 preview models return 404. Always use gemini-3+ models.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:45:29 +00:00
snomiao
c1ddb8669e fix: add 'could not be confirmed' to negative verdict patterns
"could not be confirmed" contains "confirmed" which matched the
positive reproduc|confirm check. Now caught by the negative check first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 15:17:02 +00:00
snomiao
57dbf0132d fix: correct Claude model ID — claude-sonnet-4-6 (not dated suffix)
The Agent SDK returned "model not found" for claude-sonnet-4-6-20250514.
Correct ID is claude-sonnet-4-6.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 14:36:22 +00:00
snomiao
b4c49588dc fix: cursor overlay now controlled via __moveCursor, not DOM events
Headless Chrome's Playwright CDP doesn't trigger DOM mousemove events
reliably. Now executeAction calls __moveCursor(x,y) directly after
every mouse.move/click/drag. Cursor is an SVG arrow (white + outline).
Click state shown via scale animation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 13:25:12 +00:00
snomiao
00dc10e9e6 feat: badge label includes QA date — #10397 QA0327
Shows when the QA was run so stale results are obvious at a glance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 08:43:19 +00:00
snomiao
48414fa1b5 fix: make key presses visible in video — hold + subtitle
pressKey now uses keyboard.down/up with 400ms hold instead of
instant press(). Shows subtitle "⌨ Escape" and the keyboard HUD
catches the held state for video frame capture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 08:33:22 +00:00
snomiao
e744f101b0 fix: use zod instead of zod/v4 — project zod doesn't export /v4 subpath
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 07:51:37 +00:00
snomiao
9ad8267067 fix: add claude-agent-sdk to workspace catalog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 07:26:54 +00:00
snomiao
cf54ddb6d3 feat: control/test comparison strategy + QA backlog doc
- Agent system prompt now instructs Claude to demonstrate BOTH working
  (control) and broken (test) states when bug is triggered by a setting
- Added docs/qa/backlog.md with future improvements: Type B/C comparisons,
  TTS, pre-seeding, cost optimization, environment-dependent issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 07:24:10 +00:00
snomiao
fce31cf0bf feat: all badges use vertical box style
Drop horizontal badges. Universal box badge shows:
  ┌──────────────────┐
  │    #7414 QA       │
  │ ✓ 1 reproduced   │
  │ ⚙ Fix: APPROVED  │  ← only for PRs
  └──────────────────┘

Issues show repro/not-repro/inconclusive rows.
PRs add a fix quality row (APPROVED/MINOR/MAJOR).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 07:20:05 +00:00
snomiao
050091abc6 feat: show QA pipeline commit hash + timing on report site
- Shows "QA @ abc1234" linking to the pipeline code commit
- Shows start time → deploy time in header
- Helps trace which version of QA scripts generated each report

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 07:16:41 +00:00
snomiao
548e37b9a5 feat: hybrid QA agent — Claude Sonnet 4.6 brain + Gemini vision
Architecture:
- Claude Sonnet 4.6 plans and reasons (via Claude Agent SDK)
- Gemini 2.5 Flash watches video buffer and describes what it sees
- 4 tools: observe(), inspect(), perform(), done()

observe(seconds, focus): builds video clip from screenshot buffer,
  sends to Gemini with Claude's focused question.
inspect(selector): searches a11y tree for specific element state.
perform(action, params): executes Playwright action.
done(verdict, summary): signals completion.

Falls back to Gemini-only loop if ANTHROPIC_API_KEY not set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 06:27:30 +00:00
snomiao
458b2e918c feat: pass OPENAI_API_KEY to recording step for TTS narration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 04:53:13 +00:00
snomiao
63dbe002d1 feat: subtitle overlay + OpenAI TTS narration on reproduce videos
- Agent reasoning shown as subtitle bar at bottom of video during recording
- After recording, generates TTS audio via OpenAI API (tts-1, nova voice)
- Merges audio clips at correct timestamps into the video with ffmpeg
- Requires OPENAI_API_KEY env var; gracefully skips if not set
- No-sandbox + disable-dev-shm-usage for headless Chrome compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 04:42:03 +00:00
snomiao
f3d9a8c2e4 feat: show test requirements from QA guide on report site
- Download QA guide artifact in report job
- Extract prerequisites, test focus, and steps from guide JSON
- Display below the purpose description: focus → prerequisites → steps
- Separated by a subtle divider with smaller font

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 03:28:03 +00:00
snomiao
831718bd50 feat: purpose description on report + multi-pass video link fix
- Report site shows "PR #N aims to..." or "Issue #N reports..." block
  above the video cards, extracted from pr-context.txt
- Multi-pass video links fall back to pass1 when qa-{os}.mp4 is 404
- More negative verdict patterns: "does not demonstrate", "never tested"
- Risk uses first word of Overall Risk (avoids "high confidence" match)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 03:23:50 +00:00
snomiao
7d3ddbf619 fix: verdict detection — more negative patterns, risk uses first word
- Add "does not demonstrate", "steps were not performed", "never tested"
  to NOT_REPRO patterns (fixes #9101 false positive)
- Risk detection uses first word of Overall Risk section instead of
  grepping entire text (fixes "high confidence" matching HIGH)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 02:23:18 +00:00
snomiao
a42240cb65 fix: use addScriptTag for keyboard HUD to avoid tsx __name issue
tsx compiles arrow functions with __name helpers that don't exist in
browser context. Using addScriptTag with plain JS string avoids this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 02:03:34 +00:00
snomiao
c957c0833b fix: remove TS type annotation from page.evaluate (browser context)
Set<string>() in page.evaluate causes __name ReferenceError in browser.
Use untyped Set() since browser JS doesn't support TS generics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 01:55:46 +00:00
snomiao
e01d4aaffc debug: add verdict count logging to deploy script 2026-03-27 01:54:14 +00:00
snomiao
3f226467cd fix: check negative verdicts before positive in per-report classification
"fails to reproduce" contains "reproduce" — must check negatives first
within each report. Across reports, REPRODUCED still wins (multi-pass).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 01:41:06 +00:00
snomiao
d4d8772ae7 feat: keyboard HUD overlay shows pressed keys in video
Injects a persistent overlay in bottom-right corner that displays
currently held keys (e.g. "⌨ Space", "⌨ CTRL+C"). Makes keyboard
interactions visible in the recording for both human and AI reviewers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 01:38:44 +00:00
snomiao
b578f8d7c4 feat: vertical box badge for multi-pass with breakdown
Multi-pass issues show a stacked box badge:
  ┌──────────────┐
  │  #7806 QA    │
  │ ✓ 1 reproduced    │
  │ ⚠ 1 inconclusive  │
  └──────────────┘

Single-pass issues keep the standard horizontal badge.
Badge colors: blue=reproduced, gray=not-repro, yellow=inconclusive.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 00:49:19 +00:00
snomiao
d5360ce45c feat: show pass counts in badge for multi-pass reports (X/Y REPRODUCED)
When multiple report files exist, badge shows "2/3 REPRODUCED" instead
of just "REPRODUCED". Single-pass issues still show plain verdict.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 00:45:16 +00:00
snomiao
0e6d9fd926 fix: REPRODUCED wins over INCONCLUSIVE in multi-pass badge
When multiple passes exist and one confirms while another is
inconclusive, the badge should show REPRODUCED. Previously
INCONCLUSIVE was checked first, hiding successful reproductions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 00:36:59 +00:00
snomiao
de810f88a4 fix: cloneNode uses Ctrl+C/V instead of right-click Clone menu
The "Clone" context menu item doesn't exist in Nodes 2.0 mode.
Using Ctrl+C/Ctrl+V works in both legacy and Nodes 2.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 00:22:31 +00:00
snomiao
722b01a253 fix: preflight performs actual repro steps, not just setup
- #10307: preflight clones KSampler node, hint says drag to overlap
- #7414: preflight clicks numeric widget, hint says drag to change value
- #7806: preflight takes baseline screenshot, hint gives exact coords
  for holdKeyAndDrag with spacebar
- Hints now reference "Preflight already did X, NOW do Y" pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 00:07:42 +00:00
snomiao
dfd19a3cf9 fix: tell agent what preflight already did to prevent repeated actions
Agent was wasting turns re-doing loadDefaultWorkflow and setSetting
that preflight already executed. Now the system prompt includes
"Already Done" section listing preflight actions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 23:40:48 +00:00
snomiao
3531e37ae7 fix: preflight actions + badge false-positive pattern
- Auto-execute prerequisite actions (enable Nodes 2.0, load default
  workflow) BEFORE the agentic loop starts. Agent model ignores prompt
  hints but preflight guarantees nodes are on canvas.
- Add "fails to reproduce" to NOT REPRODUCIBLE badge patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 23:25:41 +00:00
snomiao
024b231c05 feat: qa-issue label trigger + labels in issue context
- Add issues:[labeled] trigger and qa-issue label support
- Resolve github.event.issue.number for issue-triggered runs
- Include issue labels in context (feeds keyword matcher for hints)
- Remove qa-issue label after run completes (same as qa-changes/qa-full)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 22:49:00 +00:00
snomiao
0c22369c60 fix: keyword-driven action hints for agent issue reproduction
Scan issue context for keywords (clone, copy-paste, spacebar, resize,
sidebar, scroll, middle-click, node shape, Nodes 2.0, etc.) and inject
specific MUST-follow action steps into the agentic system prompt.

Addresses 9 INCONCLUSIVE issues where agent had actions available
but didn't know when to use them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 22:43:31 +00:00
snomiao
511fdf1b24 fix: restyle QA annotations to avoid misleading AI reviewer
- Annotations now use cyan dashed border + monospace "QA:" prefix
  instead of red solid labels that look like UI error messages
- Video review prompts explicitly tell reviewer to ignore QA annotations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 22:34:51 +00:00
snomiao
d756c362e3 fix: badge mismatch, multi-pass report overwrite, agent node creation
P1: Filter out QA bot's own comments from pr-context (INCONCLUSIVE loop)
P2: Grep only ## Summary section for verdict (false REPRODUCED fix)
P3: Strip markdown bold before matching Overall Risk section
P4: Deploy full placeholder page with spinner during CI
P5: Pass #NUM QA label to PREPARING/ANALYZING badges
P6: Add copyPaste, holdKeyAndDrag, resizeNode, middleClick actions
P7: preload=auto + custom seekbar (already deployed)
P8: Deploy FAILED badge on report job failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 22:02:17 +00:00
snomiao
ba512fd263 fix: video seeking — preload=auto, custom seekbar, _headers
- Change preload=metadata to preload=auto for full video download
- Add _headers file with Accept-Ranges for Cloudflare Pages
- Add custom seekbar (range input + buffer indicator) that works
  even without server HTTP range request support
- Seekbar shows buffered progress and allows dragging to any point

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 21:20:56 +00:00
snomiao
e1fb782832 fix: strengthen after-mode prompt to test PR-specific behavior
Previous prompt said "test the specific behavior" which was too vague,
leading to generic UI walkthroughs instead of targeted tests.

New prompt: explicitly instructs to read the diff, trigger the exact
scenario the PR fixes, and avoid generic menu screenshots.

Also added reload action to before/after prompt for state persistence tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 12:58:02 +09:00
snomiao
9c347642ba fix: badge mismatch, multi-pass report overwrite, agent node creation
- Fix quality badge now reads "## Overall Risk" section only
- Prevents false MAJOR ISSUES from severity labels or negated phrases
- "Low" risk → APPROVED, "High" → MAJOR ISSUES, "Medium" → MINOR ISSUES

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 03:47:54 +00:00
snomiao
5f8f40b559 fix: install pnpm before building PR frontend in sno-qa-* triggers
setup-frontend must run first to install node/pnpm, then rebuild
with PR code. Also re-install sno-skills deps after switching back
so QA scripts' dependencies are available.

Also gitignore .claude/scheduled_tasks.lock.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 12:17:26 +09:00
snomiao
264e71a9de fix: restore sno-skills scripts after building PR frontend
When triggered via sno-qa-* push, the workflow checks out the PR code
to build its frontend, but this replaces qa-record.ts which only
exists on sno-skills. Fix: build PR frontend, then checkout back to
sno-skills so QA scripts remain available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 11:56:12 +09:00
snomiao
49fa1b3caa fix: use array subshell instead of mapfile for macOS compat
mapfile is not available on macOS default bash.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:22:05 +09:00
snomiao
3142c8ead6 fix: resolve remaining shellcheck warnings in qa-deploy-pages.sh
- SC2231: quote glob expansions in for loop
- SC2002: use sed directly instead of cat | sed
- SC2086: quote variable in echo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:52:12 +09:00
snomiao
4310c5238a feat: clickable badges with #NUM label and copy button
- Badge generators accept optional label param (#NUM QA)
- Badge in PR/issue comments links to report site
- Report site shows badge with copy-to-clipboard button
- Copy button produces markdown: [![QA Badge](url/badge.svg)](url/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 15:34:23 +00:00
snomiao
a7d7d39712 fix: resolve CI failures — test, shellcheck, format
- Update DefaultThumbnail test to match size-full class change
- Fix shellcheck warnings in qa-batch.sh (SC2001, SC2207)
- Fix shellcheck warnings in qa-deploy-pages.sh (SC2034, SC2235, SC2231, SC2002)
- Add qa-report-template.html to oxfmt ignore (minified, not formattable)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:33:29 +09:00
snomiao
24ac6f1566 fix: badge pattern too narrow/broad, multi-pass video discovery
- "confirmed" didn't match "confirms"/"reproducible" — use "reproduc|confirm" stem
- "partial" matched unrelated text — require "partially reproduced" specifically
- collectVideoCandidates now finds qa-session-*.mp4 for multi-pass reviews

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 12:18:53 +00:00
snomiao
f9a5baba1a fix: badge mismatch, multi-pass report overwrite, agent node creation
- Check INCONCLUSIVE before reproduced/confirmed in badge detection
- Exclude markdown headings from reproduced grep match
- Add --pass-label to qa-video-review.ts for unique multi-pass filenames
- Pass pass label from workflow YAML when reviewing numbered sessions
- Collect all pass-specific reports in deploy script HTML
- Add addNode/cloneNode convenience actions to qa-record agent
- Improve strategy hints for visual/rendering bug reproduction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 10:48:38 +00:00
snomiao
628f64631b chore: tidy PR for merge — resolve TODOs, fix misplaced import
- Remove push trigger (was for dev testing only)
- Restore concurrency group (was commented out for dev)
- Move misplaced import in qa-analyze-pr.ts to top of file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 19:40:55 +09:00
snomiao
bc38b2ce13 fix: extract deploy step into script to fix expression length limit
The Cloudflare Pages deploy step exceeded GitHub Actions' 21000 char
expression limit due to inline HTML/CSS/JS. Extract to
scripts/qa-deploy-pages.sh + scripts/qa-report-template.html.
2026-03-25 09:24:17 +00:00
snomiao
ca8e4d2a29 fix: enforce menu navigation pattern + add CI job link to report
- Strengthen prompt: MUST use openMenu → hoverMenuItem → clickMenuItem
  in that order. Previous runs skipped openMenu causing silent failures.
- Add CI Job link to the QA report site header for quick navigation
  to the GitHub Actions run that generated the report.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 17:46:46 +09:00
snomiao
55ce174c5b fix: split dual badge generator into separate step to fix expression length
GitHub Actions has a 21000 char limit per expression. The combined
badge setup step exceeded this after adding the dual badge generator.
Split into its own step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 08:37:23 +00:00
snomiao
49176271f8 chore: retrigger QA pipeline 2026-03-25 06:23:00 +00:00
snomiao
50518449fc fix: remove Bug: prefix from PR badge, just show REPRODUCED directly
Badge now reads: QA Bot | REPRODUCED | Fix: APPROVED
Not all issues are bugs — could be feature requests too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:21:02 +09:00
snomiao
65aa03b20d fix: use blue for REPRODUCED badge (success, not failure)
Reproducing a bug is a successful outcome for the QA bot.
Blue (#2196f3) = bot succeeded. Red = bot found problems with the fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:19:28 +09:00
snomiao
ce59b6a431 feat: combine PR bug+fix into single dual-segment badge
PRs now show one badge with three segments:
  QA Bot | Bug: REPRODUCED | Fix: APPROVED

Instead of two separate badges. Uses gen-badge-dual.sh which
renders label + bug status + fix status in one SVG.

Issues still use single two-segment badge:
  QA Bot | FINISHED: REPRODUCED

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:14:35 +09:00
snomiao
203e8c8a60 feat: add visible cursor overlay and annotation action to QA recorder
- Inject fake cursor (red dot with click animation) via addInitScript
  since headless Chrome doesn't render the system cursor in video
- Add hover-before-click delay to clickByText and canvas clicks
  so viewers can see where the cursor moves before clicking
- Add 'annotate' action: shows a floating label at (x,y) for N ms
  so AI can draw viewer attention to important UI state changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:50:25 +09:00
snomiao
b485d22760 fix: feed QA guide + issue context to agentic reproduce loop
Root cause: runAgenticLoop never read the QA guide — agent saw
"No issue context provided" for issues. Now reads qaGuideFile,
parses structured fields, and injects into system prompt.

Also: fetch issue body via gh issue view in workflow, increase
budget to 120s/30 turns, add focus reminders, smarter stuck
detection (50px grid normalization + action-type frequency),
reject invalid click targets, add loadDefaultWorkflow and
openSettings convenience actions, strategy hints in prompt.

Fix pre-existing typecheck error in eslint.config.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 05:47:04 +00:00
snomiao
2d1088f79e feat: dual badges for PRs — bug reproduction + fix quality
PRs now get two separate badges:
- Bug: REPRODUCED / NOT REPRODUCIBLE / PARTIAL (before branch)
- Fix: APPROVED / MAJOR ISSUES / MINOR ISSUES (after branch)

Issues keep a single badge: FINISHED: REPRODUCED / etc.

Both badge-bug.svg and badge-fix.svg served from the deploy site.
PR comment shows all three: ![badge] ![bug] ![fix]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:39:15 +09:00
snomiao
1fe0f97aa5 fix: badge FINISHED state includes result sub-state
FINISHED is not standalone — always shows result:
- FINISHED: REPRODUCED / NOT REPRODUCIBLE / PARTIAL (issues)
- FINISHED: APPROVED / MAJOR ISSUES / MINOR ISSUES (PRs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:25:53 +09:00
snomiao
178ecc6746 feat: add SVG status badge to QA report site
Badge shows QA pipeline status, deployed at each stage:
- PREPARING (blue) — setting up artifacts
- ANALYZING (orange) — running video review
- Final status with color:
  - Issues: REPRODUCED (red) / NOT REPRODUCIBLE (gray) / PARTIAL (yellow)
  - PRs: APPROVED (green) / MAJOR ISSUES (red) / MINOR ISSUES (yellow)

Badge served as /badge.svg from the same Cloudflare Pages site.
Included in PR comment as ![QA Badge](url/badge.svg).

Also restore @ts-expect-error for import-x plugin type incompatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:04:57 +09:00
GitHub Action
20f878f929 [automated] Apply ESLint and Oxfmt fixes 2026-03-25 04:31:51 +00:00
snomiao
712c386a69 fix: regenerate pnpm-lock.yaml after rebase
The rebase introduced a duplicated mapping key in the lockfile,
causing ERR_PNPM_BROKEN_LOCKFILE in CI.
2026-03-25 04:28:26 +00:00
snomiao
8f6fa738a5 fix: use correct flash model name for agentic loop 2026-03-25 03:19:28 +00:00
snomiao
387862d8b9 feat: agentic screenshot feedback loop + multi-pass recording
Replace single-shot step generation in reproduce mode with an agentic
loop where Gemini sees the screen after each action and decides what
to do next. For multi-bug issues, decompose into sub-issues and run
separate recording passes.

- Extract executeAction() from executeSteps() for reuse
- Add reload and done action types
- Add captureScreenshotForGemini() (JPEG q50, ~50KB)
- Add runAgenticLoop() with sliding window history (3 screenshots)
- Add decomposeIssue() for multi-pass recording (1-3 sub-issues)
- Update workflow to handle numbered session videos (qa-session-1, etc.)
2026-03-25 03:18:01 +00:00
snomiao
ff98eb13e4 feat: add frame-by-frame video controls to QA report player
- Add custom video controls below each video with frame stepping
- Frame back/forward buttons (1 frame at 30fps, 10 frames skip)
- Speed selector: 0.1x, 0.25x, 0.5x (default), 1x, 1.5x, 2x
- Keyboard shortcuts: arrow keys for frame step, space for play/pause
- SMPTE-style timecode display (m:ss.ms)
- Default 0.5x speed since AI operates UI faster than humans
- Videos no longer autoplay (pause on load for inspection)
- Zero external dependencies (pure HTML5 video API)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:18:01 +00:00
snomiao
f4f80d179f fix: cap reproduce video at 5min, skip env setup in Phase 4
- Reproduce video must be max 5 minutes (short, focused demo)
- Phase 4 reuses the environment from Phase 3 (no re-setup)
- Use video-start/video-stop commands (not --save-video flag)
- Start recording right before steps, stop immediately after

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:18:01 +00:00
snomiao
f758e16b72 feat: add reproduce-issue skill with two-video architecture
New Claude agent-driven issue reproduction skill that:
- Phase 1-2: Research issue and set up environment (custom nodes, workflows, settings)
- Phase 3: Record research video while exploring interactively via playwright-cli
- Phase 4: Record clean reproduce video with only the minimal repro steps
- Phase 5: Generate structured reproduction report

Key difference from the old approach: Claude agent explores and adapts
instead of blindly executing a Gemini-generated static plan.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:18:01 +00:00
snomiao
45e309c5f8 fix: add default workflow node positions to QA prompts
Gemini was right-clicking empty canvas instead of nodes because it
didn't know where the default workflow nodes are positioned. Now the
prompt includes approximate coordinates for all 7 default nodes and
clarifies the difference between node context menu vs canvas menu.

Also fixes TS2352 in page.evaluate by using double-cast through unknown.
2026-03-25 03:18:01 +00:00
snomiao
79df405733 feat: upgrade QA pipeline to Gemini 3.x models
- qa-record.ts, qa-analyze-pr.ts: gemini-2.5-flash/pro → gemini-3.1-pro-preview
- qa-video-review.ts, qa-generate-test.ts: gemini-2.5-flash → gemini-3-flash-preview
- pr-qa.yaml: update hardcoded model reference
- Add docs/qa/models.md with model comparison and rationale
2026-03-25 03:18:01 +00:00
snomiao
27c64e1092 feat: add loadWorkflow and setSetting actions to QA recorder
- Add loadWorkflow action to load workflow JSON from URL
- Add setSetting action to configure ComfyUI settings
- Improve reproduce mode prompt to emphasize establishing prerequisites
  (save workflow first, make it dirty, add needed nodes, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:18:01 +00:00
snomiao
a1307ef35c feat: add clickable issue/PR URLs to QA reports
- Add --target-url CLI option to qa-video-review.ts
- Include target URL in generated markdown reports
- Add clickable issue/PR link in deployed HTML report header
- Workflow passes the target URL automatically

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:17:56 +00:00
snomiao
11f02a2645 fix: resolve pre-existing typecheck errors
- Remove unused @ts-expect-error directives in eslint.config.ts
- Simplify LazyImage prop types from ClassValue to string
- Fix DialogInstance to avoid infinitely deep type instantiation
- Use cn() in DefaultThumbnail for class merging

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:17:30 +00:00
snomiao
b3bcc3ff4c fix: improve fillDialog and clickSubmenuItem for litegraph UI components
- fillDialog now tries: PrimeVue dialog → node search box → focused input → keyboard fallback
- clickSubmenuItem now tries: PrimeVue tiered menu → litegraph context menu → role menuitem
- Fixes double-click-to-add-node flow and right-click context menu clicks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:30 +00:00
snomiao
49e904918e fix: add step-level error resilience and click timeout in QA recording
- Wrap each step in try/catch so failed steps don't abort the recording
- Add 5s timeout to clickByText to prevent 30s hangs on disabled elements

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:29 +00:00
snomiao
78e5b1e1b3 feat: expand QA action set and improve issue reproduction depth
- Add new canvas actions: rightClick, doubleClick, clickCanvas,
  rightClickCanvas, dragCanvas, scrollCanvas for node graph interactions
- Increase reproduce mode step limit from 3-6 to 8-15 steps
- Add ComfyUI UI context to prompts (canvas layout, node interactions)
- Add anti-hallucination instructions to video review for issue mode
- Improve issue analysis prompt with detailed action descriptions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:29 +00:00
snomiao
81e3dc72f8 fix: use pulls API instead of gh pr view for PR/issue detection
gh pr view can't distinguish PRs from issues — it succeeds for both.
Use the REST API endpoint repos/{owner}/{repo}/pulls/{number} which
returns 404 for issues.
2026-03-25 03:17:29 +00:00
snomiao
936cf83337 fix: address Copilot review feedback on QA scripts
- Enforce requestTimeoutMs via Gemini SDK requestOptions
- Add 100MB video size check before base64 encoding
- Sanitize screenshot filenames to prevent path traversal
- Sort video files by mtime for reliable rename
- Validate --mode arg against allowed values
- Add Content-Length pre-check in downloadMedia
- Add GitHub domain allowlist for media downloads (SSRF mitigation)
- Add contents:write permission and git config for report job
- Update Node.js requirement in SKILL.md from 18+ to 22+
2026-03-25 03:17:29 +00:00
snomiao
1d880bf493 fix: gracefully skip invalid pressKey values in QA recording
Instead of crashing the entire recording session when Gemini generates
an invalid key name (e.g. "mouseWheelDown"), catch the error and
continue with remaining steps.
2026-03-25 03:17:29 +00:00
snomiao
da3e6cb4cf feat: add behavior changes summary table to QA video review
Add a "Behavior Changes" table (Behavior, Before, After, Verdict)
alongside the existing timeline comparison. This gives reviewers a
quick high-level view of all behavioral differences before diving
into the frame-by-frame timeline.
2026-03-25 03:17:29 +00:00
snomiao
a39e3054cf fix: format before/after comparison as table in QA video review
Instruct Gemini to output the Before vs After section as a markdown
table with Time, Type, Severity, Before, After columns for easier
comparison. Update HTML template table styles with fixed layout and
column widths optimized for the 5-column comparison format.
2026-03-25 03:17:29 +00:00
snomiao
5a6178e924 fix: handle array response from Gemini in analyze-pr
Gemini Pro with responseMimeType: 'application/json' returns a JSON
array [before, after] instead of {before, after}. Handle both shapes.
2026-03-25 03:17:29 +00:00
snomiao
a11e8a67f8 fix: make analyze-pr non-blocking and log Gemini response
- Log raw Gemini response for debugging when parsing fails
- Handle possible wrapper keys in response
- Make qa-before/qa-after run even if analyze-pr fails (only gate
  on resolve-matrix success)
2026-03-25 03:17:29 +00:00
snomiao
6a836b7c25 fix: extract PR number from sno-qa-<number> branch name
When running on push events for sno-qa-* branches without an open PR,
extract the PR number from the branch name so analyze-pr can fetch
the full PR thread for analysis.
2026-03-25 03:17:29 +00:00
snomiao
f91f94f71a feat: add analyze-pr job to QA pipeline
Add Gemini Pro-powered PR analysis that generates targeted QA guides
from the full PR thread (description, comments, screenshots, diff).
The analyze-pr job runs on lightweight ubuntu before recordings start,
producing qa-guide-before.json and qa-guide-after.json that are
downloaded by recording jobs to produce more focused test steps.

Graceful fallback: if analysis fails, recordings proceed without guides.
2026-03-25 03:17:29 +00:00
snomiao
7502528733 fix: download before/after artifacts into separate directories
download-artifact@v7 merges all files flat regardless of
merge-multiple setting. Use separate path dirs (before/after)
and copy all files into the report directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:29 +00:00
snomiao
0656091959 fix: set merge-multiple false for download-artifact v7
download-artifact@v7 defaults merge-multiple to true, which puts all
files flat in qa-artifacts/ instead of per-artifact subdirectories.
The merge step expects qa-artifacts/qa-before-{os}-{run}/ subdirs,
so the report directory never gets created and video review finds
no files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:29 +00:00
snomiao
6515170d08 refactor: split QA into parallel before/after jobs
Instead of running before/after sequentially in a single job with
fragile git stash/checkout gymnastics, split into two independent
parallel jobs on separate runners:

  resolve-matrix → qa-before (main) ─┐
                 → qa-after  (PR)   ─┴→ report

- qa-before: uses git worktree for clean main branch build
- qa-after: normal PR build via setup-frontend
- report: downloads both artifact sets, merges, runs Gemini review

Benefits:
- Clean workspace isolation (no git checkout origin/main -- .)
- ~2x faster (parallel execution)
- Each job gets its own ComfyUI server (no shared state)
- Eliminates entire class of workspace contamination bugs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:29 +00:00
snomiao
25cbe56a34 fix: select existing user from dropdown instead of re-creating
The Pre-seed step creates qa-ci via API, so the "New user" form
shows "already exists" error. Fix by selecting the existing user
from the dropdown first, falling back to a unique username.
2026-03-25 03:17:29 +00:00
snomiao
120a531ef9 fix: use login page directly instead of localStorage bypass
The localStorage userId bypass doesn't work because the server
validates user IDs and rejects the simple 'qa-ci' string. Instead,
detect the login page by its input fields and create a user via the
"New user" text input, which is how real users would log in.
2026-03-25 03:17:29 +00:00
snomiao
389f6ca6b8 fix: switch from Firefox to Chromium for WebGL canvas support
Firefox headless doesn't support WebGL, causing "getCanvas: canvas is
null" errors. Switch to Chromium which has full headless WebGL support.
Also fix login flow to wait for async router guard to settle and
create user via text input as fallback.
2026-03-25 03:17:29 +00:00
snomiao
6298fc3a58 fix: add debug screenshot and fallback for menu button click
Add coordinate fallback when .comfy-menu-button-wrapper selector isn't
found, and capture a debug screenshot after login to diagnose what the
page looks like when the editor UI fails to render.
2026-03-25 03:17:29 +00:00
snomiao
83702c2e87 fix: use proper CSS selectors for menu interactions in QA recording
The openComfyMenu was clicking at hardcoded coordinates (20, 67) which
missed the menu button. Now uses .comfy-menu-button-wrapper selector
matching the browser tests. Also fixes menu item hover/click selectors
to use .p-menubar-item-label and .p-tieredmenu-item classes, and adds
a wait for the editor UI to fully load before executing test steps.
2026-03-25 03:17:29 +00:00
snomiao
c5d207fa9a fix: bypass login in QA recordings with localStorage pre-seeding
The QA recordings were stuck on the user selection screen because CI
has no existing users. Fix by pre-seeding localStorage with userId,
userName, and TutorialCompleted before navigation, plus creating a
qa-ci user via API as a fallback.
2026-03-25 03:17:29 +00:00
snomiao
ab6dff02c9 feat: autoplay and loop videos on QA dashboard 2026-03-25 03:17:29 +00:00
snomiao
7396d39a6a fix: handle flat artifact layout when no report.md exists
The normalize step couldn't create the qa-report-* subdir because it
only looked for *-report.md files. Add fallback to detect webm files.
2026-03-25 03:17:29 +00:00
snomiao
99e6681237 fix: exclude known video names when renaming Playwright recordings
The AFTER step was renaming qa-before-session.webm instead of the new
recording. Filter out already-named files before picking the latest.
2026-03-25 03:17:29 +00:00
snomiao
ae29224874 fix: dismiss dropdown overlay before clicking Next in QA login flow
The user dropdown shows "No available options" in CI, and the overlay
blocks the Next button. Dismiss with Escape before attempting click.
2026-03-25 03:17:29 +00:00
snomiao
47e5c39ac9 fix: install main branch deps before building, then reinstall PR deps
Main branch imports @vueuse/router which isn't in PR's node_modules.
Need both: main deps for building main, PR deps for running QA scripts.
2026-03-25 03:17:29 +00:00
snomiao
28f530d53a fix: reinstall PR deps after main branch build to restore @google/generative-ai
The main branch build step was running pnpm install with main's lockfile,
which removed @google/generative-ai from node_modules. Move the reinstall
to after restoring PR files so the QA recording script can find its deps.
2026-03-25 03:17:28 +00:00
snomiao
282754743d fix: temporarily disable concurrency group to unstick QA runs 2026-03-25 03:17:28 +00:00
snomiao
0a91ec09e8 fix: restore PR files with git checkout HEAD instead of git checkout -
git checkout - uses @{-1} which requires a previous branch switch.
Since we use 'git checkout origin/main -- .' (file checkout, not branch
switch), there is no @{-1} ref. Use HEAD to restore from current branch.

Also restore proper concurrency group.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
25fd1b2700 fix: use unique concurrency group to unstick QA runs 2026-03-25 03:17:28 +00:00
snomiao
b9d5ff0f8d ci: trigger QA run 2026-03-25 03:17:28 +00:00
snomiao
746d465912 feat: integrate comfy-qa skill test plan into QA recording pipeline
Pass the comprehensive test plan from .claude/skills/comfy-qa/SKILL.md
to Gemini when generating test steps. This gives Gemini knowledge of all
12 QA categories (canvas, menus, sidebar, settings, etc.) so it picks
the most relevant tests for each PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
e314a18b90 fix: use vite build directly to skip nx typecheck dependency
nx build runs typecheck as a prerequisite (via @nx/vite/plugin config).
Use vite build directly for the main branch comparison build.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
78fb9ef27f fix: skip typecheck when building main branch for QA comparison
Main branch may have transient TS errors when built with the PR
branch's lockfile. Since we only need the dist for visual comparison,
run nx build directly instead of pnpm build (which includes typecheck).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
41999c2e0f refactor: replace Codex with direct Playwright recording in QA pipeline
Replace the unreliable codex exec approach with a Playwright script
(qa-record.ts) that uses Gemini to generate targeted test steps from
the PR diff, then executes them deterministically via Playwright's API.

Key changes:
- New scripts/qa-record.ts: Gemini generates JSON test actions, Playwright
  executes them with reliable helper functions (menu nav, dialog fill, etc.)
- Remove codex CLI and playwright-cli dependencies
- Remove 150+ lines of prompt templates from pr-qa.yaml
- Firefox headless with video recording (same approach proven locally)
- Fallback steps if Gemini fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
5c243d16be feat: auto-generate regression tests from QA reports
- Tighten BEFORE prompt to 15s snapshot (show old state only)
- Add qa-generate-test.ts: Gemini-powered Playwright test generator
- New workflow step: generate .spec.ts and push to {branch}-add-qa-test
- Tests assert UIUX behavior (tab names, dirty state, visibility)
2026-03-25 03:17:28 +00:00
snomiao
774dcd823e feat: before/after video comparison for QA pipeline
- Build both main (dist-before/) and PR (dist/) frontends in focused mode
- Run QA twice: BEFORE on main branch frontend, AFTER on PR branch
- Send both videos to Gemini in one request for comparative analysis
- Side-by-side dashboard layout with Before (main) / After (PR) panels
- Comparative prompt evaluates whether before confirms old behavior
  and after proves the fix works
- Falls back to single-video mode when no before video available
2026-03-25 03:17:28 +00:00
snomiao
b500f826fc fix: make QA videos seekable with faststart and frequent keyframes
moov atom was at end of file (8.6MB offset) — browser had to download
the entire video before seeking. Keyframes were only every 10 seconds.

Add -movflags +faststart (moov before mdat) and -g 60 (keyframe every
2.4s at 25fps) to ffmpeg conversion.
2026-03-25 03:17:28 +00:00
snomiao
1cd9e171c6 fix: issue cards instead of dense table, rename to comfy-qa.pages.dev
- Replace 6-column confirmed issues table with vertical card blocks
  using colored severity/timestamp/confidence badges
- Rename Cloudflare Pages project from comfyui-qa-videos to comfy-qa
2026-03-25 03:17:28 +00:00
snomiao
5c0bef9b72 fix: seekable video, hide empty cards, PR-aware video review
- Remove autoplay/loop so video timeline is seekable
- Skip card generation for platforms without recordings
- Add --pr-context flag to qa-video-review.ts so Gemini evaluates
  against PR purpose instead of just describing what happened
- Workflow now builds pr-context.txt from PR title/body/diff
2026-03-25 03:17:28 +00:00
snomiao
94e1388495 feat: redesign QA dashboard with modern frontend design
OKLCH color tokens, liquid glass card surfaces, Inter + JetBrains Mono
typography, grain texture overlay, staggered fade-up animations, pill
action buttons with SVG icons, and improved report table styling.
2026-03-25 03:17:28 +00:00
snomiao
0268e8f977 fix: make settings pre-seed non-fatal and try both API endpoints
The /api/settings endpoint returned 4xx in CI. Try both /api/settings
and /settings endpoints, and don't fail the job if neither works.
2026-03-25 03:17:28 +00:00
snomiao
4f1df7c7ce fix: pre-seed Comfy.TutorialCompleted to skip template gallery in QA
The Codex agent was spending 35s browsing the "Getting Started" template
gallery instead of testing the PR's changes. Pre-seeding this setting
via the ComfyUI API ensures the agent lands directly in the graph editor.
2026-03-25 03:17:28 +00:00
snomiao
3b2fdc786a fix: tighten focused QA prompt to only test PR-specific behavior
The Codex agent was spending time on login flow, template browsing,
and general smoke testing instead of testing the PR's actual changes.

Changes:
- Add 30-second time budget for video recording
- Move video-start AFTER login and editor verification
- Explicitly prohibit template browsing and sidebar exploration
- Reduce test steps to 3-6 targeted actions
- Restructure prompt with clear Instructions/Rules sections
2026-03-25 03:17:28 +00:00
snomiao
4d1ad4dcf0 fix: render markdown in QA reports with marked.js
Replace crude sed-based markdown conversion with client-side
rendering via marked.js CDN. Adds proper table, list, and
code styling for the report section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
7ba1aaed53 fix: run report job on workflow_dispatch events
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
09a3c10d50 refactor: replace GPT frame extraction with Gemini native video analysis
Replace the OpenAI GPT-based frame extraction approach (ffmpeg + screenshots)
with Gemini 2.5 Flash's native video understanding. This eliminates false
positives from frame-based analysis (e.g. "black screen = critical bug" during
page transitions) and produces dramatically better QA reviews.

Changes:
- Remove ffmpeg frame extraction, ffprobe duration detection, and all related
  logic (~365 lines removed)
- Add @google/generative-ai SDK for native video/mp4 upload to Gemini
- Update CLI: remove --max-frames, --min-interval-seconds, --keep-frames flags
- Update env: OPENAI_API_KEY → GEMINI_API_KEY
- Update workflow: swap API key secret and model in pr-qa.yaml
- Update report: replace "Frames analyzed" with "Video size"
- Add note in prompt that brief black frames during transitions are normal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 03:17:28 +00:00
snomiao
2f30fbe060 fix: use fill+click for quick login before video recording
playwright-cli doesn't support 'evaluate' command. Instead, instruct
Codex to quickly fill the username input and click Next on user-select
page BEFORE starting video recording, so the video only shows actual
QA testing.
2026-03-25 03:17:27 +00:00
snomiao
85adbe4878 fix: use evaluate to set localStorage before video recording
storageState config doesn't work with playwright-cli. Instead, use
evaluate to set Comfy.userId/userName after opening the page, then
navigate back. This skips user-select before video-start so the
recording only shows actual QA testing.
2026-03-25 03:17:27 +00:00
snomiao
0c707b5deb fix: pre-seed localStorage to skip user-select in QA runs
Write a Playwright storageState JSON with Comfy.userId/userName pre-set
so the app loads directly to the graph editor. Saves ~40s per QA run
that was wasted on navigating the user-select page.
2026-03-25 03:17:27 +00:00
snomiao
4f98518c22 fix: prefer explicit qa-session.webm over corrupt auto-recorded videos
The convert step was using find which picked up a 0-byte file from
playwright's videos/ directory instead of the valid qa-session.webm.
Now prefers qa-session.webm explicitly and skips empty files.
2026-03-25 03:17:27 +00:00
snomiao
ca394b9ff7 fix: improve focused QA prompt to test PR-specific behavior, not random walk 2026-03-25 03:17:27 +00:00
snomiao
51878db99d fix: re-add push trigger for sno-skills and sno-qa-* branches 2026-03-25 03:17:27 +00:00
snomiao
d0534003e7 fix: also install ffprobe for GPT video review frame extraction 2026-03-25 03:17:27 +00:00
snomiao
3d0cd72465 fix: use sudo for ffmpeg static binary extraction to /usr/local/bin 2026-03-25 03:17:27 +00:00
snomiao
82849df891 fix: use static ffmpeg binary instead of apt-get (avoids dpkg lock hang) 2026-03-25 03:17:27 +00:00
snomiao
1c62c0edc3 fix: add ffmpeg install back (not pre-installed on GH runners) 2026-03-25 03:17:27 +00:00
snomiao
6ea2ce755d fix: normalize flat artifact download into expected subdirectory 2026-03-25 03:17:27 +00:00
snomiao
8fc4480ee2 fix: pre-install chromium and clarify prompt for codex
Codex was using pnpm dlx instead of the global playwright-cli.
Pre-install chromium in setup step and make prompt explicit about
using the global command directly without pnpm/npx.
2026-03-25 03:17:27 +00:00
snomiao
3515a478fd fix: add debug output to video convert step 2026-03-25 03:17:27 +00:00
snomiao
11432992d3 fix: use danger-full-access sandbox for codex on GH Actions 2026-03-25 03:17:27 +00:00
snomiao
e619b0143a fix: use correct codex model name gpt-5.4-mini 2026-03-25 03:17:27 +00:00
snomiao
7d4a008f29 feat: switch QA from Claude Code to OpenAI Codex CLI
Replace claude --print with codex exec for cheaper QA runs.
Uses codex-mini-latest model ($1.50/$6 vs Sonnet $3/$15).
Uses existing OPENAI_API_KEY secret (no new secrets needed).
2026-03-25 03:17:27 +00:00
snomiao
a3e65140a9 fix: default to linux-only QA, full 3-OS only via qa-full label
Reduces per-run cost from ~$10-16 to ~$2.50 by defaulting to
Linux-only. Use qa-full label or workflow_dispatch for 3-OS runs.
2026-03-25 03:17:27 +00:00
snomiao
f55ab36dd7 fix: use explicit video-start/stop, remove ffmpeg install, use gpt-4.1-mini
- Replace saveVideo config (didn't produce video) with explicit
  playwright-cli video-start/video-stop commands in QA prompt
- Remove apt-get install ffmpeg step (pre-installed on GH runners)
- Switch video review model from gpt-4o to gpt-4.1-mini
2026-03-25 03:17:27 +00:00
snomiao
698a894b42 fix: use auto video recording and show GPT reports on QA site
- Enable saveVideo in playwright-cli config for real video recording
- Replace screenshot stitching with webm→mp4 conversion
- Move video review step before deploy so reports are included
- Add GPT video review reports inline on the Cloudflare Pages site
- Each video card now has expandable "GPT Video Review" section
2026-03-25 03:17:27 +00:00
snomiao
fbd7f404ef fix: configure playwright-cli outputDir and improve artifact collection
- Set .playwright/cli.config.json with outputDir pointing to screenshots/
- This way bare 'playwright-cli screenshot' auto-saves to the right place
- Create screenshot directory before Claude runs (don't rely on Claude)
- Collect step now searches working directory for stray PNGs
- Simplified prompt: no --filename needed, just 'playwright-cli screenshot'
2026-03-25 03:17:27 +00:00
snomiao
d633ce19a7 fix: stitch screenshots from correct directory and simplify prompt
Screenshots were saved to artifact root but stitch looked in frames/.
Now: prompt tells Claude to save to screenshots/ dir with numbered names,
collect step consolidates PNGs there, stitch step globs from screenshots/.
Removed video-start/video-stop (Claude doesn't use them).
2026-03-25 03:17:27 +00:00
snomiao
9a7b5f88a0 fix: use playwright-cli video recording and collect default output
- Add playwright-cli config with outputDir and saveVideo
- Use video-start/video-stop instead of relying on screenshot frames
- Add fallback artifact collection from .playwright-cli/ default dir
- Simplify prompts to focus on video recording workflow
2026-03-25 03:17:27 +00:00
snomiao
be261a8a86 fix: resolve QA_ARTIFACTS path in prompt so Claude gets the literal path
The escaped \$QA_ARTIFACTS in the heredoc produced literal text
'$QA_ARTIFACTS' in the prompt. Claude's Bash tool didn't reliably
expand this env var, so no screenshots or reports were saved.
Remove the escapes so the heredoc expands the variable to the actual
path (e.g. /home/runner/work/_temp/qa-artifacts).
2026-03-25 03:17:27 +00:00
snomiao
b2c31e785f fix: escape backticks in QA prompt heredoc to prevent command substitution
Backtick-wrapped playwright-cli examples in the unquoted heredoc were
being interpreted as bash command substitution, producing empty prompts.
Replace backtick syntax with plain "Run:" prefixed commands.
2026-03-25 03:17:27 +00:00
snomiao
475f9ae5f0 fix: reorganize QA CI — remove screen recording, merge video-review into report
- Remove all Xvfb/ffmpeg screen recording infrastructure from qa job
  (captured blank display since playwright-cli runs headless)
- Add screenshot instructions to QA prompts: Claude saves sequential
  frames to $QA_ARTIFACTS/frames/ after every interaction
- Stitch screenshots into video via ffmpeg in report job (2fps)
- Merge video-review job into report job (4 jobs → 3 jobs)
- Unified PR comment with video links + video review in <details> collapse
- Clean up stale QA_VIDEO_REVIEW_COMMENT markers from prior runs
2026-03-25 03:17:27 +00:00
GitHub Action
c329022e15 [automated] Apply ESLint and Oxfmt fixes 2026-03-25 03:17:27 +00:00
snomiao
63d0df9ff0 fix: harden setup-comfyui-server against shell injection
Move extra_server_params input to env var to prevent shell injection
from untrusted input. Replace wait-for-it pip dependency with a
cross-platform curl polling loop.
2026-03-25 03:17:27 +00:00
snomiao
c1593054fb feat: add comfy-qa skill and automated QA CI pipeline
Add Claude Code skills and a label-triggered QA workflow:

- .claude/skills/comfy-qa/SKILL.md: 12-category QA test plan using
  playwright-cli for browser automation
- .github/workflows/pr-qa.yaml: CI workflow triggered by qa-changes
  (focused, Linux) or qa-full (3-OS matrix) labels. Records screen via
  ffmpeg, runs Claude CLI with playwright-cli, deploys video gallery to
  Cloudflare Pages, posts PR comment with GIF thumbnails, and runs
  OpenAI vision-based video review
- scripts/qa-video-review.ts: frame extraction + GPT-4o analysis
- scripts/qa-video-review.test.ts: unit tests for video review
- knip.config.ts: resolve knip errors for ingest-types package
2026-03-25 03:17:26 +00:00
Luke Mino-Altherr
a44fa1fdd5 fix: tighten date detection regex in formatJsonValue() (#10110)
`formatJsonValue()` uses a loose regex `^\d{4}-\d{2}-\d{2}` to detect
date-like strings, which matches non-date strings like
`"2024-01-01-beta"`.

Changes:
- Require ISO 8601 `T` separator: `/^\d{4}-\d{2}-\d{2}T/`
- Validate parse result with `!Number.isNaN(date.getTime())`
- Use `d()` i18n formatter for consistency with `formatDate()` in the
same file
2026-03-24 19:46:58 -07:00
Christian Byrne
cc3acebceb feat: scaffold Astro 5 website app + design-system base.css [1/3] (#10140)
## Summary
Scaffolds the new apps/website/ Astro 5 + Vue 3 marketing site inside
the monorepo.

## Changes
- apps/website/ with package.json, astro.config.mjs, tsconfig, Nx
targets
- @comfyorg/design-system/css/base.css — brand tokens + fonts (no
PrimeVue)
- pnpm-workspace.yaml catalog entries for Astro deps
- .gitignore and env.d.ts for Astro

## Stack (via Graphite)
- **[1/3] Scaffold** ← this PR
- #10141 [2/3] Layout Shell
- #10142 [3/3] Homepage Sections

Part of the comfy.org website refresh (replacing Framer).

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10140-feat-scaffold-Astro-5-website-app-design-system-base-css-1-3-3266d73d365081688dcee0220a03eca4)
by [Unito](https://www.unito.io)
2026-03-24 19:02:10 -07:00
Alexander Brown
23c22e4c52 🧙 feat: wire ComfyHub publish wizard with profile gate, asset upload, and submission (#10128)
## Summary 🎯

Wire the ComfyHub publish flow end-to-end: profile gate, multi-step
wizard (describe, examples, finish), asset upload, and workflow
submission via hub API.

> *A wizard of steps, from describe to the end,*
> *Upload your assets, your workflows you'll send!*
> *With tags neatly slugged and thumbnails in place,*
> *Your ComfyHub publish is ready to race!* 🏁

## Changes 🔧

- 🌐 **Hub Service** — `comfyHubService` for profile CRUD, presigned
asset uploads, and workflow publish
- 📦 **Submission** — `useComfyHubPublishSubmission` orchestrates file
uploads → publish in one flow
- 🧙 **Wizard Steps** — Describe (name/description/tags) → Examples
(drag-drop reorderable images) → Thumbnail → Finish (profile card +
private-asset warnings)
- 🖼️ **ReorderableExampleImage** — Drag-drop *and* keyboard reordering,
accessible and fun
- 🏷️ **Tag Normalization** — `normalizeTags` slugifies before publishing
- 🔄 **Re-publish Prefill** — Fetches hub workflow metadata on
re-publish, with in-memory cache fallback
- 📐 **Schema Split** — Publish-record schema separated from
hub-workflow-metadata schema
- 🙈 **Unlisted Skip** — No hub-detail prefill fetch for unlisted records
- 👤 **Profile Gate** — Username validation in `useComfyHubProfileGate`
- 🧪 **Tests Galore** — New suites for DescribeStep, ExamplesStep,
WizardContent, PublishSubmission, comfyHubService, normalizeTags, plus
expanded PublishDialog & workflowShareService coverage

## Review Focus 🔍

> *Check the service, the schema, the Zod validation too,*
> *The upload orchestration — does it carry things through?*
> *The prefill fetch strategy: status → detail → cache,*
> *And drag-drop reordering — is it keeping its place?* 🤔

- 🌐 `comfyHubService.ts` — API contract shape, error handling, Zod
parsing
- 📦 `useComfyHubPublishSubmission.ts` — Upload-then-publish flow, edge
cases (no profile, no workflow)
- 🗂️ `ComfyHubPublishDialog.vue` — Prefill fetch strategy (publish
status → hub detail → cache)
- 🖼️ `ReorderableExampleImage.vue` — Drag-drop + keyboard a11y

## Testing 🧪

```bash
pnpm test:unit -- src/platform/workflow/sharing/
pnpm typecheck
```

> *If the tests all turn green and the types all align,*
> *Then merge it on in — this publish flow's fine!* 

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Dante <bunggl@naver.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: bymyself <cbyrne@comfy.org>
Co-authored-by: GitHub Action <action@github.com>
2026-03-25 09:30:25 +09:00
Comfy Org PR Bot
88b54c6775 1.43.4 (#10411)
Patch version increment to 1.43.4

**Base branch:** `main`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10411-1-43-4-32d6d73d365081659c13eb6896c5ed96)
by [Unito](https://www.unito.io)

---------

Co-authored-by: christian-byrne <72887196+christian-byrne@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Christian Byrne <cbyrne@comfy.org>
2026-03-24 16:34:02 -07:00
Christian Byrne
e40995fb6c fix: restore Firebase getAdditionalUserInfo for sign-up telemetry OR logic (#10453)
## Summary

Restores `getAdditionalUserInfo` from Firebase Auth so sign-up telemetry
fires when *either* Firebase or the UI context identifies a new user,
fixing a regression from #10388.

## Changes

- **What**: In `loginWithGoogle` and `loginWithGithub`, call
`getAdditionalUserInfo(result)` and OR it with the UI-provided
`options?.isNewUser` flag: `is_new_user: options?.isNewUser ||
additionalUserInfo?.isNewUser || false`. Added 8 parameterized unit
tests covering the OR truth table (Firebase true, UI true, both false,
null result).

## Review Focus

The OR semantics: if either source says new user, we send `sign_up`
telemetry. Previously only the UI flag was checked, which missed cases
where the user lands directly on the OAuth provider without going
through the sign-up view.

## Testing

Unit tests cover all branches of the OR logic. An e2e test is not
feasible here because it would require completing a real OAuth flow with
Google/GitHub (interactive popup, valid credentials, CAPTCHA) and
intercepting the resulting `getAdditionalUserInfo` response from
Firebase — none of which can be reliably automated in a headless
Playwright environment without a live Firebase project seeded with
disposable accounts.

Fixes #10447
2026-03-24 12:39:48 -07:00
Christian Byrne
908a3ea418 fix: resolve subgraph node slot link misalignment during workflow load (#9121)
## Summary

Fix subgraph node slot connector links appearing misaligned after
workflow load, caused by a transform desync between LiteGraph's internal
canvas transform and the Vue TransformPane's CSS transform.

## Changes

- **What**: Changed `syncNodeSlotLayoutsFromDOM` to use DOM-relative
measurement (slot position relative to its parent `[data-node-id]`
element) instead of absolute canvas-space conversion via
`clientPosToCanvasPos`. This makes the slot offset calculation
independent of the global canvas transform, eliminating the frame-lag
desync that occurred when `fitView()` updated `lgCanvas.ds` before the
Vue CSS transform caught up.
- **Cleanup**: Removed the unreachable fallback path that still used
`clientPosToCanvasPos` when the parent node element wasn't found (every
slot element is necessarily a child of a `[data-node-id]` element — if
`closest()` fails the element is detached and measuring is meaningless).
This also removed the `conv` parameter from `syncNodeSlotLayoutsFromDOM`
and `flushScheduledSlotLayoutSync`, and the
`useSharedCanvasPositionConversion` import.
- **Test**: Added a Playwright browser test that loads a subgraph
workflow with `workflowRendererVersion: "LG"` (triggering the 1.2x scale
in `ensureCorrectLayoutScale`) as a template (triggering `fitView`), and
verifies all slot connector positions are within bounds of their parent
node element.

## Review Focus

- The core change is in `useSlotElementTracking.ts` — the new
measurement approach uses `getBoundingClientRect()` on both the slot and
its parent node element, then divides by `currentScale` to get
canvas-space offsets. This is simpler and more robust than the previous
approach.
- SubgraphNodes were disproportionately affected because they are
relatively static and don't often trigger `ResizeObserver`-based
re-syncs that would eventually correct stale offsets.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9121-fix-resolve-subgraph-node-slot-link-misalignment-during-workflow-load-3106d73d365081eca413c84f2e0571d6)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <448862+DrJKL@users.noreply.github.com>
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-24 12:28:12 -07:00
AustinMroz
0ae4b78cbc Use native context menu for focused textareas (#10454)
The custom context menu provided by the frontend exposes widget specific
options. In order to support renaming, promotion, and favoriting, there
needs to be a way to access this context menu when targeting a textarea.
However, always displaying this custom context menu will cause the user
to lose access to browser specific functionality like spell checking,
translation, and the ability to copy paste text.

This PR updates the behaviour so that the native browser context menu
will display when the text area already has focus. Our custom frontend
context menu will continue to display when it does not.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10454-Use-native-context-menu-for-focused-textareas-32d6d73d365081909673d81d6a6ba054)
by [Unito](https://www.unito.io)
2026-03-24 12:25:35 -07:00
Christian Byrne
5f14276159 [feat] Improve group title layout (#9839)
Rebased and adopted from #5774 by @felixturner.

## Changes
- Remove unused font-size properties (`NODE_TEXT_SIZE`,
`NODE_SUBTEXT_SIZE`, `DEFAULT_GROUP_FONT`) from theme palettes and color
palette schema
- Replace `DEFAULT_GROUP_FONT`/`DEFAULT_GROUP_FONT_SIZE` with a single
`GROUP_TEXT_SIZE = 20` constant (reduced from 24px)
- Use `NODE_TITLE_HEIGHT` for group header height instead of `font_size
* 1.4`
- Vertically center group title text using `textBaseline = 'middle'`
- Use `GROUP_TEXT_SIZE` directly in TitleEditor instead of per-group
`font_size`
- Remove `font_size` from group serialization (no longer per-group
configurable)

## Original PR
Closes #5774

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9839-feat-Improve-group-title-layout-3216d73d36508112a0edc2a370af20ba)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Felix Turner <felixturner@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-24 12:25:18 -07:00
Christian Byrne
df81c66725 fix: prevent blueprint cache corruption on repeated placement (#9897)
Add `structuredClone()` in `getBlueprint()` so `_deserializeItems()`
mutations (node IDs, subgraph UUIDs) don't corrupt cached data.

Includes regression test verifying cache immutability.

Fixes COM-16026

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9897-fix-prevent-blueprint-cache-corruption-on-repeated-placement-3226d73d3650815c8594fa8d266e680f)
by [Unito](https://www.unito.io)
2026-03-24 12:24:41 -07:00
Christian Byrne
c5ad6cf1aa fix: migrate V1 tab state pointers during V1→V2 draft migration (#10007)
## Problem

When upgrading from V1 to V2 draft persistence, users' open workflow
tabs were lost. V1 stored tab state (open paths + active index) in
localStorage via `setStorageValue` fallback, but the V1→V2 migration
only migrated draft payloads — not these tab state pointers.

This meant that after upgrading, all previously open tabs disappeared
and users had to manually reopen their workflows.

## Solution

Add `migrateV1TabState()` to the V1→V2 migration path. After draft
payloads are migrated, the function reads the V1 localStorage keys
(`Comfy.OpenWorkflowsPaths` and `Comfy.ActiveWorkflowIndex`) and writes
them to V2's sessionStorage format via `writeOpenPaths()`.

The `clientId` is threaded from `useWorkflowPersistenceV2` (which has
access to `api.clientId`) through to `migrateV1toV2()`.

## Changes

- **`migrateV1toV2.ts`**: Added `migrateV1TabState()` + V1 key constants
for tab state
- **`useWorkflowPersistenceV2.ts`**: Pass `api.clientId` to migration
call
- **`migrateV1toV2.test.ts`**: Two new tests proving tab state migration
works

## Testing

TDD approach — RED commit shows the test failing, GREEN commit shows it
passing.

All 123 persistence tests pass.

- Fixes #9974

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10007-fix-migrate-V1-tab-state-pointers-during-V1-V2-draft-migration-3256d73d36508103b619e521c1b603f5)
by [Unito](https://www.unito.io)
2026-03-24 12:23:59 -07:00
Terry Jia
fe1fae4de1 fix: disable preload error toast triggered by third-party plugin failures (#10445)
## Summary
The preload error toast fires whenever any custom node extension fails
to load via dynamic `import()`. In practice, this is almost always
caused by third-party plugin bugs rather than ComfyUI core issues.
Common triggers include:

- Bare module specifiers (e.g., `import from "vue"`) that the browser
cannot resolve without an import map
- Incorrect relative paths to `scripts/app.js` due to nested web
directory structures
  - Missing dependencies on other extensions (e.g., `clipspace.js`)

Since many users have multiple custom nodes installed, the toast
frequently appears on startup — sometimes multiple times — with a
generic message that offers no actionable guidance. This creates
unnecessary alarm and support burden.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10445-fix-disable-preload-error-toast-triggered-by-third-party-plugin-failures-32d6d73d365081f281efcd6fe90642a5)
by [Unito](https://www.unito.io)
2026-03-24 13:34:34 -04:00
Johnpaul Chiwetelu
38aad02fb9 test: add e2e tests for all default keybindings (#10440)
## Summary
- Add 21 Playwright E2E tests covering all previously untested default
keybindings
- Add `isReadOnly()` and `getOffset()` helpers to `CanvasHelper`

### Tests added
| Group | Keybindings | Count |
|-------|------------|-------|
| Sidebar toggles | `w`, `n`, `m`, `a` | 4 |
| Canvas controls | `Alt+=`, `Alt+-`, `.`, `h`, `v` | 5 |
| Node state | `Alt+c`, `Ctrl+m` | 2 |
| Mode/panel toggles | `Alt+m`, `Alt+Shift+m`, `` Ctrl+` `` | 3 |
| Queue/execution | `Ctrl+Enter`, `Ctrl+Shift+Enter`, `Ctrl+Alt+Enter` |
3 |
| File operations | `Ctrl+s`, `Ctrl+o` | 2 |
| Graph operations | `Ctrl+Shift+e`, `r` | 2 |

## Test plan
- [x] `pnpm exec playwright test defaultKeybindings.spec.ts` — 21/21
pass
- [x] `pnpm typecheck:browser` — clean
- [x] `pnpm lint` — clean

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10440-test-add-e2e-tests-for-all-default-keybindings-32d6d73d36508124a510d4742f9fbdbb)
by [Unito](https://www.unito.io)
2026-03-24 16:12:59 +01:00
pythongosssss
7c4234cf93 feat: App mode - add execution status messages (#10369)
## Summary

Adds custom status messages that are shown under the previews in order
to provide additional progress feedback to the user

Nodes matching the words:

Save, Preview -> Saving
Load, Loader -> Loading
Encode -> Encoding
Decode -> Decoding
Compile, Conditioning, Merge, -> Processing
Upscale, Resize -> Resizing
ToVideo -> Generating video

Specific nodes:
KSampler, KSamplerAdvanced, SamplerCustom, SamplerCustomAdvanced ->
Generating
Video Slice, GetVideoComponents, CreateVideo -> Processing video
TrainLoraNode -> Training


## Changes

- **What**: 
- add specific node lookups for non-easily matchable patterns
- add regex based matching for common patterns
- show on both latent preview & skeleton preview 
- allow app mode workflow authors to override status with custom
property `Execution Message` (no UI for doing this)

## Review Focus

This is purely pattern/lookup based, in future we could update the
backend node schema to allow nodes to define their own status key.

## Screenshots (if applicable)

<img width="757" height="461" alt="image"
src="https://github.com/user-attachments/assets/2b32cc54-c4e7-4aeb-912d-b39ac8428be7"
/>

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10369-feat-App-mode-add-execution-status-messages-32a6d73d3650814e8ca2da5eb33f3b65)
by [Unito](https://www.unito.io)

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-03-24 07:58:35 -07:00
jaeone94
4bfc730a0e fix: restore workflow tabs on browser restart (#10336)
## Problem

Since PR #8520 (`feat(persistence): fix QuotaExceededError and
cross-workspace draft leakage`), all workflow tabs are lost when the
browser is closed and reopened.

PR #8520 moved tab pointers (`ActivePath`, `OpenPaths`) from
`localStorage` to `sessionStorage` for per-tab isolation. However,
`sessionStorage` is cleared when the browser closes, so the open tab
list is lost on restart. The draft data itself survives in
`localStorage` — only the pointers to which tabs were open are lost.

Reported in
[Comfy-Org/ComfyUI#12984](https://github.com/Comfy-Org/ComfyUI/issues/12984).
Confirmed via binary search: v1.40.9 (last good) → v1.40.10 (first bad).

## Changes

Dual-write tab pointers to both storage layers:

- **sessionStorage** (scoped by `clientId`) — used for in-session
refresh, preserves per-tab isolation
- **localStorage** (scoped by `workspaceId`) — fallback for browser
restart when sessionStorage is empty

Also adds:
- `storageAvailable` guard on write functions for consistency with
`writeIndex`/`writePayload`
- `isValidPointer` validation on localStorage reads to reject stale or
malformed data

## Benefits

- Workflow tabs survive browser restart (restores V1 behavior)
- Per-tab isolation is preserved for in-session use (sessionStorage is
still preferred when available)

## Trade-offs

- On browser restart, the restored tabs come from whichever browser tab
wrote last to localStorage. If Tab A had workflows 1,2,3 and Tab B had
4,5 — the user gets whichever tab wrote most recently. This is the same
limitation V1 had with `Comfy.OpenWorkflowsPaths` in localStorage.
- Previously (post-#8520), opening a new browser tab would only restore
the single most recent draft. With this fix, a new tab restores the full
set of open tabs from the last session. This may be surprising for
multi-tab users who expect a clean slate in new tabs.

## Test plan

- [x] `pnpm typecheck` passes
- [x] `pnpm lint` passes
- [x] All 121 persistence tests pass
- [x] Manual: open multiple workflow tabs → close browser → reopen →
tabs restored
- [x] Manual: open two browser tabs with different workflows → refresh
each → correct tabs in each

Fixes Comfy-Org/ComfyUI#12984

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10336-fix-restore-workflow-tabs-on-browser-restart-3296d73d365081b7a7d3e91427d08d17)
by [Unito](https://www.unito.io)
<!-- QA_REPORT_SECTION -->
---
## 🔍 Automated QA Report

| | |
|---|---|
| **Status** |  Complete |
| **Report** |
[sno-qa-10336.comfy-qa.pages.dev](https://sno-qa-10336.comfy-qa.pages.dev/)
|
| **CI Run** | [View
workflow](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/23373697656)
|

Before/after video recordings with **Behavior Changes** and **Timeline
Comparison** tables.
2026-03-24 17:30:36 +09:00
Yourz
001916edf6 refactor: clean up essentials node organization logic (#10433)
## Summary

Refactor essentials tab node organization to eliminate duplicated logic
and restrict essentials to core nodes only.

## Changes

- **What**: 
- Extract `resolveEssentialsCategory` to centralize category resolution
(was duplicated between filter and pathExtractor).
- Add `isCoreNode` guard so third-party nodes never appear in
essentials.
- Replace `indexOf`-based sorting with precomputed rank maps
(`ESSENTIALS_CATEGORY_RANK`, `ESSENTIALS_NODE_RANK`).

<img width="589" height="769" alt="image"
src="https://github.com/user-attachments/assets/66f41f35-aef5-4e12-97d5-0f33baf0ac45"
/>


## Review Focus

- The `isCoreNode` guard in `resolveEssentialsCategory` — ensures only
core nodes can appear in essentials even if a custom node sets
`essentials_category`.
- Rank map precomputation vs previous `indexOf` — functionally
equivalent but O(1) lookup.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10433-refactor-clean-up-essentials-node-organization-logic-32d6d73d36508193a4d1f7f9c18fcef7)
by [Unito](https://www.unito.io)

Co-authored-by: Amp <amp@ampcode.com>
2026-03-24 16:22:09 +08:00
jaeone94
66daa6d645 refactor: error system cleanup — store separation, DDD fix, test improvements (#10302)
## Summary

Refactors the error system to improve separation of concerns, fix DDD
layer violations, and address code quality issues.

- Extract `missingNodesErrorStore` from `executionErrorStore`, removing
the delegation pattern that coupled missing-node logic into the
execution error store
- Extract `useNodeErrorFlagSync` composable for node error flag
reconciliation (previously inlined)
- Extract `useErrorClearingHooks` composable with explicit callback
cleanup on node removal
- Extract `useErrorActions` composable to deduplicate telemetry+command
patterns across error card components
- Move `getCnrIdFromNode`/`getCnrIdFromProperties` to
`platform/nodeReplacement` layer (DDD fix)
- Move `missingNodesErrorStore` to `platform/nodeReplacement` (DDD
alignment)
- Add unmount cancellation guard to `useErrorReport` async `onMounted`
- Return watch stop handle from `useNodeErrorFlagSync`
- Add `asyncResolvedIds` eviction on `missingNodesError` reset
- Add `console.warn` to silent catch blocks and empty array guard
- Hoist `useCommandStore` to setup scope, fix floating promises
- Add `data-testid` to error groups, image/video error spans, copy
button
- Update E2E tests to use scoped locators and testids
- Add unit tests for `onNodeRemoved` restoration and double-install
guard

Fixes #9875, Fixes #10027, Fixes #10033, Fixes #10085

## Test plan

- [x] Existing unit tests pass with updated imports and mocks
- [x] New unit tests for `useErrorClearingHooks` (callback restoration,
double-install guard)
- [x] E2E tests updated to use scoped locators and `data-testid`
- [ ] Manual: verify error tab shows runtime errors and missing nodes
correctly
- [ ] Manual: verify "Find on GitHub", "Copy", and "Get Help" buttons
work in error cards

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10302-refactor-error-system-cleanup-store-separation-DDD-fix-test-improvements-3286d73d365081838279d045b8dd957a)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-24 16:43:22 +09:00
jaeone94
32a53eeaee fix: sync advanced inputs button color with node header (#10427)
## Summary
- The "Show Advanced Inputs" footer button was missing `headerColor`
style binding, causing it to not sync with the node header color (unlike
the "Enter Subgraph" button which already had it)
- Extracted the repeated `{ backgroundColor: headerColor }` inline style
(4 occurrences) into a `headerColorStyle` computed

## Screenshots 
before 
<img width="211" height="286" alt="스크린샷 2026-03-24 154312"
src="https://github.com/user-attachments/assets/edfd9480-04fa-4cd4-813d-a95adffbe2d3"
/>

after 
<img width="261" height="333" alt="스크린샷 2026-03-24 154622"
src="https://github.com/user-attachments/assets/eab28717-889e-4a6b-8775-bfc08fa727ff"
/>

## Test plan
- [x] Set a custom color on a node with advanced inputs and verify the
footer button matches the header color
- [x] Verify subgraph enter button still syncs correctly
- [x] Verify dual-tab layouts (error + advanced, error + subgraph) both
show correct colors

### Why no E2E test
Node header color is applied as an inline style via `headerColor` prop,
which is already passed and tested through the existing subgraph enter
button path. This change simply extends the same binding to the advanced
inputs buttons — no new data flow or interaction is introduced, so a
screenshot-based E2E test would add maintenance cost without meaningful
regression coverage.
2026-03-24 16:00:55 +09:00
Jin Yi
7c6ab19484 fix: manager progress toast and install button UX issues (#10423)
## Changes

- Reset `isRestartCompleted` in `closeToast()` so the "Apply Changes"
button appears correctly on subsequent installs instead of skipping to
the success message
- Add `@click.stop` on `PackInstallButton` to prevent click from
bubbling up to card selection, which was unintentionally opening the
right info panel

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10423-fix-manager-progress-toast-and-install-button-UX-issues-32d6d73d365081eba2c6d5b9b35ab26e)
by [Unito](https://www.unito.io)
2026-03-23 21:57:16 -07:00
Dante
8f9fe3b21e refactor: rebuild SingleSelect and MultiSelect with Reka UI (#9742)
## Summary
- Rebuild `SingleSelect` with Reka UI Select primitives (SelectRoot,
SelectTrigger, SelectContent, SelectItem)
- Rebuild `MultiSelect` with Reka UI Popover + Listbox (PopoverRoot,
ListboxRoot with `multiple`)
- Remove PrimeVue dependency from both components
- Preserve existing API contract for all consumers

## TODO
- [x] Re-evaluate MultiSelect implementation (ComboboxRoot with
`multiple` may be cleaner than Popover+Listbox)
- [x] E2E verification in actual app (AssetFilterBar,
WorkflowTemplateSelectorDialog, etc.)

## Test plan
- [x] Storybook visual verification for all SingleSelect/MultiSelect
stories
- [x] Keyboard navigation (arrow keys, Enter, Escape, typeahead)
- [x] Multi-selection with badge count
- [x] Search filtering in MultiSelect
- [x] Clear all functionality
- [x] Disabled state
- [x] Invalid state (SingleSelect)
- [x] Loading state (SingleSelect)

## screenshot
<img width="519" height="475" alt="스크린샷 2026-03-20 오전 12 12 37"
src="https://github.com/user-attachments/assets/ffc7f0b0-c88c-486b-a253-73a4da73c1de"
/>
<img width="842" height="554" alt="스크린샷 2026-03-20 오전 12 23 51"
src="https://github.com/user-attachments/assets/410551d4-c843-4898-b305-13a6ad6978ca"
/>

## video


https://github.com/user-attachments/assets/2fc3a9b9-2671-4c2c-9f54-4f83598afb53



Fixes #9700

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9742-refactor-rebuild-SingleSelect-and-MultiSelect-with-Reka-UI-3206d73d36508113bee2cf160c8f2d50)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-24 04:39:20 +00:00
Dante
4411d9a417 refactor: extract shared click-vs-drag guard utility (#10357)
## Summary

Extract duplicated click-vs-drag detection logic into a shared
`useClickDragGuard` composable and `exceedsClickThreshold` pure utility
function.

## Changes

- **What**: New `useClickDragGuard(threshold)` composable in
`src/composables/useClickDragGuard.ts` that stores pointer start
position and checks squared distance against a threshold. Also exports
`exceedsClickThreshold` for non-Vue contexts.
- Migrated `DropZone.vue`, `useNodePointerInteractions.ts`, and
`Load3d.ts` to use the shared utility
- `CanvasPointer.ts` left as-is (LiteGraph internal)
- All consumers now use squared-distance comparison (no `Math.sqrt` or
per-axis `Math.abs`)

## Review Focus

- The composable uses plain `let` state instead of `ref` since
reactivity is not needed for the start position
- `Load3d.ts` uses the pure `exceedsClickThreshold` function directly
since it is a class, not a Vue component
- Threshold values preserved per-consumer: DropZone=5,
useNodePointerInteractions=3, Load3d=5

Fixes #10356

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10357-refactor-extract-shared-click-vs-drag-guard-utility-32a6d73d3650816e83f5cb89872fb184)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-24 12:27:42 +09:00
Jin Yi
98d56bdada fix: improve settings dialog UX (#10396)
## Summary

Reduce settings dialog size and autofocus search input for better
usability.

## Changes

- **What**: Reduce dialog size from `md` to `sm` (max-width 1400px →
960px); autofocus search input on open

## Review Focus

User feedback indicated the settings dialog was too wide and search
required an extra click to focus.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10396-fix-improve-settings-dialog-UX-32c6d73d365081e29eceed55afde1967)
by [Unito](https://www.unito.io)

Co-authored-by: Amp <amp@ampcode.com>
2026-03-23 18:45:40 -07:00
Christian Byrne
b4bb6d9d13 feat: add grid view mode for multi-image batches in ImagePreview (#9241)
## Summary

Add grid view mode for multi-image batches in ImagePreview (Nodes 2.0),
replicating the Nodes 1.0 grid UX where all output images are visible as
clickable thumbnails.

## Changes

- **What**: Multi-image batches now default to a grid view showing all
thumbnails. Clicking a thumbnail switches to gallery mode for that
image. A persistent back-to-grid button sits next to navigation dots,
and hover action bars provide gallery toggle, download, and remove.
Replaced PrimeVue `Skeleton` with shadcn `Skeleton`. Added `viewGrid`,
`viewGallery`, `imageCount`, `galleryThumbnail` i18n keys.

## Review Focus

- Grid column count strategy: fixed breakpoints (2 cols ≤4, 3 cols ≤9, 4
cols 10+) vs CSS auto-fill
- Default view mode: grid for multi-image, gallery for single — matches
Nodes 1.0 behavior
- `object-contain` on thumbnails to avoid cropping (with `aspect-square`
containers for uniform cells)

Fixes #9162

<!-- Pipeline-Ticket: f8f8effa-adff-4ede-b1d3-3c4f04b9c4a0 -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9241-feat-add-grid-view-mode-for-multi-image-batches-in-ImagePreview-3136d73d36508166895ed6c635150434)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: Alexander Brown <448862+DrJKL@users.noreply.github.com>
Co-authored-by: Amp <amp@ampcode.com>
2026-03-24 09:49:44 +09:00
Matt Miller
6a9fb4e1d5 fix: show clear error dialog for 403 whitelist failures (#10402)
## Summary
- When the cloud backend returns a 403 (user not whitelisted), the
frontend showed a generic "Prompt Execution Error" dialog with a cryptic
message
- Now catches 403 responses specifically and shows an "Access
Restricted" dialog with the backend's actual error message
- Adds `status` field to `PromptExecutionError` so error handlers can
distinguish HTTP status codes

## Changes
- `api.ts`: Added optional `status` to `PromptExecutionError`, pass
`res.status` from `queuePrompt`
- `app.ts`: New `else if` branch in the prompt error handler for `status
=== 403` — shows "Access Restricted" with the backend message

## Backwards compatible
- **Old backend** (`"not authorized"`): Shows "Access Restricted: not
authorized"
- **New backend**
([cloud#2941](https://github.com/Comfy-Org/cloud/pull/2941), `"your
account is not whitelisted for this feature"`): Shows "Access
Restricted: your account is not whitelisted for this feature"
- No behavior change for non-403 errors

## Related
- Backend fix: Comfy-Org/cloud#2941
- Notion: COM-16179

## Test plan
- [ ] Submit a prompt as a non-whitelisted user → should see "Access
Restricted" dialog with clear message
- [ ] Submit a prompt as a whitelisted user → no change in behavior
- [ ] Submit a prompt that fails for other reasons (missing nodes, etc.)
→ existing error handling unchanged

🤖 Generated with [Claude Code](https://claude.com/claude-code)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10402-fix-show-clear-error-dialog-for-403-whitelist-failures-32c6d73d365081eb9528d7feac4e8681)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Matt Miller <mattmiller@Matts-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 23:57:12 +00:00
Deep Mehta
2838edbb53 feat: send missing node data to ClickHouse (#10132)
## Summary
- When users open/import a workflow with missing nodes, we already track
this via Mixpanel
- This adds a parallel fire-and-forget POST to
`/api/internal/cloud_analytics` so the data also lands in **ClickHouse**
as `frontend:missing_nodes_detected` events
- Payload includes `missing_class_types[]`, `missing_count`, and
`source` (file_button/file_drop/template/unknown)

## Motivation
The frontend is where the **high-value** missing node signal lives —
most users see "missing nodes" and never submit. The backend only
catches the rare case where someone submits anyway. This change captures
both sides.

Companion cloud PR: https://github.com/Comfy-Org/cloud/pull/2886

## Changes
- `MixpanelTelemetryProvider.ts`: Added
`reportMissingNodesToClickHouse()` private method, called from
`trackWorkflowImported()` and `trackWorkflowOpened()`
- Only fires when `missing_node_count > 0`
- Fire-and-forget (`.catch(() => {})`) — no impact on user experience
- Uses existing `api.fetchApi()` which handles auth automatically

## Test plan
- [ ] Open a workflow with missing nodes → verify
`frontend:missing_nodes_detected` event appears in ClickHouse
- [ ] Open a workflow with no missing nodes → verify no event is sent
(check network tab)
- [ ] Verify Mixpanel tracking still works as before

🤖 Generated with [Claude Code](https://claude.com/claude-code)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10132-feat-send-missing-node-data-to-ClickHouse-3266d73d365081559db5ed3efde33e95)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: GitHub Action <action@github.com>
2026-03-23 15:39:04 -07:00
Arthur R Longbottom
657ae6a6c3 fix: subgraph promoted widget input label rename (#10195)
## Summary

Promoted primitive subgraph inputs (String, Int) render their link
anchor at the header position instead of the widget row. Renaming
subgraph input labels breaks the match entirely, causing connections to
detach from their widgets visually.

## Changes

- **What**: Fix widget-input slot positioning for promoted subgraph
inputs in both LiteGraph and Vue (Nodes 2.0) rendering modes
- `_arrangeWidgetInputSlots`: Removed Vue mode branch that skipped
setting `input.pos`. Promoted widget inputs aren't rendered as
`<InputSlot>` Vue components (NodeSlots filters them out), so
`input.pos` is the only position fallback
- `drawConnections`: Added pre-pass to arrange nodes with unpositioned
widget-input slots before link rendering. The background canvas renders
before the foreground canvas calls `arrange()`, so positions weren't set
on the first frame
- `SubgraphNode`: Sync `input.widget.name` with the display name on
label rename and initial setup. The `IWidgetLocator` name diverged from
`PromotedWidgetView.name` after rename, breaking all name-based
slot↔widget matching (`_arrangeWidgetInputSlots`, `getWidgetFromSlot`,
`getSlotFromWidget`)

## Review Focus

- The `_arrangeWidgetInputSlots` rewrite iterates `_concreteInputs`
directly instead of building a spread-copy map — simpler and avoids the
stale index issue
- `input.widget.name` is now kept in sync with the display name
(`input.label ?? subgraphInput.name`). This is a semantic shift from
using the raw internal name, but it's required for all name-based
matching to work after renames. The value is overwritten on deserialize
by `_setWidget` anyway
- The `_widget` fallback in `_arrangeWidgetInputSlots` is a safety net
for edge cases where the name still doesn't match (e.g., stale cache)

Fixes #9998

## Screenshots
<img width="847" height="476" alt="Screenshot 2026-03-17 at 3 05 32 PM"
src="https://github.com/user-attachments/assets/38f10563-f0bc-44dd-a1a5-f4a7832575d0"
/>
<img width="804" height="471" alt="Screenshot 2026-03-17 at 3 05 23 PM"
src="https://github.com/user-attachments/assets/3237a7ee-f3e5-4084-b330-371def3415bd"
/>
<img width="974" height="571" alt="Screenshot 2026-03-17 at 3 05 16 PM"
src="https://github.com/user-attachments/assets/cafdca46-8d9b-40e1-8561-02cbb25ee8f2"
/>
<img width="967" height="558" alt="Screenshot 2026-03-17 at 3 05 06 PM"
src="https://github.com/user-attachments/assets/fc03ce43-906c-474d-b3bc-ddf08eb37c75"
/>


┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10195-fix-subgraph-promoted-widget-input-slot-positions-after-label-rename-3266d73d365081dfa623dd94dd87c718)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jaeone94 <jaeone.prt@gmail.com>
2026-03-23 14:33:03 -07:00
Comfy Org PR Bot
f37b0daa3a 1.43.3 (#10225)
Patch version increment to 1.43.3

**Base branch:** `main`

---------

Co-authored-by: christian-byrne <72887196+christian-byrne@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Christian Byrne <cbyrne@comfy.org>
2026-03-23 14:24:37 -07:00
Terry Jia
debbdfd1f7 test: verify gradient_stops survives object spread (#10408)
## Summary
follow up https://github.com/Comfy-Org/ComfyUI_frontend/pull/10406 to
add test

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10408-test-verify-gradient_stops-survives-object-spread-32c6d73d3650815da123c88f4b9b86ce)
by [Unito](https://www.unito.io)
2026-03-23 13:39:49 -07:00
Deep Mehta
9a5ee462f1 refactor: extract model-to-node mappings into separate data file (#10237)
## Summary
- Extract all 75 `quickRegister()` mapping entries from
`modelToNodeStore.ts` into a new `modelNodeMappings.ts` constants file
- The store now iterates over the `MODEL_NODE_MAPPINGS` array instead of
having inline calls
- **Zero behavioral change** — same mappings, same order, same runtime
behavior

## Motivation
Adding new model-to-node mappings is currently a code change to the
store. By separating the data into its own file:
- New mappings are a **pure data change** (append a tuple to an array)
- The data file can have its own CODEOWNERS entry, so mapping PRs can be
merged without requiring frontend team review
- Easier to audit — all mappings visible in one place without
interleaved store logic

### Before
```ts
// 250+ lines of quickRegister() calls mixed into store logic
quickRegister('checkpoints', 'CheckpointLoaderSimple', 'ckpt_name')
quickRegister('checkpoints', 'ImageOnlyCheckpointLoader', 'ckpt_name')
// ... 73 more
```

### After
```ts
// modelNodeMappings.ts — pure data
export const MODEL_NODE_MAPPINGS = [
  ['checkpoints', 'CheckpointLoaderSimple', 'ckpt_name'],
  ['checkpoints', 'ImageOnlyCheckpointLoader', 'ckpt_name'],
  // ...
]

// modelToNodeStore.ts — just iterates
for (const [modelType, nodeClass, key] of MODEL_NODE_MAPPINGS) {
  quickRegister(modelType, nodeClass, key)
}
```

## Test plan
- [ ] "Use" button in model browser still works for all model types
- [ ] No regressions in model-to-node resolution (same mappings, same
order)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10237-refactor-extract-model-to-node-mappings-into-separate-data-file-3276d73d365081988656e2ddae772bbc)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: GitHub Action <action@github.com>
2026-03-23 13:26:42 -07:00
Johnpaul Chiwetelu
cff6a2255c fix: skip scheduled workflow runs on forks (#10407)
Adds `if: github.repository == 'Comfy-Org/ComfyUI_frontend'` to the
`Weekly Documentation Check` and `Release: Version Bump` workflows so
their scheduled runs are skipped on forks (where they fail due to
missing secrets).

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10407-fix-skip-scheduled-workflow-runs-on-forks-32c6d73d365081d1b847d9f26b20efeb)
by [Unito](https://www.unito.io)
2026-03-23 20:00:47 +00:00
AustinMroz
860d049487 Fix snap offset for reroutes and subgraph IO (#10229)
When snapping to grid, reroutes and subgraph IO would snap at an awful y
offset.

| Before | After |
| ------ | ----- |
| <img width="360" alt="before"
src="https://github.com/user-attachments/assets/285cfd52-2242-4242-b031-926677575542"
/> | <img width="360" alt="after"
src="https://github.com/user-attachments/assets/4920ef9b-8eb1-4330-8eea-9992ad662018"
/>|


Regretfully, the PR leaves more magic constants than I would like.
There's a hardcoded `0.7` multiplier on `NODE_SLOT_HEIGHT` that isn't
defined as a constant, and it seems circular imports prevent constants
being used to declare the subgraphIO `roundedRadius`

See #8838 (Sorry for the delay. This change wound up being real simple)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10229-Fix-snap-offset-for-reroutes-and-subgraph-IO-3276d73d365081c2866dc388898a1be4)
by [Unito](https://www.unito.io)
2026-03-23 12:59:47 -07:00
Terry Jia
2d88720d80 fix: make gradient_stops enumerable so it survives object spread (#10406)
## Summary
Object.defineProperty defaults to enumerable: false for new properties.
NodeWidgets.vue spreads widget options into a new object, dropping
non-enumerable properties — gradient_stops was silently lost, causing
the gradient slider to fall back to the default black-to-white gradient.

## Screenshots (if applicable)

before
<img width="1679" height="1271" alt="image"
src="https://github.com/user-attachments/assets/60a20163-56e5-4438-9c36-ea42c23c7495"
/>

after
<img width="1686" height="1421" alt="image"
src="https://github.com/user-attachments/assets/5057b3d4-32ee-4abc-9118-cc2c3f85ae85"
/>

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10406-fix-make-gradient_stops-enumerable-so-it-survives-object-spread-32c6d73d36508131b7c8c3ab3e93d221)
by [Unito](https://www.unito.io)
2026-03-23 15:55:47 -04:00
Christian Byrne
818db0d49e fix: use UI flow context for sign_up vs login telemetry (#10388)
## Problem

GA4 shows **zero** `sign_up` events from cloud.comfy.org. All new users
(Google/GitHub) are tracked as `login` instead of `sign_up`.

**Root cause:** `getAdditionalUserInfo(result)?.isNewUser` from Firebase
is unreliable for popup auth flows — it compares `creationDate` vs
`lastSignInDate` timestamps, which can differ even for genuinely new
users. When it returns `null` or `false`, the code defaults to pushing a
`login` event instead of `sign_up`.

**Evidence:** GA4 Exploration filtered to `sign_up` + `cloud.comfy.org`
shows 0 events, while `login` shows 8,804 Google users, 519 email, 193
GitHub — all new users are being misclassified as logins.

(Additionally, the ~300 `sign_up` events visible in GA4 are actually
from `blog.comfy.org` — Substack newsletter subscriptions — not from the
app at all.)

## Fix

Use the UI flow context to determine `is_new_user` instead of Firebase's
unreliable API:
- `CloudSignupView.vue` → passes `{ isNewUser: true }` (user is on the
sign-up page)
- `CloudLoginView.vue` → no flag needed, defaults to `false` (user is on
the login page)
- `SignInContent.vue` → passes `{ isNewUser: !isSignIn.value }` (dialog
toggles between sign-in/sign-up)
- Removes the unused `getAdditionalUserInfo` import

## Changes

- `firebaseAuthStore.ts`: `loginWithGoogle`/`loginWithGithub` accept
optional `{ isNewUser }` parameter instead of calling
`getAdditionalUserInfo`
- `useFirebaseAuthActions.ts`: passes the option through
- `CloudSignupView.vue`: passes `{ isNewUser: true }`
- `SignInContent.vue`: passes `{ isNewUser: !isSignIn.value }`

## Testing

- All 32 `firebaseAuthStore.test.ts` tests pass
- All 19 `GtmTelemetryProvider.test.ts` tests pass
- Typecheck passes

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10388-fix-use-UI-flow-context-for-sign_up-vs-login-telemetry-32b6d73d3650811e96cec281108abbf3)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-23 10:11:40 -07:00
Hunter
cd45efa983 feat: differentiate personal/team pricing table with two-stage team workspace flow (#9901)
## Summary

Differentiates the subscription pricing dialog between personal and team
workspaces with distinct visual treatments and a two-stage team
workspace upgrade flow.

### Changes

- **Personal pricing dialog**: Shows "P" avatar badge, "Plans for
Personal Workspace" header, and "Solo use only – Need team workspace?"
banner on each tier card
- **Team pricing dialog**: Shows workspace avatar, "Plans for Team
Workspace" header (emerald), green "Invite up to X members" badge, and
emerald border on Creator card
- **Two-stage upgrade flow**: "Need team workspace?" → closes pricing →
opens CreateWorkspaceDialog → sessionStorage flag → page reload →
WorkspaceAuthGate auto-opens team pricing dialog
- **Spacing**: Reduced vertical gaps/padding/font sizes so the table
fits without scrolling

### Key decisions

- sessionStorage key `comfy:resume-team-pricing` bridges the page reload
during workspace creation
- `onChooseTeam` prop is conditionally passed only to the personal
variant
- `resumePendingPricingFlow()` is called from WorkspaceAuthGate after
workspace initialization

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9901-feat-differentiate-personal-team-pricing-table-with-two-stage-team-workspace-flow-3226d73d365081e7af60dcca86e83673)
by [Unito](https://www.unito.io)
2026-03-23 09:17:19 -07:00
AustinMroz
bd322314bc Display a separate indicator for pending jobs in app mode (#10382)
Previously, a pending, but not yet running job was displayed as an empty
skeleton. It is instead updated to display an animated `lucide--loader`
icon.

<img width="1544" height="1347" alt="image"
src="https://github.com/user-attachments/assets/4f82185c-97dc-44af-8ea1-a012fb992fe2"
/>

As this shifts the activeQueueItem component to display even when only
the single pending item is in the queue, the count badge is also update
to not display when this is the case.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10382-Display-a-separate-indicator-for-pending-jobs-in-app-mode-32a6d73d36508189b3a9ff4b84993a98)
by [Unito](https://www.unito.io)
2026-03-23 09:09:09 -07:00
Dante
b33c8c2d30 revert: fix: prevent survey popup when surveyed feature is inactive (#10360) (#10393)
## Summary

Reverts #10360

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10393-revert-fix-prevent-survey-popup-when-surveyed-feature-is-inactive-10360-32b6d73d36508132aff9c0e21529efac)
by [Unito](https://www.unito.io)
2026-03-22 14:40:21 +09:00
271 changed files with 22289 additions and 2711 deletions

View File

@@ -0,0 +1,278 @@
---
name: reproduce-issue
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
---
# Issue Reproduction Skill
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
## Architecture
Two videos are produced:
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
```
Phase 1: Research → Read issue, understand prerequisites
Phase 2: Environment → Install custom nodes, load workflows, configure settings
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
Phase 5: Report → Generate a structured reproduction report
```
## Prerequisites
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
- `gh` CLI (authenticated, for reading issues)
- ComfyUI backend with Python environment (for installing custom nodes)
## Phase 1: Research the Issue
1. Fetch the issue details:
```bash
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
```
2. Extract from the issue body:
- **Reproduction steps** (the exact sequence)
- **Prerequisites**: specific workflows, custom nodes, settings, models
- **Environment**: OS, browser, ComfyUI version
- **Media**: screenshots or videos showing the bug
3. Search the codebase for related code:
- Find the feature/component mentioned in the issue
- Understand how it works currently
- Identify what state the UI needs to be in
## Phase 2: Environment Setup
Set up everything the issue requires BEFORE attempting reproduction.
### Custom Nodes
If the issue mentions custom nodes:
```bash
# Find the custom node repo
# Clone into ComfyUI's custom_nodes directory
cd <comfyui_path>/custom_nodes
git clone <custom_node_repo_url>
# Install dependencies if needed
cd <custom_node_name>
pip install -r requirements.txt 2>/dev/null || true
# Restart ComfyUI server to load the new nodes
```
### Workflows
If the issue references a specific workflow:
```bash
# Download workflow JSON if a URL is provided
curl -L "<workflow_url>" -o /tmp/test-workflow.json
# Load it via the API
curl -X POST http://127.0.0.1:8188/api/workflow \
-H "Content-Type: application/json" \
-d @/tmp/test-workflow.json
```
Or load via playwright-cli:
```bash
playwright-cli goto "http://127.0.0.1:8188"
# Drag-and-drop or use File > Open to load the workflow
```
### Settings
If the issue requires specific settings:
```bash
# Use playwright-cli to open settings and change them
playwright-cli press "Control+,"
playwright-cli snapshot
# Find and modify the relevant setting
```
## Phase 3: Interactive Exploration — Research Video
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
```bash
# Open browser and start video recording
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
# Take a snapshot to see current state
playwright-cli snapshot
# Interact based on what you see
playwright-cli click <ref>
playwright-cli fill <ref> "text"
playwright-cli press "Control+s"
# Check results
playwright-cli snapshot
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
```
### Key Principles
- **Observe before acting**: Always `snapshot` before interacting
- **Retry and adapt**: If a step fails, try a different approach
- **Document what works**: Keep notes on which steps trigger the bug
- **Don't give up**: Try multiple approaches if the first doesn't work
- **Establish prerequisites**: Many bugs require specific UI state:
- Save a workflow first (File > Save)
- Make changes to dirty the workflow
- Open multiple tabs
- Add specific node types
- Change settings
- Resize the window
### Common ComfyUI Interactions via playwright-cli
| Action | Command |
| ------------------- | -------------------------------------------------------------- |
| Open hamburger menu | `playwright-cli click` on the C logo button |
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
| Add node | Double-click canvas → type node name → select from results |
| Connect nodes | Drag from output slot to input slot |
| Save workflow | `playwright-cli press "Control+s"` |
| Save As | Menu > File > Save As |
| Select node | Click on the node |
| Delete node | Select → `playwright-cli press "Delete"` |
| Right-click menu | `playwright-cli click <ref> --button right` |
| Keyboard shortcut | `playwright-cli press "Control+z"` |
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
Once the bug is confirmed, **stop the research video** and **close the research browser**:
```bash
playwright-cli video-stop
playwright-cli close
```
Now start a **fresh browser session** for the clean reproduce video (Video 2).
**IMPORTANT constraints:**
- **Max 5 minutes** — the reproduce video must be short and focused
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
1. **Open browser and start recording**:
```bash
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
```
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
3. **Take key screenshots** at critical moments:
```bash
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
# ... trigger the bug ...
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
```
4. **Stop recording and close** immediately after the bug is demonstrated:
```bash
playwright-cli video-stop
playwright-cli close
```
## Phase 5: Generate Report
Create a reproduction report at `tmp/qa/reproduce-report.md`:
```markdown
# Issue Reproduction Report
- **Issue**: <issue_url>
- **Title**: <issue_title>
- **Date**: <today>
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
## Environment
- ComfyUI Server: <url>
- OS: <os>
- Custom Nodes Installed: <list or "none">
- Settings Changed: <list or "none">
## Prerequisites
List everything that had to be set up before the bug could be triggered:
1. ...
2. ...
## Reproduction Steps
Minimal steps to reproduce (the clean sequence):
1. ...
2. ...
3. ...
## Expected Behavior
<from the issue>
## Actual Behavior
<what actually happened>
## Evidence
- Research video: `research-video/video.webm` (full exploration session)
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
- Screenshots: `before-bug.png`, `bug-visible.png`
## Root Cause Analysis (if identified)
<code pointers, hypothesis about what's going wrong>
## Notes
<any additional observations, workarounds discovered, related issues>
```
## Handling Failures
If the bug **cannot be reproduced**:
1. Document what you tried and why it didn't work
2. Check if the issue was already fixed (search git log for related commits)
3. Check if it's environment-specific (OS, browser, specific version)
4. Set report status to "Not Reproduced" with detailed notes
5. The report is still valuable — it saves others from repeating the same investigation
## CI Integration
In CI, this skill runs as a Claude Code agent with:
- `ANTHROPIC_API_KEY` for Claude
- `GEMINI_API_KEY` for initial issue analysis (optional)
- ComfyUI server pre-started in the container
- `playwright-cli` pre-installed
The CI workflow:
1. Gemini generates a reproduce guide (markdown) from the issue
2. Claude agent receives the guide and runs this skill
3. Claude explores interactively, installs dependencies, retries
4. Claude records a clean demo once reproduced
5. Video and report are uploaded as artifacts

View File

@@ -0,0 +1,361 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Perform comprehensive quality assurance of the ComfyUI frontend application by navigating all routes, clicking interactive elements, and testing features. Generate a structured report and submit it as a draft PR.
## Prerequisites
- Node.js 22+
- `pnpm` package manager
- `gh` CLI (authenticated)
- `playwright-cli` (browser automation): `npm install -g @playwright/cli@latest`
## Step 1: Environment Detection & Setup
Detect the runtime environment and ensure the app is accessible.
### CI Environment
If `CI=true` is set:
1. The ComfyUI backend is pre-configured in the CI container (`ghcr.io/comfy-org/comfyui-ci-container`)
2. Frontend dist is already built and served by the backend
3. Server runs at `http://127.0.0.1:8188`
4. Skip user prompts — run fully automated
### Local Environment
If `CI` is not set:
1. **Ask the user**: "Is a ComfyUI server already running? If so, what URL? (default: http://127.0.0.1:8188)"
- If yes: use the provided URL
- If no: offer to start one:
```bash
# Option A: Use existing ComfyUI installation
# Ask for the path to ComfyUI, then:
cd <comfyui_path>
python main.py --cpu --multi-user --front-end-root <frontend_dist_path> &
# Option B: Build frontend and use preview server (no backend features)
pnpm build && pnpm preview &
```
2. Wait for server readiness by polling the URL (retry with 2s intervals, 60s timeout)
### Browser Automation Setup
Use **playwright-cli** for browser interaction via Bash commands:
```bash
playwright-cli open http://127.0.0.1:8188 # open browser and navigate
playwright-cli snapshot # capture snapshot with element refs
playwright-cli click e1 # click by element ref from snapshot
playwright-cli press Tab # keyboard shortcuts
playwright-cli screenshot --filename=f.png # save screenshot
```
playwright-cli is headless by default (CI-friendly). Each command outputs the current page snapshot with element references (`e1`, `e2`, …) that you use for subsequent `click`, `fill`, `hover` commands. Always run `snapshot` before interacting to get fresh refs.
For local dev servers behind proxies, adjust the URL accordingly (e.g., `https://[port].stukivx.xyz` pattern if configured).
## Step 2: QA Test Plan
Navigate to the application URL and systematically test each area below. For each test, record:
- **Status**: pass / fail / skip (with reason)
- **Notes**: any issues, unexpected behavior, or visual glitches
- **Screenshots**: take screenshots of failures or notable states
### 2.1 Application Load & Routes
| Test | Steps |
| ----------------- | ------------------------------------------------------------ |
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| Default redirect | If multi-user mode, `/` redirects to `/user-select` first |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### 2.2 Canvas & Graph View
| Test | Steps |
| ------------------------- | -------------------------------------------------------------- |
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Fit view | Press `.` key — canvas fits to content |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Add node via search | Open search box, find and add a node |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Disconnect nodes | Right-click a link and remove, or drag from connected slot |
| Multi-select | Shift+click or drag-select multiple nodes |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Node context menu | Right-click a node — menu appears with all expected options |
| Canvas context menu | Right-click empty canvas — menu appears |
### 2.3 Node Operations
| Test | Steps |
| ------------------- | ---------------------------------------------------------- |
| Bypass node | Select node, Ctrl+B — node shows bypass state |
| Mute node | Select node, Ctrl+M — node shows muted state |
| Collapse node | Select node, Alt+C — node collapses |
| Pin node | Select node, press P — node becomes pinned |
| Rename node | Double-click node title — edit mode activates |
| Node color | Right-click > Color — color picker works |
| Group nodes | Select multiple nodes, Ctrl+G — group created |
| Ungroup | Right-click group > Ungroup |
| Widget interactions | Toggle checkboxes, adjust sliders, type in text fields |
| Combo widget | Click dropdown widgets — options appear and are selectable |
### 2.4 Sidebar Tabs
| Test | Steps |
| ---------------------- | ------------------------------------------------------ |
| Workflows tab | Press W — workflows sidebar opens with saved workflows |
| Node Library tab | Press N — node library opens with categories |
| Model Library tab | Press M — model library opens |
| Assets tab | Press A — assets browser opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
| Drag node from library | Drag a node from library onto canvas |
### 2.5 Topbar & Workflow Tabs
| Test | Steps |
| -------------------- | ------------------------------------------------------ |
| Workflow tab display | Current workflow name shown in tab bar |
| New workflow | Ctrl+N — new blank workflow created |
| Rename workflow | Double-click workflow tab |
| Tab context menu | Right-click workflow tab — menu with Close/Rename/etc. |
| Multiple tabs | Open multiple workflows, switch between them |
| Queue button | Click Queue/Run button — prompt queues |
| Batch count | Click batch count editor, change value |
| Menu hamburger | Click hamburger menu — options appear |
### 2.6 Settings Dialog
| Test | Steps |
| ---------------- | ---------------------------------------------------- |
| Open settings | Press Ctrl+, or click settings button |
| Settings tabs | Navigate through all setting categories |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Keybindings tab | Navigate to keybindings panel |
| About tab | Navigate to about panel — version info shown |
| Close settings | Press Escape or click close button |
### 2.7 Bottom Panel
| Test | Steps |
| ------------------- | -------------------------------------- |
| Toggle panel | Press Ctrl+` — bottom panel opens |
| Logs tab | Logs/terminal tab shows server output |
| Shortcuts tab | Shortcuts reference is displayed |
| Keybindings display | Press Ctrl+Shift+K — keybindings panel |
### 2.8 Execution & Queue
| Test | Steps |
| -------------- | ----------------------------------------------------- |
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
| Job history | Open job history sidebar — past executions listed |
| Clear history | Clear execution history via menu |
### 2.9 Workflow File Operations
| Test | Steps |
| --------------- | ------------------------------------------------- |
| Save workflow | Ctrl+S — workflow saves (check for prompt if new) |
| Open workflow | Ctrl+O — file picker or workflow browser opens |
| Export JSON | Menu > Export — workflow JSON downloads |
| Import workflow | Drag a .json workflow file onto canvas |
| Load default | Menu > Load Default — default workflow loads |
| Clear workflow | Menu > Clear — canvas clears (after confirmation) |
### 2.10 Advanced Features
| Test | Steps |
| --------------- | ------------------------------------------------- |
| Minimap | Alt+M — minimap toggle |
| Focus mode | Toggle focus mode |
| Canvas lock | Press H to lock, V to unlock |
| Link visibility | Ctrl+Shift+L — toggle links |
| Subgraph | Select nodes > Ctrl+Shift+E — convert to subgraph |
### 2.11 Error Handling
| Test | Steps |
| --------------------- | -------------------------------------------- |
| Missing nodes dialog | Load workflow with non-existent node types |
| Missing models dialog | Trigger missing model warning |
| Network error | Disconnect backend, verify graceful handling |
| Invalid workflow | Try loading malformed JSON |
### 2.12 Responsive & Accessibility
| Test | Steps |
| ------------------- | ------------------------------------- |
| Window resize | Resize browser window — layout adapts |
| Keyboard navigation | Tab through interactive elements |
| Sidebar resize | Drag sidebar edge to resize |
## Step 3: Generate Report
After completing all tests, generate a markdown report file.
### Report Location
```
docs/qa/YYYY-MM-DD-NNN-report.md
```
Where:
- `YYYY-MM-DD` is today's date
- `NNN` is a zero-padded increment index (001, 002, etc.)
To determine the increment, check existing files:
```bash
ls docs/qa/ | grep "$(date +%Y-%m-%d)" | wc -l
```
### Report Template
```markdown
# QA Report: ComfyUI Frontend
**Date**: YYYY-MM-DD
**Environment**: CI / Local (OS, Browser)
**Frontend Version**: (git sha or version)
**Agent**: Claude / Codex / Other
**Server URL**: http://...
## Summary
| Category | Pass | Fail | Skip | Total |
| --------------- | ---- | ---- | ---- | ----- |
| Routes & Load | | | | |
| Canvas | | | | |
| Node Operations | | | | |
| Sidebar | | | | |
| Topbar | | | | |
| Settings | | | | |
| Bottom Panel | | | | |
| Execution | | | | |
| File Operations | | | | |
| Advanced | | | | |
| Error Handling | | | | |
| Responsive | | | | |
| **Total** | | | | |
## Results
### Routes & Load
- [x] Root route loads — pass
- [ ] ...
### Canvas & Graph View
- [x] Canvas renders — pass
- [ ] ...
(repeat for each category)
## Issues Found
### Issue 1: [Title]
- **Severity**: critical / major / minor / cosmetic
- **Steps to reproduce**: ...
- **Expected**: ...
- **Actual**: ...
- **Screenshot**: (if available)
## Notes
Any additional observations, performance notes, or suggestions.
```
## Step 4: Commit and Push Report
### In CI (when `CI=true`)
Save the report directly to `$QA_ARTIFACTS` (the CI workflow uploads this as
an artifact and posts results as a PR comment). Do **not** commit, push, or
create a new PR.
### Local / interactive use
When running locally, create a draft PR after committing:
```bash
# Ensure on a feature branch
BRANCH_NAME="qa/$(date +%Y-%m-%d)-$(git rev-parse --short HEAD)"
git checkout -b "$BRANCH_NAME" 2>/dev/null || git checkout "$BRANCH_NAME"
git add docs/qa/
git commit -m "docs: add QA report $(date +%Y-%m-%d)
Automated QA report covering all frontend routes and features."
git push -u origin "$BRANCH_NAME"
# Create draft PR assigned to comfy-pr-bot
gh pr create \
--draft \
--title "QA Report: $(date +%Y-%m-%d)" \
--body "## QA Report
Automated frontend QA run covering all routes and interactive features.
See \`docs/qa/\` for the full report.
/cc @comfy-pr-bot" \
--assignee comfy-pr-bot
```
## Execution Notes
### Cross-Platform Considerations
- **Windows**: Use `pwsh` or `cmd` equivalents for shell commands. `gh` CLI works on all platforms.
- **macOS**: Keyboard shortcuts use Cmd instead of Ctrl in the actual app, but Playwright sends OS-appropriate keys.
- **Linux**: Primary CI platform. Screenshot baselines are Linux-only.
### Agent Compatibility
This skill uses **playwright-cli** (`@playwright/cli`) — a token-efficient CLI designed for coding agents. Install it once with `npm install -g @playwright/cli@latest`, then use `Bash` to run commands.
The key operations and their playwright-cli equivalents:
| Action | Command |
| ---------------- | ---------------------------------------- |
| Navigate to URL | `playwright-cli goto <url>` |
| Get element refs | `playwright-cli snapshot` |
| Click element | `playwright-cli click <ref>` |
| Type text | `playwright-cli fill <ref> <text>` |
| Press shortcut | `playwright-cli press <key>` |
| Take screenshot | `playwright-cli screenshot --filename=f` |
| Hover element | `playwright-cli hover <ref>` |
| Select dropdown | `playwright-cli select <ref> <value>` |
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation or major interactions to refresh refs before acting.
### Tips for Reliable QA
1. **Wait for page stability** before interacting — check that elements are visible and enabled
2. **Take a snapshot after each major navigation** to verify state
3. **Don't use fixed timeouts** — poll for expected conditions
4. **Record the full page snapshot** at the start for baseline comparison
5. **If a test fails**, document it and continue — don't abort the entire QA run
6. **Group related tests** — complete one category before moving to the next

View File

@@ -44,12 +44,17 @@ runs:
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Start ComfyUI server
if: ${{ inputs.launch_server == 'true' }}
shell: bash
working-directory: ComfyUI
env:
EXTRA_SERVER_PARAMS: ${{ inputs.extra_server_params }}
run: |
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
wait-for-it --service 127.0.0.1:8188 -t 600
python main.py --cpu --multi-user --front-end-root ../dist $EXTRA_SERVER_PARAMS &
for i in $(seq 1 300); do
curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1 && echo "Server ready" && exit 0
sleep 2
done
echo "::error::ComfyUI server did not start within 600s" && exit 1

1100
.github/workflows/pr-qa.yaml vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -30,6 +30,7 @@ concurrency:
jobs:
bump-version:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
permissions:
contents: write

View File

@@ -18,6 +18,7 @@ concurrency:
jobs:
docs-check:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
timeout-minutes: 45
steps:

5
.gitignore vendored
View File

@@ -99,4 +99,7 @@ vitest.config.*.timestamp*
# Weekly docs check output
/output.txt
.amp
.amp
.playwright-cli/
.playwright/
.claude/scheduled_tasks.lock

View File

@@ -9,6 +9,7 @@
"packages/registry-types/src/comfyRegistryTypes.ts",
"public/materialdesignicons.min.css",
"src/types/generatedManagerTypes.ts",
"**/__fixtures__/**/*.json"
"**/__fixtures__/**/*.json",
"scripts/qa-report-template.html"
]
}

View File

@@ -51,6 +51,9 @@
# Manager
/src/workbench/extensions/manager/ @viva-jinyi @christian-byrne @ltdrdata
# Model-to-node mappings (cloud team)
/src/platform/assets/mappings/ @deepme987
# LLM Instructions (blank on purpose)
.claude/
.cursor/

2
apps/website/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
dist/
.astro/

View File

@@ -0,0 +1,24 @@
import { defineConfig } from 'astro/config'
import vue from '@astrojs/vue'
import tailwindcss from '@tailwindcss/vite'
export default defineConfig({
site: 'https://comfy.org',
output: 'static',
integrations: [vue()],
vite: {
plugins: [tailwindcss()]
},
build: {
assetsPrefix: process.env.VERCEL_URL
? `https://${process.env.VERCEL_URL}`
: undefined
},
i18n: {
locales: ['en', 'zh-CN'],
defaultLocale: 'en',
routing: {
prefixDefaultLocale: false
}
}
})

80
apps/website/package.json Normal file
View File

@@ -0,0 +1,80 @@
{
"name": "@comfyorg/website",
"version": "0.0.1",
"private": true,
"type": "module",
"scripts": {
"dev": "astro dev",
"build": "astro build",
"preview": "astro preview"
},
"dependencies": {
"@comfyorg/design-system": "workspace:*",
"@vercel/analytics": "catalog:",
"vue": "catalog:"
},
"devDependencies": {
"@astrojs/vue": "catalog:",
"@tailwindcss/vite": "catalog:",
"astro": "catalog:",
"tailwindcss": "catalog:",
"typescript": "catalog:"
},
"nx": {
"tags": [
"scope:website",
"type:app"
],
"targets": {
"dev": {
"executor": "nx:run-commands",
"continuous": true,
"options": {
"cwd": "apps/website",
"command": "astro dev"
}
},
"serve": {
"executor": "nx:run-commands",
"continuous": true,
"options": {
"cwd": "apps/website",
"command": "astro dev"
}
},
"build": {
"executor": "nx:run-commands",
"cache": true,
"dependsOn": [
"^build"
],
"options": {
"cwd": "apps/website",
"command": "astro build"
},
"outputs": [
"{projectRoot}/dist"
]
},
"preview": {
"executor": "nx:run-commands",
"continuous": true,
"dependsOn": [
"build"
],
"options": {
"cwd": "apps/website",
"command": "astro preview"
}
},
"typecheck": {
"executor": "nx:run-commands",
"cache": true,
"options": {
"cwd": "apps/website",
"command": "astro check"
}
}
}
}
}

Binary file not shown.

Binary file not shown.

1
apps/website/src/env.d.ts vendored Normal file
View File

@@ -0,0 +1 @@
/// <reference types="astro/client" />

View File

@@ -0,0 +1,2 @@
@import 'tailwindcss';
@import '@comfyorg/design-system/css/base.css';

View File

@@ -0,0 +1,9 @@
{
"extends": "astro/tsconfigs/strict",
"compilerOptions": {
"paths": {
"@/*": ["./src/*"]
}
},
"include": ["src/**/*", "astro.config.mjs"]
}

View File

@@ -0,0 +1,407 @@
{
"id": "0cc04f4c-d744-462d-8638-4e5f5e3947e7",
"revision": 0,
"last_node_id": 19,
"last_link_id": 24,
"nodes": [
{
"id": 14,
"type": "CLIPLoader",
"pos": [143.16716182216328, 290.16372862874033],
"size": [270, 117.3125],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "CLIP",
"type": "CLIP",
"links": [21]
}
],
"properties": {
"Node name for S&R": "CLIPLoader"
},
"widgets_values": [null, "stable_diffusion", "default"]
},
{
"id": 18,
"type": "PreviewImage",
"pos": [1305.1455526601603, 472.17095792625025],
"size": [225, 48],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 24
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewImage"
},
"widgets_values": []
},
{
"id": 19,
"type": "314bbb9f-f1cc-456c-b14f-2ba92bd4a597",
"pos": [794.198171390827, 452.45433419677147],
"size": [225, 172],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"label": "renamed_clip",
"name": "clip",
"type": "CLIP",
"link": 21
},
{
"label": "renamed_seed",
"name": "seed",
"type": "INT",
"widget": {
"name": "seed"
},
"link": 22
},
{
"label": "renamed_vae",
"name": "vae",
"type": "VAE",
"link": 23
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [24]
}
],
"title": "Input Test Subgraph",
"properties": {
"proxyWidgets": [
["12", "seed"],
["15", "text"]
]
},
"widgets_values": []
},
{
"id": 13,
"type": "PrimitiveInt",
"pos": [155.04048166054417, 773.3816055422594],
"size": [270, 82],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [22]
}
],
"title": "Seed Int",
"properties": {
"Node name for S&R": "PrimitiveInt"
},
"widgets_values": [0, "randomize"]
},
{
"id": 17,
"type": "VAELoader",
"pos": [163.6043676075426, 543.9624492717659],
"size": [270, 82.65625],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [23]
}
],
"properties": {
"Node name for S&R": "VAELoader"
},
"widgets_values": ["pixel_space"]
}
],
"links": [
[21, 14, 0, 19, 0, "CLIP"],
[22, 13, 0, 19, 1, "INT"],
[23, 17, 0, 19, 2, "VAE"],
[24, 19, 0, 18, 0, "IMAGE"]
],
"groups": [],
"definitions": {
"subgraphs": [
{
"id": "314bbb9f-f1cc-456c-b14f-2ba92bd4a597",
"version": 1,
"state": {
"lastGroupId": 0,
"lastNodeId": 19,
"lastLinkId": 24,
"lastRerouteId": 0
},
"revision": 0,
"config": {},
"name": "Input Test Subgraph",
"inputNode": {
"id": -10,
"bounding": [
358.8694807105848, 439.23932667242485, 123.14453125,
99.99999999999994
]
},
"outputNode": {
"id": -20,
"bounding": [1408.5510580294986, 463.2512895126797, 120, 60]
},
"inputs": [
{
"id": "cfaad2dc-7758-412c-a4ac-dc2e6d37b28c",
"name": "clip",
"type": "CLIP",
"linkIds": [16],
"localized_name": "clip",
"label": "renamed_clip",
"pos": [462.0140119605848, 459.23932667242485]
},
{
"id": "2e4600ea-e1b1-42ca-b43a-e066fd080774",
"name": "seed",
"type": "INT",
"linkIds": [15],
"localized_name": "seed",
"label": "renamed_seed",
"pos": [462.0140119605848, 479.23932667242485]
},
{
"id": "86ed2da7-db02-454a-9362-70a3fa3e91bf",
"name": "vae",
"type": "VAE",
"linkIds": [19],
"localized_name": "vae",
"label": "renamed_vae",
"pos": [462.0140119605848, 499.23932667242485]
}
],
"outputs": [
{
"id": "8670d1a7-0d44-4688-b7dd-d4b423f1aee0",
"name": "IMAGE",
"type": "IMAGE",
"linkIds": [20],
"localized_name": "IMAGE",
"pos": [1428.5510580294986, 483.2512895126797]
}
],
"widgets": [],
"nodes": [
{
"id": 12,
"type": "KSampler",
"pos": [769.2424728654022, 512.726159169824],
"size": [270, 262],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": null
},
{
"localized_name": "positive",
"name": "positive",
"type": "CONDITIONING",
"link": 17
},
{
"localized_name": "negative",
"name": "negative",
"type": "CONDITIONING",
"link": null
},
{
"localized_name": "latent_image",
"name": "latent_image",
"type": "LATENT",
"link": null
},
{
"localized_name": "seed",
"name": "seed",
"type": "INT",
"widget": {
"name": "seed"
},
"link": 15
}
],
"outputs": [
{
"localized_name": "LATENT",
"name": "LATENT",
"type": "LATENT",
"links": [18]
}
],
"properties": {
"Node name for S&R": "KSampler"
},
"widgets_values": [0, "randomize", 20, 8, "euler", "simple", 1]
},
{
"id": 16,
"type": "VAEDecode",
"pos": [1208.5510580294986, 469.21581253470083],
"size": [140, 46],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"localized_name": "samples",
"name": "samples",
"type": "LATENT",
"link": 18
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 19
}
],
"outputs": [
{
"localized_name": "IMAGE",
"name": "IMAGE",
"type": "IMAGE",
"links": [20]
}
],
"properties": {
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 15,
"type": "CLIPTextEncode",
"pos": [681.4596332342014, 243.17567172890932],
"size": [400, 200],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"localized_name": "clip",
"name": "clip",
"type": "CLIP",
"link": 16
},
{
"label": "renamed_from_sidepanel",
"localized_name": "text",
"name": "text",
"type": "STRING",
"widget": {
"name": "text"
},
"link": null
}
],
"outputs": [
{
"localized_name": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [17]
}
],
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [""]
}
],
"groups": [],
"links": [
{
"id": 17,
"origin_id": 15,
"origin_slot": 0,
"target_id": 12,
"target_slot": 1,
"type": "CONDITIONING"
},
{
"id": 18,
"origin_id": 12,
"origin_slot": 0,
"target_id": 16,
"target_slot": 0,
"type": "LATENT"
},
{
"id": 16,
"origin_id": -10,
"origin_slot": 0,
"target_id": 15,
"target_slot": 0,
"type": "CLIP"
},
{
"id": 15,
"origin_id": -10,
"origin_slot": 1,
"target_id": 12,
"target_slot": 4,
"type": "INT"
},
{
"id": 19,
"origin_id": -10,
"origin_slot": 2,
"target_id": 16,
"target_slot": 1,
"type": "VAE"
},
{
"id": 20,
"origin_id": 16,
"origin_slot": 0,
"target_id": -20,
"target_slot": 0,
"type": "IMAGE"
}
],
"extra": {}
}
]
},
"config": {},
"extra": {
"ds": {
"scale": 0.6727925600199565,
"offset": [446.69747171876463, 99.95078257277316]
}
},
"version": 0.4
}

View File

@@ -91,6 +91,12 @@ export class CanvasHelper {
await this.page.mouse.move(10, 10)
}
async isReadOnly(): Promise<boolean> {
return this.page.evaluate(() => {
return window.app!.canvas.state.readOnly
})
}
async getScale(): Promise<number> {
return this.page.evaluate(() => {
return window.app!.canvas.ds.scale

View File

@@ -28,10 +28,15 @@ export const TestIds = {
settingsTabAbout: 'settings-tab-about',
confirm: 'confirm-dialog',
errorOverlay: 'error-overlay',
errorOverlaySeeErrors: 'error-overlay-see-errors',
runtimeErrorPanel: 'runtime-error-panel',
missingNodeCard: 'missing-node-card',
errorCardFindOnGithub: 'error-card-find-on-github',
errorCardCopy: 'error-card-copy',
about: 'about-panel',
whatsNewSection: 'whats-new-section'
whatsNewSection: 'whats-new-section',
missingNodePacksGroup: 'error-group-missing-node',
missingModelsGroup: 'error-group-missing-model'
},
keybindings: {
presetMenu: 'keybinding-preset-menu'
@@ -76,6 +81,10 @@ export const TestIds = {
},
user: {
currentUserIndicator: 'current-user-indicator'
},
errors: {
imageLoadError: 'error-loading-image',
videoLoadError: 'error-loading-video'
}
} as const
@@ -101,3 +110,4 @@ export type TestIdValue =
(id: string) => string
>
| (typeof TestIds.user)[keyof typeof TestIds.user]
| (typeof TestIds.errors)[keyof typeof TestIds.errors]

View File

@@ -1,3 +1,5 @@
import type { Page } from '@playwright/test'
import type { LGraph, Subgraph } from '../../src/lib/litegraph/src/litegraph'
import { isSubgraph } from '../../src/utils/typeGuardUtil'
@@ -14,3 +16,30 @@ export function assertSubgraph(
)
}
}
/**
* Returns the widget-input slot Y position and the node title height
* for the promoted "text" input on the SubgraphNode.
*
* The slot Y should be at the widget row, not the header. A value near
* zero or negative indicates the slot is positioned at the header (the bug).
*/
export function getTextSlotPosition(page: Page, nodeId: string) {
return page.evaluate((id) => {
const node = window.app!.canvas.graph!.getNodeById(id)
if (!node) return null
const titleHeight = window.LiteGraph!.NODE_TITLE_HEIGHT
for (const input of node.inputs) {
if (!input.widget || input.type !== 'STRING') continue
return {
hasPos: !!input.pos,
posY: input.pos?.[1] ?? null,
widgetName: input.widget.name,
titleHeight
}
}
return null
}, nodeId)
}

View File

@@ -38,16 +38,13 @@ const customColorPalettes = {
CLEAR_BACKGROUND_COLOR: '#222222',
NODE_TITLE_COLOR: 'rgba(255,255,255,.75)',
NODE_SELECTED_TITLE_COLOR: '#FFF',
NODE_TEXT_SIZE: 14,
NODE_TEXT_COLOR: '#b8b8b8',
NODE_SUBTEXT_SIZE: 12,
NODE_DEFAULT_COLOR: 'rgba(0,0,0,.8)',
NODE_DEFAULT_BGCOLOR: 'rgba(22,22,22,.8)',
NODE_DEFAULT_BOXCOLOR: 'rgba(255,255,255,.75)',
NODE_DEFAULT_SHAPE: 'box',
NODE_BOX_OUTLINE_COLOR: '#236692',
DEFAULT_SHADOW_COLOR: 'rgba(0,0,0,0)',
DEFAULT_GROUP_FONT: 24,
WIDGET_BGCOLOR: '#242424',
WIDGET_OUTLINE_COLOR: '#333',
WIDGET_TEXT_COLOR: '#a3a3a8',
@@ -102,16 +99,13 @@ const customColorPalettes = {
CLEAR_BACKGROUND_COLOR: '#000',
NODE_TITLE_COLOR: 'rgba(255,255,255,.75)',
NODE_SELECTED_TITLE_COLOR: '#FFF',
NODE_TEXT_SIZE: 14,
NODE_TEXT_COLOR: '#b8b8b8',
NODE_SUBTEXT_SIZE: 12,
NODE_DEFAULT_COLOR: 'rgba(0,0,0,.8)',
NODE_DEFAULT_BGCOLOR: 'rgba(22,22,22,.8)',
NODE_DEFAULT_BOXCOLOR: 'rgba(255,255,255,.75)',
NODE_DEFAULT_SHAPE: 'box',
NODE_BOX_OUTLINE_COLOR: '#236692',
DEFAULT_SHADOW_COLOR: 'rgba(0,0,0,0)',
DEFAULT_GROUP_FONT: 24,
WIDGET_BGCOLOR: '#242424',
WIDGET_OUTLINE_COLOR: '#333',
WIDGET_TEXT_COLOR: '#a3a3a8',

View File

@@ -0,0 +1,318 @@
import { expect } from '@playwright/test'
import type { ComfyPage } from '../fixtures/ComfyPage'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
async function pressKeyAndExpectRequest(
comfyPage: ComfyPage,
key: string,
urlPattern: string,
method: string = 'POST'
) {
const requestPromise = comfyPage.page.waitForRequest(
(req) => req.url().includes(urlPattern) && req.method() === method,
{ timeout: 5000 }
)
await comfyPage.page.keyboard.press(key)
return requestPromise
}
test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
test.describe('Sidebar Toggle Shortcuts', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.canvas.click({ position: { x: 400, y: 400 } })
await comfyPage.nextFrame()
})
const sidebarTabs = [
{ key: 'KeyW', tabId: 'workflows', label: 'workflows' },
{ key: 'KeyN', tabId: 'node-library', label: 'node library' },
{ key: 'KeyM', tabId: 'model-library', label: 'model library' },
{ key: 'KeyA', tabId: 'assets', label: 'assets' }
] as const
for (const { key, tabId, label } of sidebarTabs) {
test(`'${key}' toggles ${label} sidebar`, async ({ comfyPage }) => {
const selectedButton = comfyPage.page.locator(
`.${tabId}-tab-button.side-bar-button-selected`
)
await expect(selectedButton).not.toBeVisible()
await comfyPage.canvas.press(key)
await comfyPage.nextFrame()
await expect(selectedButton).toBeVisible()
await comfyPage.canvas.press(key)
await comfyPage.nextFrame()
await expect(selectedButton).not.toBeVisible()
})
}
})
test.describe('Canvas View Controls', () => {
test("'Alt+=' zooms in", async ({ comfyPage }) => {
const initialScale = await comfyPage.canvasOps.getScale()
await comfyPage.canvas.press('Alt+Equal')
await comfyPage.nextFrame()
const newScale = await comfyPage.canvasOps.getScale()
expect(newScale).toBeGreaterThan(initialScale)
})
test("'Alt+-' zooms out", async ({ comfyPage }) => {
const initialScale = await comfyPage.canvasOps.getScale()
await comfyPage.canvas.press('Alt+Minus')
await comfyPage.nextFrame()
const newScale = await comfyPage.canvasOps.getScale()
expect(newScale).toBeLessThan(initialScale)
})
test("'.' fits view to nodes", async ({ comfyPage }) => {
// Set scale very small so fit-view will zoom back to fit nodes
await comfyPage.canvasOps.setScale(0.1)
const scaleBefore = await comfyPage.canvasOps.getScale()
expect(scaleBefore).toBeCloseTo(0.1, 1)
// Click canvas to ensure focus is within graph-canvas-container
await comfyPage.canvas.click({ position: { x: 400, y: 400 } })
await comfyPage.nextFrame()
await comfyPage.canvas.press('Period')
await comfyPage.nextFrame()
const scaleAfter = await comfyPage.canvasOps.getScale()
expect(scaleAfter).toBeGreaterThan(scaleBefore)
})
test("'h' locks canvas", async ({ comfyPage }) => {
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
await comfyPage.canvas.press('KeyH')
await comfyPage.nextFrame()
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
})
test("'v' unlocks canvas", async ({ comfyPage }) => {
// Lock first
await comfyPage.command.executeCommand('Comfy.Canvas.Lock')
await comfyPage.nextFrame()
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
await comfyPage.canvas.press('KeyV')
await comfyPage.nextFrame()
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
})
})
test.describe('Node State Toggles', () => {
test("'Alt+c' collapses and expands selected nodes", async ({
comfyPage
}) => {
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
expect(nodes.length).toBeGreaterThan(0)
const node = nodes[0]
await node.click('title')
await comfyPage.nextFrame()
expect(await node.isCollapsed()).toBe(false)
await comfyPage.canvas.press('Alt+KeyC')
await comfyPage.nextFrame()
expect(await node.isCollapsed()).toBe(true)
await comfyPage.canvas.press('Alt+KeyC')
await comfyPage.nextFrame()
expect(await node.isCollapsed()).toBe(false)
})
test("'Ctrl+m' mutes and unmutes selected nodes", async ({ comfyPage }) => {
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
expect(nodes.length).toBeGreaterThan(0)
const node = nodes[0]
await node.click('title')
await comfyPage.nextFrame()
// Normal mode is ALWAYS (0)
const getMode = () =>
comfyPage.page.evaluate((nodeId) => {
return window.app!.canvas.graph!.getNodeById(nodeId)!.mode
}, node.id)
expect(await getMode()).toBe(0)
await comfyPage.canvas.press('Control+KeyM')
await comfyPage.nextFrame()
// NEVER (2) = muted
expect(await getMode()).toBe(2)
await comfyPage.canvas.press('Control+KeyM')
await comfyPage.nextFrame()
expect(await getMode()).toBe(0)
})
})
test.describe('Mode and Panel Toggles', () => {
test("'Alt+m' toggles app mode", async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
// Set up linearData so app mode has something to show
await comfyPage.appMode.enterAppModeWithInputs([])
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
// Toggle off with Alt+m
await comfyPage.page.keyboard.press('Alt+KeyM')
await comfyPage.nextFrame()
await expect(comfyPage.appMode.linearWidgets).not.toBeVisible()
// Toggle on again
await comfyPage.page.keyboard.press('Alt+KeyM')
await comfyPage.nextFrame()
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
})
test("'Alt+Shift+m' toggles minimap", async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.settings.setSetting('Comfy.Minimap.Visible', true)
await comfyPage.settings.setSetting('Comfy.Graph.CanvasMenu', true)
await comfyPage.workflow.loadWorkflow('default')
const minimap = comfyPage.page.locator('.litegraph-minimap')
await expect(minimap).toBeVisible()
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
await comfyPage.nextFrame()
await expect(minimap).not.toBeVisible()
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
await comfyPage.nextFrame()
await expect(minimap).toBeVisible()
})
test("'Ctrl+`' toggles terminal/logs panel", async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
await comfyPage.page.keyboard.press('Control+Backquote')
await comfyPage.nextFrame()
await expect(comfyPage.bottomPanel.root).toBeVisible()
await comfyPage.page.keyboard.press('Control+Backquote')
await comfyPage.nextFrame()
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
})
})
test.describe('Queue and Execution', () => {
test("'Ctrl+Enter' queues prompt", async ({ comfyPage }) => {
const request = await pressKeyAndExpectRequest(
comfyPage,
'Control+Enter',
'/prompt',
'POST'
)
expect(request.url()).toContain('/prompt')
})
test("'Ctrl+Shift+Enter' queues prompt to front", async ({ comfyPage }) => {
const request = await pressKeyAndExpectRequest(
comfyPage,
'Control+Shift+Enter',
'/prompt',
'POST'
)
const body = request.postDataJSON()
expect(body.front).toBe(true)
})
test("'Ctrl+Alt+Enter' interrupts execution", async ({ comfyPage }) => {
const request = await pressKeyAndExpectRequest(
comfyPage,
'Control+Alt+Enter',
'/interrupt',
'POST'
)
expect(request.url()).toContain('/interrupt')
})
})
test.describe('File Operations', () => {
test("'Ctrl+s' triggers save workflow", async ({ comfyPage }) => {
// On a new unsaved workflow, Ctrl+s triggers Save As dialog.
// The dialog appearing proves the keybinding was intercepted by the app.
await comfyPage.page.keyboard.press('Control+s')
await comfyPage.nextFrame()
// The Save As dialog should appear (p-dialog overlay)
const dialogOverlay = comfyPage.page.locator('.p-dialog-mask')
await expect(dialogOverlay).toBeVisible({ timeout: 3000 })
// Dismiss the dialog
await comfyPage.page.keyboard.press('Escape')
await comfyPage.nextFrame()
})
test("'Ctrl+o' triggers open workflow", async ({ comfyPage }) => {
// Ctrl+o calls app.ui.loadFile() which clicks a hidden file input.
// Detect the file input click via an event listener.
await comfyPage.page.evaluate(() => {
window.TestCommand = false
const fileInputs =
document.querySelectorAll<HTMLInputElement>('input[type="file"]')
for (const input of fileInputs) {
input.addEventListener('click', () => {
window.TestCommand = true
})
}
})
await comfyPage.page.keyboard.press('Control+o')
await comfyPage.nextFrame()
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(true)
})
})
test.describe('Graph Operations', () => {
test("'Ctrl+Shift+e' converts selection to subgraph", async ({
comfyPage
}) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
expect(initialCount).toBeGreaterThan(1)
// Select all nodes
await comfyPage.canvas.press('Control+a')
await comfyPage.nextFrame()
await comfyPage.page.keyboard.press('Control+Shift+KeyE')
await comfyPage.nextFrame()
// After conversion, node count should decrease
// (multiple nodes replaced by single subgraph node)
await expect
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
timeout: 5000
})
.toBeLessThan(initialCount)
})
test("'r' refreshes node definitions", async ({ comfyPage }) => {
const request = await pressKeyAndExpectRequest(
comfyPage,
'KeyR',
'/object_info',
'GET'
)
expect(request.url()).toContain('/object_info')
})
})
})

View File

@@ -28,7 +28,7 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
)
await expect(errorOverlay).toBeVisible()
const missingNodesTitle = comfyPage.page.getByText(/Missing Node Packs/)
const missingNodesTitle = errorOverlay.getByText(/Missing Node Packs/)
await expect(missingNodesTitle).toBeVisible()
})
@@ -42,11 +42,13 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
)
await expect(errorOverlay).toBeVisible()
const missingNodesTitle = comfyPage.page.getByText(/Missing Node Packs/)
const missingNodesTitle = errorOverlay.getByText(/Missing Node Packs/)
await expect(missingNodesTitle).toBeVisible()
// Click "See Errors" to open the errors tab and verify subgraph node content
await errorOverlay.getByRole('button', { name: 'See Errors' }).click()
await errorOverlay
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
.click()
await expect(errorOverlay).not.toBeVisible()
const missingNodeCard = comfyPage.page.getByTestId(
@@ -75,7 +77,9 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
await expect(errorOverlay).toBeVisible()
// Click "See Errors" to open the right side panel errors tab
await errorOverlay.getByRole('button', { name: 'See Errors' }).click()
await errorOverlay
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
.click()
await expect(errorOverlay).not.toBeVisible()
// Verify MissingNodeCard is rendered in the errors tab
@@ -165,17 +169,19 @@ test.describe('Error actions in Errors Tab', { tag: '@ui' }, () => {
TestIds.dialogs.errorOverlay
)
await expect(errorOverlay).toBeVisible()
await errorOverlay.getByRole('button', { name: 'See Errors' }).click()
await errorOverlay
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
.click()
await expect(errorOverlay).not.toBeVisible()
// Verify Find on GitHub button is present in the error card
const findOnGithubButton = comfyPage.page.getByRole('button', {
name: 'Find on GitHub'
})
const findOnGithubButton = comfyPage.page.getByTestId(
TestIds.dialogs.errorCardFindOnGithub
)
await expect(findOnGithubButton).toBeVisible()
// Verify Copy button is present in the error card
const copyButton = comfyPage.page.getByRole('button', { name: 'Copy' })
const copyButton = comfyPage.page.getByTestId(TestIds.dialogs.errorCardCopy)
await expect(copyButton).toBeVisible()
})
})
@@ -204,7 +210,7 @@ test.describe('Missing models in Error Tab', () => {
)
await expect(errorOverlay).toBeVisible()
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
const missingModelsTitle = errorOverlay.getByText(/Missing Models/)
await expect(missingModelsTitle).toBeVisible()
})
@@ -220,7 +226,7 @@ test.describe('Missing models in Error Tab', () => {
)
await expect(errorOverlay).toBeVisible()
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
const missingModelsTitle = errorOverlay.getByText(/Missing Models/)
await expect(missingModelsTitle).toBeVisible()
})
@@ -231,13 +237,10 @@ test.describe('Missing models in Error Tab', () => {
'missing/model_metadata_widget_mismatch'
)
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
await expect(missingModelsTitle).not.toBeVisible()
const errorOverlay = comfyPage.page.getByTestId(
TestIds.dialogs.errorOverlay
)
await expect(errorOverlay).not.toBeVisible()
await expect(
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
).not.toBeVisible()
await expect(comfyPage.page.getByText(/Missing Models/)).not.toBeVisible()
})
// Flaky test after parallelization

View File

@@ -764,13 +764,13 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
)
})
const generateUniqueFilename = (extension = '') =>
`${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}${extension}`
test.describe('Restore all open workflows on reload', () => {
let workflowA: string
let workflowB: string
const generateUniqueFilename = (extension = '') =>
`${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}${extension}`
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
@@ -829,6 +829,82 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
})
})
test.describe('Restore workflow tabs after browser restart', () => {
let workflowA: string
let workflowB: string
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
workflowA = generateUniqueFilename()
await comfyPage.menu.topbar.saveWorkflow(workflowA)
workflowB = generateUniqueFilename()
await comfyPage.menu.topbar.triggerTopbarCommand(['New'])
await comfyPage.menu.topbar.saveWorkflow(workflowB)
// Wait for localStorage fallback pointers to be written
await comfyPage.page.waitForFunction(() => {
for (let i = 0; i < window.localStorage.length; i++) {
const key = window.localStorage.key(i)
if (key?.startsWith('Comfy.Workflow.LastOpenPaths:')) {
return true
}
}
return false
})
// Simulate browser restart: clear sessionStorage (lost on close)
// but keep localStorage (survives browser restart)
await comfyPage.page.evaluate(() => {
sessionStorage.clear()
})
await comfyPage.setup({ clearStorage: false })
})
test('Restores topbar workflow tabs after browser restart', async ({
comfyPage
}) => {
await comfyPage.settings.setSetting(
'Comfy.Workflow.WorkflowTabsPosition',
'Topbar'
)
// Wait for both restored tabs to render (localStorage fallback is async)
await expect(
comfyPage.page.locator('.workflow-tabs .workflow-label', {
hasText: workflowA
})
).toBeVisible()
const tabs = await comfyPage.menu.topbar.getTabNames()
const activeWorkflowName = await comfyPage.menu.topbar.getActiveTabName()
expect(tabs).toEqual(expect.arrayContaining([workflowA, workflowB]))
expect(tabs.indexOf(workflowA)).toBeLessThan(tabs.indexOf(workflowB))
expect(activeWorkflowName).toEqual(workflowB)
})
test('Restores sidebar workflows after browser restart', async ({
comfyPage
}) => {
await comfyPage.settings.setSetting(
'Comfy.Workflow.WorkflowTabsPosition',
'Sidebar'
)
await comfyPage.menu.workflowsTab.open()
const openWorkflows =
await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
const activeWorkflowName =
await comfyPage.menu.workflowsTab.getActiveWorkflowName()
expect(openWorkflows).toEqual(
expect.arrayContaining([workflowA, workflowB])
)
expect(openWorkflows.indexOf(workflowA)).toBeLessThan(
openWorkflows.indexOf(workflowB)
)
expect(activeWorkflowName).toEqual(workflowB)
})
})
test('Auto fit view after loading workflow', async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
'Comfy.EnableWorkflowViewRestore',

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

After

Width:  |  Height:  |  Size: 93 KiB

View File

@@ -0,0 +1,86 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
import { getTextSlotPosition } from '../helpers/subgraphTestUtils'
test.describe(
'Subgraph promoted widget-input slot position',
{ tag: '@subgraph' },
() => {
test('Promoted text widget slot is positioned at widget row, not header', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow(
'subgraphs/subgraph-with-promoted-text-widget'
)
// Render a few frames so arrange() runs
await comfyPage.nextFrame()
await comfyPage.nextFrame()
const result = await getTextSlotPosition(comfyPage.page, '11')
expect(result).not.toBeNull()
expect(result!.hasPos).toBe(true)
// The slot Y position should be well below the title area.
// If it's near 0 or negative, the slot is stuck at the header (the bug).
expect(result!.posY).toBeGreaterThan(result!.titleHeight)
})
test('Slot position remains correct after renaming subgraph input label', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow(
'subgraphs/subgraph-with-promoted-text-widget'
)
await comfyPage.nextFrame()
await comfyPage.nextFrame()
// Verify initial position is correct
const before = await getTextSlotPosition(comfyPage.page, '11')
expect(before).not.toBeNull()
expect(before!.hasPos).toBe(true)
expect(before!.posY).toBeGreaterThan(before!.titleHeight)
// Navigate into subgraph and rename the text input
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('11')
await subgraphNode.navigateIntoSubgraph()
const initialLabel = await comfyPage.page.evaluate(() => {
const graph = window.app!.canvas.graph
if (!graph || !('inputNode' in graph)) return null
const textInput = graph.inputs?.find(
(i: { type: string }) => i.type === 'STRING'
)
return textInput?.label || textInput?.name || null
})
if (!initialLabel)
throw new Error('Could not find STRING input in subgraph')
await comfyPage.subgraph.rightClickInputSlot(initialLabel)
await comfyPage.contextMenu.clickLitegraphMenuItem('Rename Slot')
await comfyPage.nextFrame()
const dialog = '.graphdialog input'
await comfyPage.page.waitForSelector(dialog, { state: 'visible' })
await comfyPage.page.fill(dialog, '')
await comfyPage.page.fill(dialog, 'my_custom_prompt')
await comfyPage.page.keyboard.press('Enter')
await comfyPage.page.waitForSelector(dialog, { state: 'hidden' })
// Navigate back to parent graph
await comfyPage.subgraph.exitViaBreadcrumb()
// Verify slot position is still at the widget row after rename
const after = await getTextSlotPosition(comfyPage.page, '11')
expect(after).not.toBeNull()
expect(after!.hasPos).toBe(true)
expect(after!.posY).toBeGreaterThan(after!.titleHeight)
// widget.name is the stable identity key — it does NOT change on rename.
// The display label is on input.label, read via PromotedWidgetView.label.
expect(after!.widgetName).not.toBe('my_custom_prompt')
})
}
)

View File

@@ -0,0 +1,57 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
import { getPromotedWidgetNames } from '../helpers/promotedWidgets'
test.describe(
'Subgraph promoted widget DOM position',
{ tag: '@subgraph' },
() => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Disabled')
})
test('Promoted seed widget renders in node body, not header', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow('default')
// Convert KSampler (id 3) to subgraph — seed is auto-promoted.
const ksampler = await comfyPage.nodeOps.getNodeRefById('3')
await ksampler.click('title')
const subgraphNode = await ksampler.convertToSubgraph()
await comfyPage.nextFrame()
// Enable Vue nodes now that the subgraph has been created
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
const subgraphNodeId = String(subgraphNode.id)
const promotedNames = await getPromotedWidgetNames(
comfyPage,
subgraphNodeId
)
expect(promotedNames).toContain('seed')
// Wait for Vue nodes to render
await comfyPage.vueNodes.waitForNodes()
const nodeLocator = comfyPage.vueNodes.getNodeLocator(subgraphNodeId)
await expect(nodeLocator).toBeVisible()
// The seed widget should be visible inside the node body
const seedWidget = nodeLocator.getByLabel('seed', { exact: true }).first()
await expect(seedWidget).toBeVisible()
// Verify widget is inside the node body, not the header
const headerBox = await nodeLocator
.locator('[data-testid^="node-header-"]')
.boundingBox()
const widgetBox = await seedWidget.boundingBox()
expect(headerBox).not.toBeNull()
expect(widgetBox).not.toBeNull()
// Widget top should be below the header bottom
expect(widgetBox!.y).toBeGreaterThan(headerBox!.y + headerBox!.height)
})
}
)

View File

@@ -0,0 +1,117 @@
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
const WORKFLOW = 'subgraphs/test-values-input-subgraph'
const RENAMED_LABEL = 'my_seed'
/**
* Regression test for subgraph input slot rename propagation.
*
* Renaming a SubgraphInput slot (e.g. "seed") inside the subgraph must
* update the promoted widget label shown on the parent SubgraphNode and
* keep the widget positioned in the node body (not the header).
*
* See: https://github.com/Comfy-Org/ComfyUI_frontend/pull/10195
*/
test.describe(
'Subgraph input slot rename propagation',
{ tag: ['@subgraph', '@widget'] },
() => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
})
test('Renaming a subgraph input slot updates the widget label on the parent node', async ({
comfyPage
}) => {
const { page } = comfyPage
// 1. Load workflow with subgraph containing a promoted seed widget input
await comfyPage.workflow.loadWorkflow(WORKFLOW)
await comfyPage.vueNodes.waitForNodes()
const sgNode = comfyPage.vueNodes.getNodeLocator('19')
await expect(sgNode).toBeVisible()
// 2. Verify the seed widget is visible on the parent node
const seedWidget = sgNode.getByLabel('seed', { exact: true })
await expect(seedWidget).toBeVisible()
// Verify widget is in the node body, not the header
const headerBox = await sgNode
.locator('[data-testid^="node-header-"]')
.boundingBox()
const widgetBox = await seedWidget.boundingBox()
expect(headerBox).not.toBeNull()
expect(widgetBox).not.toBeNull()
expect(widgetBox!.y).toBeGreaterThan(headerBox!.y + headerBox!.height)
// 3. Enter the subgraph and rename the seed slot.
// The subgraph IO rename uses canvas.prompt() which requires the
// litegraph context menu, so temporarily disable Vue nodes.
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
await comfyPage.nextFrame()
const sgNodeRef = await comfyPage.nodeOps.getNodeRefById('19')
await sgNodeRef.navigateIntoSubgraph()
// Find the seed SubgraphInput slot
const seedSlotName = await page.evaluate(() => {
const graph = window.app!.canvas.graph
if (!graph) return null
const inputs = (
graph as { inputs?: Array<{ name: string; type: string }> }
).inputs
return inputs?.find((i) => i.name.includes('seed'))?.name ?? null
})
expect(seedSlotName).not.toBeNull()
// 4. Right-click the seed input slot and rename it
await comfyPage.subgraph.rightClickInputSlot(seedSlotName!)
await comfyPage.contextMenu.clickLitegraphMenuItem('Rename Slot')
await comfyPage.nextFrame()
const dialog = '.graphdialog input'
await page.waitForSelector(dialog, { state: 'visible' })
await page.fill(dialog, '')
await page.fill(dialog, RENAMED_LABEL)
await page.keyboard.press('Enter')
await page.waitForSelector(dialog, { state: 'hidden' })
// 5. Navigate back to parent graph and re-enable Vue nodes
await comfyPage.subgraph.exitViaBreadcrumb()
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.vueNodes.waitForNodes()
// 6. Verify the widget label updated to the renamed value
const sgNodeAfter = comfyPage.vueNodes.getNodeLocator('19')
await expect(sgNodeAfter).toBeVisible()
const updatedLabel = await page.evaluate(() => {
const node = window.app!.canvas.graph!.getNodeById('19')
if (!node) return null
const w = node.widgets?.find((w: { name: string }) =>
w.name.includes('seed')
)
return w?.label || w?.name || null
})
expect(updatedLabel).toBe(RENAMED_LABEL)
// 7. Verify the widget is still in the body, not the header
const seedWidgetAfter = sgNodeAfter.getByLabel('seed', { exact: true })
await expect(seedWidgetAfter).toBeVisible()
const headerAfter = await sgNodeAfter
.locator('[data-testid^="node-header-"]')
.boundingBox()
const widgetAfter = await seedWidgetAfter.boundingBox()
expect(headerAfter).not.toBeNull()
expect(widgetAfter).not.toBeNull()
expect(widgetAfter!.y).toBeGreaterThan(
headerAfter!.y + headerAfter!.height
)
})
}
)

View File

@@ -0,0 +1,132 @@
import { readFileSync } from 'fs'
import { resolve } from 'path'
import { expect } from '@playwright/test'
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
interface SlotMeasurement {
key: string
offsetX: number
offsetY: number
}
interface NodeSlotData {
nodeId: string
isSubgraph: boolean
nodeW: number
nodeH: number
slots: SlotMeasurement[]
}
/**
* Regression test for link misalignment on SubgraphNodes when loading
* workflows with workflowRendererVersion: "LG".
*
* Root cause: ensureCorrectLayoutScale scales nodes by 1.2x for LG workflows,
* and fitView() updates lgCanvas.ds immediately. The Vue TransformPane's CSS
* transform lags by a frame, causing clientPosToCanvasPos to produce wrong
* slot offsets. The fix uses DOM-relative measurement instead.
*/
test.describe(
'Subgraph slot alignment after LG layout scale',
{ tag: ['@subgraph', '@canvas'] },
() => {
test('slot positions stay within node bounds after loading LG workflow', async ({
comfyPage
}) => {
const SLOT_BOUNDS_MARGIN = 20
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
const workflowPath = resolve(
import.meta.dirname,
'../assets/subgraphs/basic-subgraph.json'
)
const workflow = JSON.parse(
readFileSync(workflowPath, 'utf-8')
) as ComfyWorkflowJSON
workflow.extra = {
...workflow.extra,
workflowRendererVersion: 'LG'
}
await comfyPage.page.evaluate(
(wf) =>
window.app!.loadGraphData(wf as ComfyWorkflowJSON, true, true, null, {
openSource: 'template'
}),
workflow
)
await comfyPage.nextFrame()
// Wait for slot elements to appear in DOM
await comfyPage.page.locator('[data-slot-key]').first().waitFor()
const result: NodeSlotData[] = await comfyPage.page.evaluate(() => {
const nodes = window.app!.graph._nodes
const slotData: NodeSlotData[] = []
for (const node of nodes) {
const nodeId = String(node.id)
const nodeEl = document.querySelector(
`[data-node-id="${nodeId}"]`
) as HTMLElement | null
if (!nodeEl) continue
const slotEls = nodeEl.querySelectorAll('[data-slot-key]')
if (slotEls.length === 0) continue
const slots: SlotMeasurement[] = []
const nodeRect = nodeEl.getBoundingClientRect()
for (const slotEl of slotEls) {
const slotRect = slotEl.getBoundingClientRect()
const slotKey = (slotEl as HTMLElement).dataset.slotKey ?? 'unknown'
slots.push({
key: slotKey,
offsetX: slotRect.left + slotRect.width / 2 - nodeRect.left,
offsetY: slotRect.top + slotRect.height / 2 - nodeRect.top
})
}
slotData.push({
nodeId,
isSubgraph: !!node.isSubgraphNode?.(),
nodeW: nodeRect.width,
nodeH: nodeRect.height,
slots
})
}
return slotData
})
const subgraphNodes = result.filter((n) => n.isSubgraph)
expect(subgraphNodes.length).toBeGreaterThan(0)
for (const node of subgraphNodes) {
for (const slot of node.slots) {
expect(
slot.offsetX,
`Slot ${slot.key} on node ${node.nodeId}: X offset ${slot.offsetX} outside node width ${node.nodeW}`
).toBeGreaterThanOrEqual(-SLOT_BOUNDS_MARGIN)
expect(
slot.offsetX,
`Slot ${slot.key} on node ${node.nodeId}: X offset ${slot.offsetX} outside node width ${node.nodeW}`
).toBeLessThanOrEqual(node.nodeW + SLOT_BOUNDS_MARGIN)
expect(
slot.offsetY,
`Slot ${slot.key} on node ${node.nodeId}: Y offset ${slot.offsetY} outside node height ${node.nodeH}`
).toBeGreaterThanOrEqual(-SLOT_BOUNDS_MARGIN)
expect(
slot.offsetY,
`Slot ${slot.key} on node ${node.nodeId}: Y offset ${slot.offsetY} outside node height ${node.nodeH}`
).toBeLessThanOrEqual(node.nodeH + SLOT_BOUNDS_MARGIN)
}
}
})
}
)

View File

@@ -206,6 +206,31 @@ test.describe('Templates', { tag: ['@slow', '@workflow'] }, () => {
await expect(nav).toBeVisible() // Nav should be visible at tablet size
})
test(
'select components in filter bar render correctly',
{ tag: '@screenshot' },
async ({ comfyPage }) => {
await comfyPage.command.executeCommand('Comfy.BrowseTemplates')
await expect(comfyPage.templates.content).toBeVisible()
// Wait for filter bar select components to render
const dialog = comfyPage.page.getByRole('dialog')
const sortBySelect = dialog.getByRole('combobox', { name: /Sort/ })
await expect(sortBySelect).toBeVisible()
// Screenshot the filter bar containing MultiSelect and SingleSelect
const filterBar = sortBySelect.locator(
'xpath=ancestor::div[contains(@class, "justify-between")]'
)
await expect(filterBar).toHaveScreenshot(
'template-filter-bar-select-components.png',
{
mask: [comfyPage.page.locator('.p-toast')]
}
)
}
)
test(
'template cards descriptions adjust height dynamically',
{ tag: '@screenshot' },

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 113 KiB

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 57 KiB

View File

@@ -47,6 +47,46 @@ test.describe('Vue Node Moving', () => {
}
)
test('should not move node when pointer moves less than drag threshold', async ({
comfyPage
}) => {
const headerPos = await getLoadCheckpointHeaderPos(comfyPage)
// Move only 2px — below the 3px drag threshold in useNodePointerInteractions
await comfyPage.page.mouse.move(headerPos.x, headerPos.y)
await comfyPage.page.mouse.down()
await comfyPage.page.mouse.move(headerPos.x + 2, headerPos.y + 1, {
steps: 5
})
await comfyPage.page.mouse.up()
await comfyPage.nextFrame()
const afterPos = await getLoadCheckpointHeaderPos(comfyPage)
expect(afterPos.x).toBeCloseTo(headerPos.x, 0)
expect(afterPos.y).toBeCloseTo(headerPos.y, 0)
// The small movement should have selected the node, not dragged it
expect(await comfyPage.vueNodes.getSelectedNodeCount()).toBe(1)
})
test('should move node when pointer moves beyond drag threshold', async ({
comfyPage
}) => {
const headerPos = await getLoadCheckpointHeaderPos(comfyPage)
// Move 50px — well beyond the 3px drag threshold
await comfyPage.page.mouse.move(headerPos.x, headerPos.y)
await comfyPage.page.mouse.down()
await comfyPage.page.mouse.move(headerPos.x + 50, headerPos.y + 50, {
steps: 20
})
await comfyPage.page.mouse.up()
await comfyPage.nextFrame()
const afterPos = await getLoadCheckpointHeaderPos(comfyPage)
await expectPosChanged(headerPos, afterPos)
})
test(
'@mobile should allow moving nodes by dragging on touch devices',
{ tag: '@screenshot' },

View File

@@ -2,6 +2,7 @@ import {
comfyExpect as expect,
comfyPageFixture as test
} from '../../../../fixtures/ComfyPage'
import { TestIds } from '../../../../fixtures/selectors'
test.describe('Vue Upload Widgets', () => {
test.beforeEach(async ({ comfyPage }) => {
@@ -19,10 +20,14 @@ test.describe('Vue Upload Widgets', () => {
).not.toBeVisible()
await expect
.poll(() => comfyPage.page.getByText('Error loading image').count())
.poll(() =>
comfyPage.page.getByTestId(TestIds.errors.imageLoadError).count()
)
.toBeGreaterThan(0)
await expect
.poll(() => comfyPage.page.getByText('Error loading video').count())
.poll(() =>
comfyPage.page.getByTestId(TestIds.errors.videoLoadError).count()
)
.toBeGreaterThan(0)
})
})

View File

@@ -46,4 +46,16 @@ test.describe('Vue Multiline String Widget', () => {
await expect(textarea).toHaveValue('Keep me around')
})
test('should use native context menu when focused', async ({ comfyPage }) => {
const textarea = getFirstMultilineStringWidget(comfyPage)
const vueContextMenu = comfyPage.page.locator('.p-contextmenu')
await textarea.focus()
await textarea.click({ button: 'right' })
await expect(vueContextMenu).not.toBeVisible()
await textarea.blur()
await textarea.click({ button: 'right' })
await expect(vueContextMenu).toBeVisible()
})
})

View File

@@ -0,0 +1,73 @@
# QA Pipeline Troubleshooting
## Common Failures
### `set -euo pipefail` + grep with no match
**Symptom**: Deploy script crashes silently, badge shows FAILED.
**Cause**: `grep -oP` returns exit code 1 when no match. Under `pipefail`, this kills the entire script.
**Fix**: Always append `|| true` to grep pipelines in bash scripts.
### `__name is not defined` in page.evaluate
**Symptom**: Recording crashes with `ReferenceError: __name is not defined`.
**Cause**: tsx compiles arrow functions inside `page.evaluate()` with `__name` helpers. The browser context doesn't have these.
**Fix**: Use `page.addScriptTag({ content: '...' })` with plain JS strings instead of `page.evaluate(() => { ... })` with arrow functions.
### `Set<string>()` in page.evaluate
**Symptom**: Same `__name` error.
**Cause**: TypeScript generics like `new Set<string>()` get compiled incorrectly for browser context.
**Fix**: Use `new Set()` without type parameter.
### `zod/v4` import error
**Symptom**: `ERR_PACKAGE_PATH_NOT_EXPORTED: Package subpath './v4' is not defined`.
**Cause**: claude-agent-sdk depends on `zod/v4` internally, but the project's zod doesn't export it.
**Fix**: Import from `zod` (not `zod/v4`) in project code.
### `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`
**Symptom**: pnpm install fails with frozen lockfile mismatch.
**Cause**: Adding a new dependency changes the workspace catalog but lockfile wasn't regenerated.
**Fix**: Run `pnpm install` to regenerate lockfile, commit `pnpm-workspace.yaml` + `pnpm-lock.yaml`.
### `loadDefaultWorkflow` — "Load Default" not found
**Symptom**: Menu item "Load Default" not found, canvas stays empty.
**Cause**: The menu item name varies by version/locale. Menu navigation is fragile.
**Fix**: Use `app.resetToDefaultWorkflow()` JS API via `page.evaluate` instead of menu navigation.
### Model ID not found (Claude Agent SDK)
**Symptom**: `There's an issue with the selected model (claude-sonnet-4-6-20250514)`.
**Cause**: Dated model IDs like `claude-sonnet-4-6-20250514` don't exist.
**Fix**: Use `claude-sonnet-4-6` (no date suffix).
### Model not found (Gemini)
**Symptom**: 404 from Gemini API.
**Cause**: Preview model names like `gemini-2.5-flash-preview-05-20` expire.
**Fix**: Use `gemini-3-flash-preview` (latest stable).
## Badge Mismatches
### False REPRODUCED
**Symptom**: Badge says REPRODUCED but AI review says "could not reproduce".
**Root cause**: Grep pattern `reproduc|confirm` matches neutral words like "reproduction steps" or "could not be confirmed".
**Fix**: Use structured JSON verdict from AI (`## Verdict` section with `{"verdict": "..."}`) instead of regex matching the prose.
### INCONCLUSIVE feedback loop
**Symptom**: Once an issue gets INCONCLUSIVE, all future runs stay INCONCLUSIVE.
**Cause**: QA bot's own previous comments contain "INCONCLUSIVE", which gets fed back into pr-context.txt.
**Fix**: Filter out `github-actions[bot]` comments when building pr-context.
### pressKey with hold prevents event propagation
**Symptom**: BEFORE video doesn't show the bug (e.g., Escape doesn't close dialog).
**Cause**: `keyboard.down()` + 400ms sleep + `keyboard.up()` changes event timing. Some UI frameworks handle held keys differently than instant presses.
**Fix**: Use instant `keyboard.press()` for testing. Show key name via subtitle overlay instead.
## Cursor Not Visible
**Symptom**: No mouse cursor in recorded videos.
**Cause**: Headless Chrome doesn't render system cursor. The CSS cursor overlay relies on DOM `mousemove` events which Playwright CDP doesn't reliably trigger.
**Fix**: Monkey-patch `page.mouse.move/click/dblclick/down/up` to call `__moveCursor(x,y)` on the injected cursor div. This makes ALL mouse operations update the overlay.
## Agent Doesn't Perform Steps
**Symptom**: Agent opens menus and settings but never interacts with the canvas.
**Causes**:
1. `loadDefaultWorkflow` failed (no nodes on canvas)
2. Agent ran out of turn budget (30 turns / 120s)
3. Gemini Flash (old agent) ignores prompt hints
**Fix**: Use hybrid agent (Claude Sonnet 4.6 + Gemini vision). Claude's superior reasoning follows instructions precisely.

59
docs/qa/backlog.md Normal file
View File

@@ -0,0 +1,59 @@
# QA Pipeline Backlog
## Comparison Modes
### Type A: Same code, different settings (IMPLEMENTED)
Agent demonstrates both working (control) and broken (test) states in one session by toggling settings. E.g., Nodes 2.0 OFF → drag works, Nodes 2.0 ON → drag broken.
### Type B: Different commits
For regressions reported as "worked in vX.Y, broken in vX.Z":
- `qa-analyze-pr.ts` detects regression markers ("since v1.38", "after PR #1234")
- Pipeline checks out the old commit, records control video
- Records test video on current main
- Side-by-side comparison on report page (reuses PR before/after infra)
### Type C: Different browsers
For browser-specific bugs ("works on Chrome, broken on Firefox"):
- Run recording with different Playwright browser contexts
- Compare behavior across browsers in one report
## Agent Improvements
### TTS Narration
- OpenAI TTS (`tts-1`, nova voice) generates audio from agent reasoning
- Merged into video via ffmpeg at correct timestamps
- Currently in qa-record.ts but needs wiring into hybrid agent path
### Image/Screenshot Reading
- `qa-analyze-pr.ts` already downloads and sends images from issue bodies to Gemini
- Could also send them to the Claude agent as context ("the reporter showed this screenshot")
### Placeholder Page
- Deploy a status page immediately when CI starts
- Auto-refreshes every 30s until final report replaces it
- Shows spinner, CI link, badge
### Pre-seed Assets
- Upload test images via ComfyUI API before recording
- Enables reproduction of bugs requiring assets (#10424 zoom button)
### Environment-Dependent Issues
- #7942: needs custom TestNode — could install a test custom node pack in CI
- #9101: needs completed generation — could run with a tiny model checkpoint
## Cost Optimization
### Lazy A11y Tree
- `inspect(selector)` searches tree for specific element (~20 tokens)
- `getUIChanges()` diffs against previous snapshot (~100 tokens)
- vs dumping full tree every turn (~2000 tokens)
### Gemini Video vs Images
- 30s video clip: ~7,700 tokens (258 tok/s)
- 15 screenshots: ~19,500 tokens (1,300 tok/frame)
- Video is 2.5x cheaper and shows temporal changes
### Model Selection
- Claude Sonnet 4.6: $3/$15 per 1M in/out — best reasoning
- Gemini 2.5 Flash: $0.10/$0.40 per 1M — best vision-per-dollar
- Hybrid uses each where it's strongest

60
docs/qa/models.md Normal file
View File

@@ -0,0 +1,60 @@
# QA Pipeline Model Selection
## Current Configuration
| Script | Role | Model | Why |
| --------------------- | -------------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------- |
| `qa-analyze-pr.ts` | PR/issue analysis, QA guide generation | `gemini-3.1-pro-preview` | Needs deep reasoning over PR diffs, screenshots, and issue threads |
| `qa-record.ts` | Playwright step generation | `gemini-3.1-pro-preview` | Step quality is critical — must understand ComfyUI's canvas UI and produce precise action sequences |
| `qa-video-review.ts` | Video comparison review | `gemini-3-flash-preview` | Video analysis with structured output; flash is sufficient and faster |
| `qa-generate-test.ts` | Regression test generation | `gemini-3-flash-preview` | Code generation from QA reports; flash handles this well |
## Model Comparison
### Gemini 3.1 Pro vs GPT-5.4
| | Gemini 3.1 Pro Preview | GPT-5.4 |
| ----------------- | ---------------------- | ----------------- |
| Context window | 1M tokens | 1M tokens |
| Max output | 65K tokens | 128K tokens |
| Video input | Yes | No |
| Image input | Yes | Yes |
| Audio input | Yes | No |
| Pricing (input) | $2/1M tokens | $2.50/1M tokens |
| Pricing (output) | $12/1M tokens | $15/1M tokens |
| Function calling | Yes | Yes |
| Code execution | Yes | Yes (interpreter) |
| Structured output | Yes | Yes |
**Why Gemini over GPT for QA:**
- Native video understanding (can review recordings directly)
- Lower cost at comparable quality
- Native multimodal input (screenshots, videos, audio from issue threads)
- Better price/performance for high-volume CI usage
### Gemini 3 Flash vs GPT-5.4 Mini
| | Gemini 3 Flash Preview | GPT-5.4 Mini |
| ---------------- | ---------------------- | --------------- |
| Context window | 1M tokens | 1M tokens |
| Pricing (input) | $0.50/1M tokens | $0.40/1M tokens |
| Pricing (output) | $3/1M tokens | $1.60/1M tokens |
| Video input | Yes | No |
**Why Gemini Flash for video review:**
- Video input support is required — GPT models cannot process video files
- Good enough quality for structured comparison reports
## Upgrade History
| Date | Change | Reason |
| ---------- | ---------------------------------------------------------------- | -------------------------------------------------------------------------- |
| 2026-03-24 | `gemini-2.5-flash``gemini-3.1-pro-preview` (record) | Shallow step generation; pro model needed for complex ComfyUI interactions |
| 2026-03-24 | `gemini-2.5-pro``gemini-3.1-pro-preview` (analyze) | Keep analysis on latest pro |
| 2026-03-24 | `gemini-2.5-flash``gemini-3-flash-preview` (review, test-gen) | Latest flash for cost-efficient tasks |
## Override
All scripts accept `--model <name>` to override the default. Pass any Gemini model ID.

View File

@@ -27,9 +27,20 @@ const config: KnipConfig = {
},
'packages/ingest-types': {
project: ['src/**/*.{js,ts}']
},
'apps/website': {
entry: [
'src/pages/**/*.astro',
'src/layouts/**/*.astro',
'src/components/**/*.vue',
'src/styles/global.css',
'astro.config.ts'
],
project: ['src/**/*.{astro,vue,ts}', '*.{js,ts,mjs}'],
ignoreDependencies: ['@comfyorg/design-system', '@vercel/analytics']
}
},
ignoreBinaries: ['python3'],
ignoreBinaries: ['python3', 'wrangler'],
ignoreDependencies: [
// Weird importmap things
'@iconify-json/lucide',

View File

@@ -1,6 +1,6 @@
{
"name": "@comfyorg/comfyui-frontend",
"version": "1.43.2",
"version": "1.43.4",
"private": true,
"description": "Official front-end implementation of ComfyUI",
"homepage": "https://comfy.org",
@@ -39,6 +39,7 @@
"preinstall": "pnpm dlx only-allow pnpm",
"prepare": "husky || true && git config blame.ignoreRevsFile .git-blame-ignore-revs || true",
"preview": "nx preview",
"qa:video-review": "tsx scripts/qa-video-review.ts",
"storybook": "nx storybook",
"storybook:desktop": "nx run @comfyorg/desktop-ui:storybook",
"stylelint:fix": "stylelint --cache --fix '{apps,packages,src}/**/*.{css,vue}'",
@@ -120,7 +121,9 @@
"zod-validation-error": "catalog:"
},
"devDependencies": {
"@anthropic-ai/claude-agent-sdk": "catalog:",
"@eslint/js": "catalog:",
"@google/generative-ai": "catalog:",
"@intlify/eslint-plugin-vue-i18n": "catalog:",
"@lobehub/i18n-cli": "catalog:",
"@nx/eslint": "catalog:",

View File

@@ -0,0 +1,46 @@
/*
* Design System Base — Brand tokens + fonts only.
* For marketing sites that don't use PrimeVue or the node editor.
* Import the full style.css instead for the desktop app.
*/
@import './fonts.css';
@theme {
/* Font Families */
--font-inter: 'Inter', sans-serif;
/* Palette Colors */
--color-charcoal-100: #55565e;
--color-charcoal-200: #494a50;
--color-charcoal-300: #3c3d42;
--color-charcoal-400: #313235;
--color-charcoal-500: #2d2e32;
--color-charcoal-600: #262729;
--color-charcoal-700: #202121;
--color-charcoal-800: #171718;
--color-neutral-550: #636363;
--color-ash-300: #bbbbbb;
--color-ash-500: #828282;
--color-ash-800: #444444;
--color-smoke-100: #f3f3f3;
--color-smoke-200: #e9e9e9;
--color-smoke-300: #e1e1e1;
--color-smoke-400: #d9d9d9;
--color-smoke-500: #c5c5c5;
--color-smoke-600: #b4b4b4;
--color-smoke-700: #a0a0a0;
--color-smoke-800: #8a8a8a;
--color-white: #ffffff;
--color-black: #000000;
/* Brand Colors */
--color-electric-400: #f0ff41;
--color-sapphire-700: #172dd7;
--color-brand-yellow: var(--color-electric-400);
--color-brand-blue: var(--color-sapphire-700);
}

2124
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -4,9 +4,12 @@ packages:
catalog:
'@alloc/quick-lru': ^5.2.0
'@anthropic-ai/claude-agent-sdk': ^0.2.85
'@astrojs/vue': ^5.0.0
'@comfyorg/comfyui-electron-types': 0.6.2
'@eslint/js': ^9.39.1
'@formkit/auto-animate': ^0.9.0
'@google/generative-ai': ^0.24.1
'@iconify-json/lucide': ^1.1.178
'@iconify/json': ^2.2.380
'@iconify/tailwind4': ^1.2.0
@@ -50,6 +53,7 @@ catalog:
'@types/node': ^24.1.0
'@types/semver': ^7.7.0
'@types/three': ^0.169.0
'@vercel/analytics': ^2.0.1
'@vitejs/plugin-vue': ^6.0.0
'@vitest/coverage-v8': ^4.0.16
'@vitest/ui': ^4.0.16
@@ -58,6 +62,7 @@ catalog:
'@vueuse/integrations': ^14.2.0
'@webgpu/types': ^0.1.66
algoliasearch: ^5.21.0
astro: ^5.10.0
axios: ^1.13.5
cross-env: ^10.1.0
cva: 1.0.0-beta.4

347
scripts/qa-agent.ts Normal file
View File

@@ -0,0 +1,347 @@
#!/usr/bin/env tsx
/**
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
*
* Instead of driving a browser interactively, Claude:
* 1. Reads the issue + a11y snapshot of the UI
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
* 3. Runs the test → reads errors → rewrites → repeats until it works
* 4. Outputs the passing test + verdict
*
* Tools:
* - inspect(selector) — read a11y tree to understand UI state
* - writeTest(code) — write a Playwright test file
* - runTest() — execute the test and get results
* - done(verdict, summary, testCode) — finish with the working test
*/
import type { Page } from '@playwright/test'
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
import { z } from 'zod'
import { mkdirSync, writeFileSync } from 'fs'
import { execSync } from 'child_process'
// ── Types ──
interface ResearchOptions {
page: Page
issueContext: string
qaGuide: string
outputDir: string
serverUrl: string
anthropicApiKey?: string
maxTurns?: number
timeBudgetMs?: number
}
export interface ResearchResult {
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
summary: string
evidence: string
testCode: string
log: Array<{
turn: number
timestampMs: number
toolName: string
toolInput: unknown
toolResult: string
}>
}
// ── Main research function ──
export async function runResearchPhase(
opts: ResearchOptions
): Promise<ResearchResult> {
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
opts
const maxTurns = opts.maxTurns ?? 50
const timeBudgetMs = opts.timeBudgetMs ?? 600_000 // 10 min for write→run→fix loops
let agentDone = false
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
let finalSummary = 'Agent did not complete'
let finalEvidence = ''
let finalTestCode = ''
let turnCount = 0
const startTime = Date.now()
const researchLog: ResearchResult['log'] = []
const testDir = `${outputDir}/research`
mkdirSync(testDir, { recursive: true })
const testPath = `${testDir}/reproduce.spec.ts`
// Get initial a11y snapshot for context
let initialA11y = ''
try {
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
initialA11y = initialA11y.slice(0, 3000)
} catch {
initialA11y = '(could not capture initial a11y snapshot)'
}
// ── Tool: inspect ──
const inspectTool = tool(
'inspect',
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
{
selector: z
.string()
.optional()
.describe(
'Optional filter — only show elements matching this name/role. Omit for full tree.'
)
},
async (args) => {
let resultText: string
try {
const ariaText = await page
.locator('body')
.ariaSnapshot({ timeout: 5000 })
if (args.selector) {
const lines = ariaText.split('\n')
const matches = lines.filter((l: string) =>
l.toLowerCase().includes(args.selector!.toLowerCase())
)
resultText =
matches.length > 0
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
} else {
resultText = ariaText.slice(0, 3000)
}
} catch (e) {
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'inspect',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: writeTest ──
const writeTestTool = tool(
'writeTest',
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
{
code: z
.string()
.describe('Complete Playwright test file content (.spec.ts)')
},
async (args) => {
writeFileSync(testPath, args.code)
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'writeTest',
toolInput: { path: testPath, codeLength: args.code.length },
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
})
return {
content: [
{
type: 'text' as const,
text: `Test written to ${testPath}. Use runTest() to execute it.`
}
]
}
}
)
// ── Tool: runTest ──
// Place test in browser_tests/ so Playwright config finds fixtures
const projectRoot = process.cwd()
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
const runTestTool = tool(
'runTest',
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
{},
async () => {
turnCount++
// Copy the test to browser_tests/tests/ where Playwright expects it
const { copyFileSync } = await import('fs')
try {
copyFileSync(testPath, browserTestPath)
} catch {
// directory may not exist
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
copyFileSync(testPath, browserTestPath)
}
let resultText: string
try {
const output = execSync(
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
{
timeout: 90000,
encoding: 'utf-8',
env: {
...process.env,
COMFYUI_BASE_URL: serverUrl
}
}
)
resultText = `TEST PASSED:\n${output.slice(-1500)}`
} catch (e) {
const err = e as { stdout?: string; stderr?: string; message?: string }
const output = (err.stdout || '') + '\n' + (err.stderr || '')
resultText = `TEST FAILED:\n${output.slice(-2000)}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'runTest',
toolInput: { testPath },
toolResult: resultText.slice(0, 1000)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: done ──
const doneTool = tool(
'done',
'Finish research with verdict and the final test code.',
{
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
summary: z.string().describe('What you found and why'),
evidence: z.string().describe('Test output that proves the verdict'),
testCode: z
.string()
.describe(
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
)
},
async (args) => {
agentDone = true
finalVerdict = args.verdict
finalSummary = args.summary
finalEvidence = args.evidence
finalTestCode = args.testCode
writeFileSync(testPath, args.testCode)
return {
content: [
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
]
}
}
)
// ── MCP Server ──
const server = createSdkMcpServer({
name: 'qa-research',
version: '1.0.0',
tools: [inspectTool, writeTestTool, runTestTool, doneTool]
})
// ── System prompt ──
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
## Your tools
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
- writeTest(code) — Write a Playwright test file (.spec.ts)
- runTest() — Execute the test and get results (pass/fail + errors)
- done(verdict, summary, evidence, testCode) — Finish with the final test
## Workflow
1. Read the issue description carefully
2. Use inspect() to understand the current UI state and discover element selectors
3. Write a Playwright test that:
- Navigates to ${serverUrl}
- Performs the exact reproduction steps from the issue
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
4. Run the test with runTest()
5. If it fails: read the error, fix the test, run again (max 5 attempts)
6. Call done() with the final verdict and test code
## Test writing guidelines
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
- Import expect: \`import { expect } from '@playwright/test'\`
- The fixture provides \`comfyPage\` which has:
- \`comfyPage.page\` — the Playwright Page object
- \`comfyPage.menu.topbar\` — topbar actions (saveWorkflowAs, getTabNames, getWorkflowTab)
- \`comfyPage.menu.topbar.triggerTopbarCommand(label)\` — click a menu command
- \`comfyPage.workflow\` — workflow helpers (isCurrentWorkflowModified, setupWorkflowsDirectory)
- \`comfyPage.canvas\` — canvas element for mouse interactions
- \`comfyPage.settings.setSetting(id, value)\` — change settings
- \`comfyPage.nextFrame()\` — wait for next render frame
- \`comfyPage.loadWorkflow(name)\` — load a named workflow
- Use beforeEach to set up settings and workflow directory
- Use afterEach to clean up (setupWorkflowsDirectory({}))
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
- Keep tests focused and minimal — test ONLY the reported bug
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
## Current UI state (accessibility tree)
${initialA11y}
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
## Issue to Reproduce
${issueContext}`
// ── Run the agent ──
console.warn('Starting research phase (Claude writes E2E tests)...')
try {
for await (const message of query({
prompt:
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
options: {
model: 'claude-sonnet-4-6',
systemPrompt,
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
maxTurns,
mcpServers: { 'qa-research': server },
allowedTools: [
'mcp__qa-research__inspect',
'mcp__qa-research__writeTest',
'mcp__qa-research__runTest',
'mcp__qa-research__done'
]
}
})) {
if (message.type === 'assistant' && message.message?.content) {
for (const block of message.message.content) {
if ('text' in block && block.text) {
console.warn(` Claude: ${block.text.slice(0, 200)}`)
}
if ('name' in block) {
console.warn(
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
)
}
}
}
if (agentDone) break
}
} catch (e) {
console.warn(`Research error: ${e instanceof Error ? e.message : e}`)
}
const result: ResearchResult = {
verdict: finalVerdict,
summary: finalSummary,
evidence: finalEvidence,
testCode: finalTestCode,
log: researchLog
}
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
console.warn(
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
)
return result
}

View File

@@ -0,0 +1,84 @@
import { describe, expect, it } from 'vitest'
import { extractMediaUrls } from './qa-analyze-pr'
describe('extractMediaUrls', () => {
it('extracts markdown image URLs', () => {
const text = '![screenshot](https://example.com/image.png)'
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
})
it('extracts multiple markdown images', () => {
const text = [
'![before](https://example.com/before.png)',
'Some text',
'![after](https://example.com/after.jpg)'
].join('\n')
expect(extractMediaUrls(text)).toEqual([
'https://example.com/before.png',
'https://example.com/after.jpg'
])
})
it('extracts raw URLs with media extensions', () => {
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
})
it('extracts GitHub user-attachments URLs', () => {
const text =
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
expect(extractMediaUrls(text)).toEqual([
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
])
})
it('extracts private-user-images URLs', () => {
const text =
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
expect(extractMediaUrls(text)).toEqual([
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
])
})
it('extracts URLs with query parameters', () => {
const text = 'https://example.com/image.png?w=800&h=600'
expect(extractMediaUrls(text)).toEqual([
'https://example.com/image.png?w=800&h=600'
])
})
it('deduplicates URLs', () => {
const text = [
'![img](https://example.com/same.png)',
'![img2](https://example.com/same.png)',
'Also https://example.com/same.png'
].join('\n')
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
})
it('returns empty array for empty input', () => {
expect(extractMediaUrls('')).toEqual([])
})
it('returns empty array for text with no media URLs', () => {
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
})
it('handles mixed media types', () => {
const text = [
'![screen](https://example.com/screenshot.png)',
'Video: https://example.com/demo.webm',
'![gif](https://example.com/animation.gif)'
].join('\n')
const urls = extractMediaUrls(text)
expect(urls).toContain('https://example.com/screenshot.png')
expect(urls).toContain('https://example.com/demo.webm')
expect(urls).toContain('https://example.com/animation.gif')
})
it('ignores non-http URLs in markdown', () => {
const text = '![local](./local-image.png)'
expect(extractMediaUrls(text)).toEqual([])
})
})

799
scripts/qa-analyze-pr.ts Normal file
View File

@@ -0,0 +1,799 @@
#!/usr/bin/env tsx
/**
* QA PR Analysis Script
*
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
* for before/after recording sessions. Fetches PR thread, extracts media,
* and produces structured test plans.
*
* Usage:
* pnpm exec tsx scripts/qa-analyze-pr.ts \
* --pr-number 10270 \
* --repo owner/repo \
* --output-dir qa-guides/ \
* [--model gemini-3.1-pro-preview]
*
* Env: GEMINI_API_KEY (required)
*/
import { execSync } from 'node:child_process'
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
// ── Types ──
interface QaGuideStep {
action: string
description: string
expected_before?: string
expected_after?: string
}
interface QaGuide {
summary: string
test_focus: string
prerequisites: string[]
steps: QaGuideStep[]
visual_checks: string[]
}
interface PrThread {
title: string
body: string
labels: string[]
issueComments: string[]
reviewComments: string[]
reviews: string[]
diff: string
}
type TargetType = 'pr' | 'issue'
interface Options {
prNumber: string
repo: string
outputDir: string
model: string
apiKey: string
mediaBudgetBytes: number
maxVideoBytes: number
type: TargetType
}
// ── CLI parsing ──
function parseArgs(): Options {
const args = process.argv.slice(2)
const opts: Partial<Options> = {
model: 'gemini-3.1-pro-preview',
apiKey: process.env.GEMINI_API_KEY || '',
mediaBudgetBytes: 20 * 1024 * 1024,
maxVideoBytes: 10 * 1024 * 1024,
type: 'pr'
}
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--pr-number':
opts.prNumber = args[++i]
break
case '--repo':
opts.repo = args[++i]
break
case '--output-dir':
opts.outputDir = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--type':
opts.type = args[++i] as TargetType
break
case '--help':
console.warn(
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
)
process.exit(0)
}
}
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
console.error(
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
)
process.exit(1)
}
if (!opts.apiKey) {
console.error('GEMINI_API_KEY environment variable is required')
process.exit(1)
}
return opts as Options
}
// ── PR thread fetching ──
function ghExec(cmd: string): string {
try {
return execSync(cmd, {
encoding: 'utf-8',
timeout: 30_000,
stdio: ['pipe', 'pipe', 'pipe']
}).trim()
} catch (err) {
console.warn(`gh command failed: ${cmd}`)
console.warn((err as Error).message)
return ''
}
}
function fetchPrThread(prNumber: string, repo: string): PrThread {
console.warn('Fetching PR thread...')
const prView = ghExec(
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
)
const prData = prView
? JSON.parse(prView)
: { title: '', body: '', labels: [] }
const issueCommentsRaw = ghExec(
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
)
const issueComments: string[] = issueCommentsRaw
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewCommentsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
)
const reviewComments: string[] = reviewCommentsRaw
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
)
const reviews: string[] = reviewsRaw
? JSON.parse(reviewsRaw)
.filter((r: { body: string }) => r.body)
.map((r: { body: string }) => r.body)
: []
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
console.warn(
`PR #${prNumber}: "${prData.title}" | ` +
`${issueComments.length} issue comments, ` +
`${reviewComments.length} review comments, ` +
`${reviews.length} reviews, ` +
`diff: ${diff.length} chars`
)
return {
title: prData.title || '',
body: prData.body || '',
labels: (prData.labels || []).map((l: { name: string }) => l.name),
issueComments,
reviewComments,
reviews,
diff
}
}
interface IssueThread {
title: string
body: string
labels: string[]
comments: string[]
}
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
console.warn('Fetching issue thread...')
const issueView = ghExec(
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
)
const issueData = issueView
? JSON.parse(issueView)
: { title: '', body: '', labels: [] }
const commentsRaw = ghExec(
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
)
const comments: string[] = commentsRaw
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
: []
console.warn(
`Issue #${issueNumber}: "${issueData.title}" | ` +
`${comments.length} comments`
)
return {
title: issueData.title || '',
body: issueData.body || '',
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
comments
}
}
// ── Media extraction ──
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
const MEDIA_URL_PATTERNS = [
// Markdown images: ![alt](url)
/!\[[^\]]*\]\(([^)]+)\)/g,
// GitHub user-attachments
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
// Private user images
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
// Raw URLs with media extensions (standalone or in text)
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
]
export function extractMediaUrls(text: string): string[] {
if (!text) return []
const urls = new Set<string>()
for (const pattern of MEDIA_URL_PATTERNS) {
// Reset lastIndex for global patterns
pattern.lastIndex = 0
let match: RegExpExecArray | null
while ((match = pattern.exec(text)) !== null) {
// For markdown images, the URL is in capture group 1
const url = match[1] || match[0]
// Clean trailing markdown/html artifacts
const cleaned = url.replace(/[)>"'\s]+$/, '')
if (cleaned.startsWith('http')) {
urls.add(cleaned)
}
}
}
return [...urls]
}
// ── Media downloading ──
const ALLOWED_MEDIA_DOMAINS = [
'github.com',
'raw.githubusercontent.com',
'user-images.githubusercontent.com',
'private-user-images.githubusercontent.com',
'objects.githubusercontent.com',
'github.githubassets.com'
]
function isAllowedMediaDomain(url: string): boolean {
try {
const hostname = new URL(url).hostname
return ALLOWED_MEDIA_DOMAINS.some(
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
)
} catch {
return false
}
}
async function downloadMedia(
urls: string[],
outputDir: string,
budgetBytes: number,
maxVideoBytes: number
): Promise<Array<{ path: string; mimeType: string }>> {
const downloaded: Array<{ path: string; mimeType: string }> = []
let totalBytes = 0
const mediaDir = resolve(outputDir, 'media')
mkdirSync(mediaDir, { recursive: true })
for (const url of urls) {
if (totalBytes >= budgetBytes) {
console.warn(
`Media budget exhausted (${totalBytes} bytes), skipping rest`
)
break
}
if (!isAllowedMediaDomain(url)) {
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
continue
}
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(15_000),
headers: { Accept: 'image/*,video/*' },
redirect: 'follow'
})
if (!response.ok) {
console.warn(`Failed to download ${url}: ${response.status}`)
continue
}
const contentLength = response.headers.get('content-length')
if (contentLength) {
const declaredSize = Number.parseInt(contentLength, 10)
if (declaredSize > budgetBytes - totalBytes) {
console.warn(
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
)
continue
}
}
const contentType = response.headers.get('content-type') || ''
const buffer = Buffer.from(await response.arrayBuffer())
// Skip oversized videos
const isVideo =
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
if (isVideo && buffer.length > maxVideoBytes) {
console.warn(
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
)
continue
}
if (totalBytes + buffer.length > budgetBytes) {
console.warn(`Would exceed budget, skipping ${url}`)
continue
}
const ext = guessExtension(url, contentType)
const filename = `media-${downloaded.length}${ext}`
const filepath = resolve(mediaDir, filename)
writeFileSync(filepath, buffer)
totalBytes += buffer.length
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
downloaded.push({ path: filepath, mimeType })
console.warn(
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
)
} catch (err) {
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
}
}
console.warn(
`Downloaded ${downloaded.length}/${urls.length} media files ` +
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
)
return downloaded
}
function guessExtension(url: string, contentType: string): string {
const urlMatch = url.match(MEDIA_EXTENSIONS)
if (urlMatch) return urlMatch[0].toLowerCase()
const typeMap: Record<string, string> = {
'image/png': '.png',
'image/jpeg': '.jpg',
'image/gif': '.gif',
'image/webp': '.webp',
'video/mp4': '.mp4',
'video/webm': '.webm'
}
return typeMap[contentType.split(';')[0]] || '.bin'
}
function guessMimeType(ext: string): string {
const map: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime'
}
return map[ext] || 'application/octet-stream'
}
// ── Gemini analysis ──
function buildIssueAnalysisPrompt(issue: IssueThread): string {
const allText = [
`# Issue: ${issue.title}`,
'',
'## Description',
issue.body,
'',
issue.comments.length > 0
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
: ''
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
The UI has:
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
- Nodes are boxes with input/output slots connected by wires
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
- Sidebars (Workflows, Node Library, Models)
- A topbar with workflow tabs and Queue button
- The default workflow loads with these nodes (approximate center coordinates):
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
${allText}
## Available test actions
Each step must use one of these actions:
### Menu actions
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
### Element actions (by visible text)
- "click" — clicks an element by visible text (text required)
- "rightClick" — right-clicks an element to open context menu (text required)
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
### Canvas actions (by coordinates — viewport is 1280x720)
- "clickCanvas" — click at coordinates (x, y required)
- "rightClickCanvas" — right-click at coordinates (x, y required)
- "doubleClick" — double-click at coordinates to open node search (x, y)
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
### Utility
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Common ComfyUI interactions
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
- Double-click empty canvas → opens node search dialog
- Ctrl+C / Ctrl+V → copy/paste selected nodes
- Delete key → remove selected node
- Ctrl+G → group selected nodes
- Drag from output slot to input slot → create connection
- Click a node to select it, Shift+click for multi-select
## Output format
Return a JSON object with exactly one key: "reproduce", containing:
{
"summary": "One sentence: what bug this issue reports",
"test_focus": "Specific behavior to reproduce",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "clickCanvas",
"description": "Click on first node to select it",
"expected_before": "What should happen if the bug is present"
}
],
"visual_checks": ["Specific visual evidence of the bug to look for"]
}
## Rules
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
- Do NOT include login steps.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
function buildAnalysisPrompt(thread: PrThread): string {
const allText = [
`# PR: ${thread.title}`,
'',
'## Description',
thread.body,
'',
thread.issueComments.length > 0
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
: '',
thread.reviewComments.length > 0
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
: '',
thread.reviews.length > 0
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
: '',
'',
'## Diff (truncated)',
'```',
thread.diff.slice(0, 8000),
'```'
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
${allText}
## Available test actions
Each step must use one of these actions:
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required)
- "click" — clicks an element by visible text (text required)
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Output format
Return a JSON object with exactly two keys: "before" and "after", each containing:
{
"summary": "One sentence: what this PR changes",
"test_focus": "Specific behaviors to verify in this recording",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "openMenu",
"description": "Open the main menu to check file options",
"expected_before": "Old behavior description (before key only)",
"expected_after": "New behavior description (after key only)"
}
],
"visual_checks": ["Specific visual elements to look for"]
}
## Rules
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
- Use information from PR description, screenshots, and comments to understand intended behavior.
- Include at least one screenshot step in each guide.
- Do NOT include login steps.
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
async function analyzeWithGemini(
thread: PrThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<{ before: QaGuide; after: QaGuide }> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildAnalysisPrompt(thread)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
// Add media as inline data
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
// Strip markdown fences if present
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
// Handle different response shapes from Gemini
let before: QaGuide
let after: QaGuide
if (Array.isArray(parsed) && parsed.length >= 2) {
// Array format: [before, after]
before = parsed[0]
after = parsed[1]
} else if (parsed.before && parsed.after) {
// Object format: { before, after }
before = parsed.before
after = parsed.after
} else {
// Try nested wrapper keys
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
if (inner.before && inner.after) {
before = inner.before
after = inner.after
} else {
console.warn(
'Full response:',
JSON.stringify(parsed, null, 2).slice(0, 2000)
)
throw new Error(
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
)
}
}
return { before, after }
}
async function analyzeIssueWithGemini(
issue: IssueThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<QaGuide> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildIssueAnalysisPrompt(issue)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
const guide: QaGuide =
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
return guide
}
// ── Main ──
async function main() {
const opts = parseArgs()
mkdirSync(opts.outputDir, { recursive: true })
if (opts.type === 'issue') {
await analyzeIssue(opts)
} else {
await analyzePr(opts)
}
}
async function analyzeIssue(opts: Options) {
const issue = fetchIssueThread(opts.prNumber, opts.repo)
const allText = [issue.body, ...issue.comments].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guide = await analyzeIssueWithGemini(
issue,
media,
opts.model,
opts.apiKey
)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
console.warn(`Wrote QA guide:`)
console.warn(` Reproduce: ${beforePath}`)
}
async function analyzePr(opts: Options) {
const thread = fetchPrThread(opts.prNumber, opts.repo)
const allText = [
thread.body,
...thread.issueComments,
...thread.reviewComments,
...thread.reviews
].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
console.warn(`Wrote QA guides:`)
console.warn(` Before: ${beforePath}`)
console.warn(` After: ${afterPath}`)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
if (isExecutedAsScript(import.meta.url)) {
main().catch((err) => {
console.error('PR analysis failed:', err)
process.exit(1)
})
}

176
scripts/qa-batch.sh Executable file
View File

@@ -0,0 +1,176 @@
#!/usr/bin/env bash
# Batch-trigger QA runs by creating and pushing sno-qa-* branches.
#
# Usage:
# ./scripts/qa-batch.sh 10394 10238 9996 # Trigger specific numbers
# ./scripts/qa-batch.sh --from tmp/issues.md --top 5 # From triage file
# ./scripts/qa-batch.sh --dry-run 10394 10238 # Preview only
# ./scripts/qa-batch.sh --cleanup # Delete old sno-qa-* branches
set -euo pipefail
DELAY=5
DRY_RUN=false
CLEANUP=false
FROM_FILE=""
TOP_N=0
NUMBERS=()
die() { echo "error: $*" >&2; exit 1; }
usage() {
cat <<'EOF'
Usage: qa-batch.sh [options] [numbers...]
Options:
--from <file> Extract numbers from a triage markdown file
--top <N> Take first N entries from Tier 1 (requires --from)
--dry-run Print what would happen without pushing
--cleanup Delete all sno-qa-* remote branches
--delay <secs> Seconds between pushes (default: 5)
-h, --help Show this help
EOF
exit 0
}
# --- Parse args ---
while [[ $# -gt 0 ]]; do
case "$1" in
--from) FROM_FILE="$2"; shift 2 ;;
--top) TOP_N="$2"; shift 2 ;;
--dry-run) DRY_RUN=true; shift ;;
--cleanup) CLEANUP=true; shift ;;
--delay) DELAY="$2"; shift 2 ;;
-h|--help) usage ;;
-*) die "unknown option: $1" ;;
*) NUMBERS+=("$1"); shift ;;
esac
done
# --- Cleanup mode ---
if $CLEANUP; then
echo "Fetching remote sno-qa-* branches..."
branches=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' | awk '{print $2}' | sed 's|refs/heads/||')
if [[ -z "$branches" ]]; then
echo "No sno-qa-* branches found on remote."
exit 0
fi
echo "Found branches:"
while IFS= read -r b; do echo " $b"; done <<< "$branches"
echo
if $DRY_RUN; then
echo "[dry-run] Would delete the above branches."
exit 0
fi
read -rp "Delete all of the above? [y/N] " confirm
if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then
echo "Aborted."
exit 0
fi
for branch in $branches; do
echo "Deleting origin/$branch..."
git push origin --delete "$branch"
done
echo "Done. Cleaned up $(echo "$branches" | wc -l | tr -d ' ') branches."
exit 0
fi
# --- Extract numbers from markdown ---
if [[ -n "$FROM_FILE" ]]; then
[[ -f "$FROM_FILE" ]] || die "file not found: $FROM_FILE"
[[ "$TOP_N" -gt 0 ]] || die "--top N required with --from"
# Extract Tier 1 table rows: | N | [#NNNNN](...) | ...
# Stop at the next ## heading after Tier 1
extracted=$(awk '/^## Tier 1/,/^## Tier [^1]/' "$FROM_FILE" \
| grep -oP '\[#\K\d+' \
| head -n "$TOP_N")
if [[ -z "$extracted" ]]; then
die "no numbers found in $FROM_FILE"
fi
while IFS= read -r num; do
NUMBERS+=("$num")
done <<< "$extracted"
fi
[[ ${#NUMBERS[@]} -gt 0 ]] || die "no numbers specified. Use positional args or --from/--top."
# --- Validate ---
for num in "${NUMBERS[@]}"; do
[[ "$num" =~ ^[0-9]+$ ]] || die "invalid number: $num"
done
# Deduplicate
# shellcheck disable=SC2207 # mapfile not available on macOS default bash
NUMBERS=($(printf '%s\n' "${NUMBERS[@]}" | sort -un))
# --- Push branches ---
echo "Triggering QA for: ${NUMBERS[*]}"
if $DRY_RUN; then
echo "[dry-run]"
fi
echo
pushed=()
skipped=()
# Fetch remote refs once
remote_refs=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' 2>/dev/null | awk '{print $2}' | sed 's|refs/heads/||')
for num in "${NUMBERS[@]}"; do
branch="sno-qa-$num"
# Check if already exists on remote
if echo "$remote_refs" | grep -qx "$branch"; then
echo " skip: $branch (already exists on remote)"
skipped+=("$num")
continue
fi
if $DRY_RUN; then
echo " would push: $branch"
pushed+=("$num")
continue
fi
# Create branch at current HEAD and push
git branch -f "$branch" HEAD
git push origin "$branch"
pushed+=("$num")
echo " pushed: $branch"
# Clean up local branch
git branch -D "$branch" 2>/dev/null || true
# Delay between pushes to avoid CI concurrency storm
if [[ "$num" != "${NUMBERS[-1]}" ]]; then
echo " waiting ${DELAY}s..."
sleep "$DELAY"
fi
done
# --- Summary ---
echo
echo "=== Summary ==="
echo "Triggered: ${#pushed[@]}"
echo "Skipped: ${#skipped[@]}"
if [[ ${#pushed[@]} -gt 0 ]]; then
echo
echo "Triggered numbers: ${pushed[*]}"
repo_url=$(git remote get-url origin | sed 's/\.git$//' | sed 's|git@github.com:|https://github.com/|')
echo "Actions: ${repo_url}/actions"
fi
if [[ ${#skipped[@]} -gt 0 ]]; then
echo
echo "Skipped (already exist): ${skipped[*]}"
echo "Use --cleanup first to remove old branches."
fi

358
scripts/qa-deploy-pages.sh Executable file
View File

@@ -0,0 +1,358 @@
#!/usr/bin/env bash
# Deploy QA report to Cloudflare Pages.
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
# Writes outputs to GITHUB_OUTPUT: badge_status, url
set -euo pipefail
npm install -g wrangler@4.74.0 >/dev/null 2>&1
DEPLOY_DIR=$(mktemp -d)
mkdir -p "$DEPLOY_DIR"
for os in Linux macOS Windows; do
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
for prefix in qa qa-before; do
VID="${DIR}/${prefix}-session.mp4"
if [ -f "$VID" ]; then
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
cp "$VID" "$DEST"
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
fi
done
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
for numbered in "$DIR"/qa-session-[0-9].mp4; do
[ -f "$numbered" ] || continue
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
cp "$numbered" "$DEST"
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
done
# Generate GIF thumbnail from after video (or first pass)
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
if [ -f "$THUMB_SRC" ]; then
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| echo "GIF generation failed for ${os} (non-fatal)"
fi
done
# Build video cards and report sections
CARDS=""
# shellcheck disable=SC2034 # accessed via eval
ICONS_Linux="&#x1F427;" ICONS_macOS="&#x1F34E;" ICONS_Windows="&#x1FA9F;"
CARD_COUNT=0
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
for os in Linux macOS Windows; do
eval "ICON=\$ICONS_${os}"
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
[ "$HAS_AFTER" = "0" ] && continue
# Collect all reports for this platform (single + multi-pass)
REPORT_FILES=""
REPORT_LINK=""
REPORT_HTML=""
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
done
if [ -n "$REPORT_FILES" ]; then
# Concatenate all reports into one combined report file
COMBINED_MD=""
for rpt in $REPORT_FILES; do
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
RPT_MD=$(sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' "$rpt")
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD}&#10;&#10;---&#10;&#10;"
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
done
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
FIRST_BASENAME=$(basename "$FIRST_REPORT")
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
fi
if [ "$HAS_BEFORE" = "1" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
else
PASS_VIDEOS=""
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
[ -f "$pass_vid" ] || continue
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
done
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
fi
CARD_COUNT=$((CARD_COUNT + 1))
done
# Build commit info and target link for the report header
COMMIT_HTML=""
REPO_URL="https://github.com/${REPO}"
if [ -n "${TARGET_NUM:-}" ]; then
if [ "$TARGET_TYPE" = "issue" ]; then
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
else
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
fi
fi
if [ -n "${BEFORE_SHA:-}" ]; then
SHORT_BEFORE="${BEFORE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
fi
if [ -n "${AFTER_SHA:-}" ]; then
SHORT_AFTER="${AFTER_SHA:0:7}"
AFTER_LABEL="PR"
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
fi
if [ -n "${PIPELINE_SHA:-}" ]; then
SHORT_PIPE="${PIPELINE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
fi
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" &middot; ${COMMIT_HTML}"
RUN_LINK=""
if [ -n "${RUN_URL:-}" ]; then
RUN_LINK=" &middot; <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
fi
# Timing info
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
TIMING_HTML=""
if [ -n "${RUN_START_TIME:-}" ]; then
TIMING_HTML=" &middot; <span class=sha title='Pipeline timing'>${RUN_START_TIME} &rarr; ${DEPLOY_TIME}</span>"
fi
# Generate index.html from template
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
# Write dynamic content to temp files for safe substitution
# Cloudflare Pages _headers file — enable range requests for video seeking
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
/*.mp4
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
HEADERSEOF
# Build purpose description from pr-context.txt
PURPOSE_HTML=""
if [ -f pr-context.txt ]; then
# Extract title line and first paragraph of description
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt | sed 's/^Title: //')
if [ "$TARGET_TYPE" = "issue" ]; then
PURPOSE_LABEL="Issue #${TARGET_NUM}"
PURPOSE_VERB="reports"
else
PURPOSE_LABEL="PR #${TARGET_NUM}"
PURPOSE_VERB="aims to"
fi
# Get first ~300 chars of description body (after "Description:" line)
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400)
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400)
# Build requirements from QA guide JSON
REQS_HTML=""
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1)
if [ -f "$QA_GUIDE" ]; then
PREREQS=$(python3 -c "
import json, sys, html
try:
g = json.load(open(sys.argv[1]))
prereqs = g.get('prerequisites', [])
steps = g.get('steps', [])
focus = g.get('test_focus', '')
parts = []
if focus:
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
if prereqs:
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
if steps:
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
if len(steps) > 6:
parts[-1] += ' → ...'
print('<br>'.join(parts))
except: pass
" "$QA_GUIDE" 2>/dev/null)
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
fi
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
fi
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
# Badge HTML with copy button (placeholder URL filled after deploy)
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
python3 -c "
import sys, pathlib
d = pathlib.Path(sys.argv[1])
t = pathlib.Path(sys.argv[2]).read_text()
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
sys.stdout.write(t)
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
ERROREOF
# Copy research log to deploy dir if it exists
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
if [ -f "$rlog" ]; then
cp "$rlog" "$DEPLOY_DIR/research-log.json"
echo "Found research log: $rlog"
break
fi
done
# Generate badge SVGs into deploy dir
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
# Try research log first (ground truth from a11y assertions)
RESEARCH_VERDICT=""
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none}"
if [ -n "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=1
case "$RESEARCH_VERDICT" in
REPRODUCED) REPRO_COUNT=1 ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
INCONCLUSIVE) INCONC_COUNT=1 ;;
esac
fi
fi
# Fall back to video review verdicts if no research log
if [ -z "$RESEARCH_VERDICT" ] && [ -d video-reviews ]; then
for rpt in video-reviews/*-qa-video-report.md; do
[ -f "$rpt" ] || continue
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
# Try structured JSON verdict first (from ## Verdict section)
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
RISK_JSON=$(grep -oP '"risk":\s*"[a-z]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[a-z]+$' || true)
if [ -n "$VERDICT_JSON" ]; then
case "$VERDICT_JSON" in
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
esac
else
# Fallback: grep Summary section (for older reports without ## Verdict)
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
INCONC_COUNT=$((INCONC_COUNT + 1))
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
REPRO_COUNT=$((REPRO_COUNT + 1))
fi
fi
done
fi
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
echo "Verdict: ${REPRO_COUNT}${NOT_REPRO_COUNT}${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
# Badge text:
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
if [ "$TOTAL_REPORTS" -le 1 ]; then
# Single report — simple label
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
elif [ "$FAIL_COUNT" -gt 0 ]; then
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
fi
else
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
PARTS=""
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}"
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}"
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}"
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
# Color based on best outcome
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#9f9f9f"
fi
fi
# Badge label: #NUM QA0327 (with today's date)
QA_DATE=$(date -u '+%m%d')
BADGE_LABEL="QA${QA_DATE}"
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
# For PRs, also extract fix quality from Overall Risk section
FIX_RESULT="" FIX_COLOR="#4c1"
if [ "$TARGET_TYPE" != "issue" ]; then
# Try structured JSON risk first
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
if [ -n "$ALL_RISKS" ]; then
# Use worst risk across all reports
if echo "$ALL_RISKS" | grep -q 'high'; then
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
elif echo "$ALL_RISKS" | grep -q 'medium'; then
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
elif echo "$ALL_RISKS" | grep -q 'low'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
else
# Fallback: grep Overall Risk section
RISK_TEXT=""
if [ -d video-reviews ]; then
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20)
fi
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]')
if [ -n "$RISK_FIRST" ]; then
case "$RISK_FIRST" in
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
esac
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
fi
fi
# Always use vertical box badge
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
"$FIX_RESULT" "$FIX_COLOR"
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
URL=$(wrangler pages deploy "$DEPLOY_DIR" \
--project-name="comfy-qa" \
--branch="$BRANCH" 2>&1 \
| grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1)
echo "url=${URL:-https://${BRANCH}.comfy-qa.pages.dev}" >> "$GITHUB_OUTPUT"
echo "Deployed to: ${URL}"

208
scripts/qa-generate-test.ts Normal file
View File

@@ -0,0 +1,208 @@
#!/usr/bin/env tsx
/**
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
*
* Usage:
* pnpm exec tsx scripts/qa-generate-test.ts \
* --qa-report <path> QA video review report (markdown)
* --pr-diff <path> PR diff file
* --output <path> Output .spec.ts file path
* --model <name> Gemini model (default: gemini-3-flash-preview)
*/
import { readFile, writeFile } from 'node:fs/promises'
import { basename, resolve } from 'node:path'
import { GoogleGenerativeAI } from '@google/generative-ai'
interface CliOptions {
qaReport: string
prDiff: string
output: string
model: string
}
const DEFAULTS: CliOptions = {
qaReport: '',
prDiff: '',
output: '',
model: 'gemini-3-flash-preview'
}
// ── Fixture API reference for the prompt ────────────────────────────
const FIXTURE_API = `
## ComfyUI Playwright Test Fixture API
Import pattern:
\`\`\`typescript
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
\`\`\`
### Available helpers on \`comfyPage\`:
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
- \`.getActiveTabName(): Promise<string>\` — get active tab name
- \`.saveWorkflow(name)\` — Save via File > Save dialog
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
- \`.exportWorkflow(name)\` — Export via File > Export dialog
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
- \`.getWorkflowTab(name)\` — get a tab locator by name
- \`.closeWorkflowTab(name)\` — close a tab
- \`.openTopbarMenu()\` — open the hamburger menu
- \`.openSubmenu(label)\` — hover to open a submenu
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
- \`.open()\` / \`.close()\` — toggle sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
- \`.getPersistedItem(name)\` — get a workflow item locator
- \`comfyPage.workflow\` — WorkflowHelper:
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\` — delete a workflow
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
- \`comfyPage.settings.setSetting(key, value)\` — change settings
- \`comfyPage.keyboard\` — KeyboardHelper:
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
- \`comfyPage.nodeOps\` — NodeOperationsHelper
- \`comfyPage.canvas\` — CanvasHelper
- \`comfyPage.contextMenu\` — ContextMenu
- \`comfyPage.toast\` — ToastHelper
- \`comfyPage.confirmDialog\` — confirmation dialog
- \`comfyPage.nextFrame()\` — wait for Vue re-render
### Test patterns:
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
`
// ── Prompt builder ──────────────────────────────────────────────────
function buildPrompt(qaReport: string, prDiff: string): string {
return `You are a Playwright test generator for the ComfyUI frontend.
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
described in the QA report below. The test must:
1. Use the ComfyUI Playwright fixture API (documented below)
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
3. NOT test code implementation details
4. Be concise — only test the behavior that the PR changed
5. Follow existing test conventions (see API reference)
${FIXTURE_API}
## QA Video Review Report
${qaReport}
## PR Diff (for context on what changed)
${prDiff.slice(0, 8000)}
## Output Requirements
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
- Start with imports, end with closing brace
- Use descriptive test names that explain the expected behavior
- Add screenshot assertions where visual verification matters
- Keep it focused: 2-5 test cases covering the core behavioral change
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
`
}
// ── Gemini call ─────────────────────────────────────────────────────
async function generateTest(
qaReport: string,
prDiff: string,
model: string
): Promise<string> {
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
const genAI = new GoogleGenerativeAI(apiKey)
const genModel = genAI.getGenerativeModel({ model })
const prompt = buildPrompt(qaReport, prDiff)
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
const result = await genModel.generateContent({
contents: [{ role: 'user', parts: [{ text: prompt }] }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192
}
})
const text = result.response.text()
// Strip markdown fences if model wraps output
return text
.replace(/^```(?:typescript|ts)?\n?/, '')
.replace(/\n?```$/, '')
.trim()
}
// ── CLI ─────────────────────────────────────────────────────────────
function parseArgs(): CliOptions {
const args = process.argv.slice(2)
const opts = { ...DEFAULTS }
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--qa-report':
opts.qaReport = args[++i]
break
case '--pr-diff':
opts.prDiff = args[++i]
break
case '--output':
opts.output = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--help':
console.warn(`Usage:
pnpm exec tsx scripts/qa-generate-test.ts [options]
Options:
--qa-report <path> QA video review report (markdown) [required]
--pr-diff <path> PR diff file [required]
--output <path> Output .spec.ts path [required]
--model <name> Gemini model (default: gemini-3-flash-preview)`)
process.exit(0)
}
}
if (!opts.qaReport || !opts.prDiff || !opts.output) {
console.error('Missing required args. Run with --help for usage.')
process.exit(1)
}
return opts
}
async function main() {
const opts = parseArgs()
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
console.warn(
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
)
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
const testCode = await generateTest(qaReport, prDiff, opts.model)
const outputPath = resolve(opts.output)
await writeFile(outputPath, testCode + '\n')
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})

2126
scripts/qa-record.ts Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,135 @@
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<style>
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
*{margin:0;padding:0;box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
.container{max-width:1200px;margin:0 auto}
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
.header-icon svg{color:var(--primary)}
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
.icon{font-size:1.125rem}
.links{display:flex;gap:.5rem}
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
.comp-panel{border-right:1px solid var(--border-faint)}
.comp-panel:last-child{border-right:none}
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
.report summary:hover{color:var(--fg)}
.report summary svg{flex-shrink:0;opacity:.5}
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
.report-body p{margin:.4rem 0}
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
.report-body li{margin:.25rem 0}
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
.report-body strong{color:var(--fg)}
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
.badge-img{height:20px;display:block}
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
.purpose strong{color:var(--fg);font-weight:600}
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
</style></head><body><div class=container>
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend &middot; Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
</div><script>
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[![QA Badge]('+b+')]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
document.querySelectorAll('.video-wrap video').forEach(v=>{
v.playbackRate=0.5;v.removeAttribute('autoplay');v.pause();
const c=document.createElement('div');c.className='vctrl';
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
btn('\u23EE',()=>{v.currentTime=0});
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
sep();
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===0.5)b.classList.add('active');return b});
sep();c.appendChild(time);
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
// Custom seekbar — works even without server range request support
const seekWrap=document.createElement('div');seekWrap.className='vseek';
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
let seeking=false;
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
seekBar.onchange=()=>{seeking=false};
v.closest('.video-wrap').after(seekWrap);
seekWrap.after(c);
v.ontimeupdate=()=>{
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
};
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
v.parentElement.addEventListener('keydown',e=>{
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
});
v.parentElement.setAttribute('tabindex','0');
});
</script></body></html>

253
scripts/qa-reproduce.ts Normal file
View File

@@ -0,0 +1,253 @@
#!/usr/bin/env tsx
/**
* QA Reproduce Phase — Deterministic replay of research plan with narration
*
* Takes a reproduction plan from the research phase and replays it:
* 1. Execute each action deterministically (no AI decisions)
* 2. Capture a11y snapshot before/after each action
* 3. Gemini describes what visually changed (narration for humans)
* 4. Output: narration-log.json with full evidence chain
*/
import type { Page } from '@playwright/test'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { mkdirSync, writeFileSync } from 'fs'
import type { ActionResult } from './qa-record.js'
// ── Types ──
interface ReproductionStep {
action: Record<string, unknown> & { action: string }
expectedAssertion: string
}
interface NarrationEntry {
step: number
action: string
params: Record<string, unknown>
result: ActionResult
a11yBefore: unknown
a11yAfter: unknown
assertionExpected: string
assertionPassed: boolean
assertionActual: string
geminiNarration: string
timestampMs: number
}
export interface NarrationLog {
entries: NarrationEntry[]
allAssertionsPassed: boolean
}
interface ReproduceOptions {
page: Page
plan: ReproductionStep[]
geminiApiKey: string
outputDir: string
}
// ── A11y helpers ──
interface A11yNode {
role: string
name: string
value?: string
checked?: boolean
disabled?: boolean
expanded?: boolean
children?: A11yNode[]
}
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
if (!node) return null
const sel = selector.toLowerCase()
if (
node.name?.toLowerCase().includes(sel) ||
node.role?.toLowerCase().includes(sel)
) {
return node
}
if (node.children) {
for (const child of node.children) {
const found = searchA11y(child, selector)
if (found) return found
}
}
return null
}
function summarizeA11y(node: A11yNode | null): string {
if (!node) return 'null'
const parts = [`role=${node.role}`, `name="${node.name}"`]
if (node.value !== undefined) parts.push(`value="${node.value}"`)
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
if (node.disabled) parts.push('disabled')
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
return `{${parts.join(', ')}}`
}
// ── Subtitle overlay ──
async function showSubtitle(page: Page, text: string, step: number) {
const encoded = encodeURIComponent(
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
)
await page.addScriptTag({
content: `(function(){
var id='qa-subtitle';
var el=document.getElementById(id);
if(!el){
el=document.createElement('div');
el.id=id;
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
document.body.appendChild(el);
}
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
})()`
})
}
// ── Gemini visual narration ──
async function geminiDescribe(
page: Page,
geminiApiKey: string,
focus: string
): Promise<string> {
try {
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
const genAI = new GoogleGenerativeAI(geminiApiKey)
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
const result = await model.generateContent([
{
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
},
{
inlineData: {
mimeType: 'image/jpeg',
data: screenshot.toString('base64')
}
}
])
return result.response.text().trim()
} catch (e) {
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
}
}
// ── Main reproduce function ──
export async function runReproducePhase(
opts: ReproduceOptions
): Promise<NarrationLog> {
const { page, plan, geminiApiKey, outputDir } = opts
const { executeAction } = await import('./qa-record.js')
const narrationDir = `${outputDir}/narration`
mkdirSync(narrationDir, { recursive: true })
const entries: NarrationEntry[] = []
const startMs = Date.now()
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
for (let i = 0; i < plan.length; i++) {
const step = plan[i]
const actionObj = step.action
const elapsed = Date.now() - startMs
// Show subtitle
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
// Capture a11y BEFORE
const a11yBefore = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Execute action
const result = await executeAction(
page,
actionObj as Parameters<typeof executeAction>[1],
outputDir
)
await new Promise((r) => setTimeout(r, 500))
// Capture a11y AFTER
const a11yAfter = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Check assertion
let assertionPassed = false
let assertionActual = ''
if (step.expectedAssertion) {
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
const selectorName = parts[0]
const expectedState = parts.slice(1).join(':').trim()
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
if (expectedState === 'visible' || expectedState === 'exists') {
assertionPassed = found !== null
} else if (expectedState === 'hidden' || expectedState === 'gone') {
assertionPassed = found === null
} else {
// Generic: check if the actual state contains the expected text
assertionPassed = assertionActual
.toLowerCase()
.includes(expectedState.toLowerCase())
}
console.warn(
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
)
}
// Gemini narration (visual description for humans)
const geminiNarration = await geminiDescribe(
page,
geminiApiKey,
`What changed after ${actionObj.action}?`
)
entries.push({
step: i + 1,
action: actionObj.action,
params: actionObj,
result,
a11yBefore,
a11yAfter,
assertionExpected: step.expectedAssertion,
assertionPassed,
assertionActual,
geminiNarration,
timestampMs: elapsed
})
}
// Final screenshot
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
const log: NarrationLog = {
entries,
allAssertionsPassed: entries.every((e) => e.assertionPassed)
}
writeFileSync(
`${narrationDir}/narration-log.json`,
JSON.stringify(log, null, 2)
)
console.warn(
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
)
return log
}

View File

@@ -0,0 +1,150 @@
import { describe, expect, it } from 'vitest'
import {
extractPlatformFromArtifactDirName,
pickLatestVideosByPlatform,
selectVideoCandidateByFile
} from './qa-video-review'
describe('extractPlatformFromArtifactDirName', () => {
it('extracts and normalizes known qa artifact directory names', () => {
expect(
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
).toBe('windows')
expect(
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
).toBe('macos')
expect(
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
).toBe('linux')
})
it('falls back to slugifying unknown directory names', () => {
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
'custom-platform-run'
)
})
})
describe('pickLatestVideosByPlatform', () => {
it('keeps only the latest candidate per platform', () => {
const selected = pickLatestVideosByPlatform([
{
platformName: 'windows',
videoPath: '/tmp/windows-old.mp4',
mtimeMs: 100
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
},
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
}
])
expect(selected).toEqual([
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
}
])
})
})
describe('selectVideoCandidateByFile', () => {
it('selects a single candidate by artifacts-relative path', () => {
const selected = selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-Linux-1/qa-session.mp4'
}
)
expect(selected).toEqual({
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
})
})
it('throws when basename matches multiple videos', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-session.mp4'
}
)
).toThrow('matched 2 videos')
})
it('throws when there is no matching video', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-macOS-1/qa-session.mp4'
}
)
).toThrow('No video matched')
})
it('throws when video file is missing', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: ' '
}
)
).toThrow('--video-file is required')
})
})

765
scripts/qa-video-review.ts Normal file
View File

@@ -0,0 +1,765 @@
#!/usr/bin/env tsx
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
import { basename, dirname, extname, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { globSync } from 'glob'
interface CliOptions {
artifactsDir: string
videoFile: string
beforeVideo: string
outputDir: string
model: string
requestTimeoutMs: number
dryRun: boolean
prContext: string
targetUrl: string
passLabel: string
}
interface VideoCandidate {
platformName: string
videoPath: string
mtimeMs: number
}
const DEFAULT_OPTIONS: CliOptions = {
artifactsDir: './tmp/qa-artifacts',
videoFile: '',
beforeVideo: '',
outputDir: './tmp',
model: 'gemini-3-flash-preview',
requestTimeoutMs: 300_000,
dryRun: false,
prContext: '',
targetUrl: '',
passLabel: ''
}
const USAGE = `Usage:
pnpm exec tsx scripts/qa-video-review.ts [options]
Options:
--artifacts-dir <path> Artifacts root directory
(default: ./tmp/qa-artifacts)
--video-file <name-or-path> Video file to analyze (required)
(supports basename or relative/absolute path)
--before-video <path> Before video (main branch) for comparison
When provided, sends both videos to Gemini
for comparative before/after analysis
--output-dir <path> Output directory for markdown reports
(default: ./tmp)
--model <name> Gemini model
(default: gemini-3-flash-preview)
--request-timeout-ms <n> Request timeout in milliseconds
(default: 300000)
--pr-context <file> File with PR context (title, body, diff)
for PR-aware review
--target-url <url> Issue or PR URL to include in the report
--pass-label <label> Label for multi-pass reports (e.g. pass1)
Output becomes {platform}-{label}-qa-video-report.md
--dry-run Discover videos and output targets only
--help Show this help text
Environment:
GEMINI_API_KEY Required unless --dry-run
`
function parsePositiveInteger(rawValue: string, flagName: string): number {
const parsedValue = Number.parseInt(rawValue, 10)
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
}
return parsedValue
}
function parseCliOptions(args: string[]): CliOptions {
const options: CliOptions = { ...DEFAULT_OPTIONS }
for (let index = 0; index < args.length; index += 1) {
const argument = args[index]
const nextValue = args[index + 1]
const requireValue = (flagName: string): string => {
if (!nextValue || nextValue.startsWith('--')) {
throw new Error(`Missing value for ${flagName}`)
}
index += 1
return nextValue
}
if (argument === '--help') {
process.stdout.write(USAGE)
process.exit(0)
}
if (argument === '--artifacts-dir') {
options.artifactsDir = requireValue(argument)
continue
}
if (argument === '--video-file') {
options.videoFile = requireValue(argument)
continue
}
if (argument === '--output-dir') {
options.outputDir = requireValue(argument)
continue
}
if (argument === '--model') {
options.model = requireValue(argument)
continue
}
if (argument === '--request-timeout-ms') {
options.requestTimeoutMs = parsePositiveInteger(
requireValue(argument),
argument
)
continue
}
if (argument === '--before-video') {
options.beforeVideo = requireValue(argument)
continue
}
if (argument === '--pr-context') {
options.prContext = requireValue(argument)
continue
}
if (argument === '--target-url') {
options.targetUrl = requireValue(argument)
continue
}
if (argument === '--pass-label') {
options.passLabel = requireValue(argument)
continue
}
if (argument === '--dry-run') {
options.dryRun = true
continue
}
throw new Error(`Unknown argument: ${argument}`)
}
return options
}
function normalizePlatformName(value: string): string {
const slug = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
return slug.length > 0 ? slug : 'unknown-platform'
}
export function extractPlatformFromArtifactDirName(dirName: string): string {
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
return normalizePlatformName(matchedValue ?? dirName)
}
function extractPlatformFromVideoPath(videoPath: string): string {
const artifactDirName = basename(dirname(videoPath))
return extractPlatformFromArtifactDirName(artifactDirName)
}
export function pickLatestVideosByPlatform(
candidates: VideoCandidate[]
): VideoCandidate[] {
const latestByPlatform = new Map<string, VideoCandidate>()
for (const candidate of candidates) {
const current = latestByPlatform.get(candidate.platformName)
if (!current || candidate.mtimeMs > current.mtimeMs) {
latestByPlatform.set(candidate.platformName, candidate)
}
}
return [...latestByPlatform.values()].sort((a, b) =>
a.platformName.localeCompare(b.platformName)
)
}
function toProjectRelativePath(targetPath: string): string {
const relativePath = relative(process.cwd(), targetPath)
if (relativePath.startsWith('.')) {
return relativePath
}
return `./${relativePath}`
}
function errorToString(error: unknown): string {
return error instanceof Error ? error.message : String(error)
}
function normalizePathForMatch(value: string): string {
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
}
export function selectVideoCandidateByFile(
candidates: VideoCandidate[],
options: { artifactsDir: string; videoFile: string }
): VideoCandidate {
const requestedValue = options.videoFile.trim()
if (requestedValue.length === 0) {
throw new Error('--video-file is required')
}
const artifactsRoot = resolve(options.artifactsDir)
const requestedAbsolutePath = resolve(requestedValue)
const requestedPathKey = normalizePathForMatch(requestedValue)
const matches = candidates.filter((candidate) => {
const candidateAbsolutePath = resolve(candidate.videoPath)
if (candidateAbsolutePath === requestedAbsolutePath) {
return true
}
const candidateBaseName = basename(candidate.videoPath)
if (candidateBaseName === requestedValue) {
return true
}
const relativeToCwd = normalizePathForMatch(
relative(process.cwd(), candidateAbsolutePath)
)
if (relativeToCwd === requestedPathKey) {
return true
}
const relativeToArtifacts = normalizePathForMatch(
relative(artifactsRoot, candidateAbsolutePath)
)
return relativeToArtifacts === requestedPathKey
})
if (matches.length === 1) {
return matches[0]
}
if (matches.length === 0) {
const availableVideos = candidates.map((candidate) =>
toProjectRelativePath(candidate.videoPath)
)
throw new Error(
[
`No video matched --video-file "${options.videoFile}".`,
'Available videos:',
...availableVideos.map((videoPath) => `- ${videoPath}`)
].join('\n')
)
}
throw new Error(
[
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
'Please pass a more specific path.',
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
].join('\n')
)
}
async function collectVideoCandidates(
artifactsDir: string
): Promise<VideoCandidate[]> {
const absoluteArtifactsDir = resolve(artifactsDir)
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
cwd: absoluteArtifactsDir,
absolute: true,
nodir: true
}).sort()
const candidates = await Promise.all(
videoPaths.map(async (videoPath) => {
const videoStat = await stat(videoPath)
return {
platformName: extractPlatformFromVideoPath(videoPath),
videoPath,
mtimeMs: videoStat.mtimeMs
}
})
)
return candidates
}
function getMimeType(filePath: string): string {
const ext = extname(filePath).toLowerCase()
const mimeMap: Record<string, string> = {
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime',
'.avi': 'video/x-msvideo',
'.mkv': 'video/x-matroska',
'.m4v': 'video/mp4'
}
return mimeMap[ext] || 'video/mp4'
}
function buildReviewPrompt(options: {
platformName: string
videoPath: string
prContext: string
isComparative: boolean
}): string {
const { platformName, videoPath, prContext, isComparative } = options
if (isComparative) {
return buildComparativePrompt(platformName, videoPath, prContext)
}
return buildSingleVideoPrompt(platformName, videoPath, prContext)
}
function buildComparativePrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
'',
'You are given TWO videos:',
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
'',
'Both videos show the same test steps executed on different code versions.',
''
]
if (prContext) {
lines.push('## PR Context', prContext, '')
}
lines.push(
'## Your Task',
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
'',
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
' Describe what you observe — this establishes the baseline.',
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
'3. **Comparison**: What specifically changed between before and after?',
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
' that were NOT present in the BEFORE video?',
'',
'Note: Brief black frames during page transitions are NORMAL.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible differences. Avoid speculation.',
'',
'Return markdown with these sections exactly:',
'## Summary',
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
'',
'## Behavior Changes',
'Summarize ALL behavioral differences as a markdown TABLE:',
'| Behavior | Before (main) | After (PR) | Verdict |',
'',
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
'- **Before (main)**: how it works/looks in the BEFORE video',
'- **After (PR)**: how it works/looks in the AFTER video',
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
'',
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
'that were tested, so reviewers can confirm nothing was missed.',
'',
'## Timeline Comparison',
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
'| Time | Type | Severity | Before (main) | After (PR) |',
'',
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
'- **Before (main)**: what is observed in the BEFORE video at that time',
'- **After (PR)**: what is observed in the AFTER video at that time',
'',
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
'omit that row. Focus on meaningful differences, not narrating every frame.',
'',
'## Confirmed Issues',
'For each issue, use this exact format:',
'',
'### [Short issue title]',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'',
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
'',
'**Evidence:** [What you observed at the given timestamp in which video]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'(Assess whether the PR achieves its goal based on the before/after comparison)',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
)
return lines.filter(Boolean).join('\n')
}
function buildSingleVideoPrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer reviewing a UI test session recording.',
'',
'## ANTI-HALLUCINATION RULES (READ FIRST)',
'- Describe ONLY what you can directly observe in the video frames',
'- NEVER infer or assume what "must have happened" between frames',
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
'- Your job is to be a CAMERA — report facts, not interpretations',
''
]
const isIssueContext =
prContext &&
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
prContext
)
if (prContext) {
lines.push(
'## Phase 1: Blind Observation (describe what you SEE)',
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
'- What elements does the user click/hover/type?',
'- What dialogs/menus open and close?',
'- What keyboard indicators appear? (look for subtitle overlays)',
'- What is the BEFORE state and AFTER state of each action?',
'',
'## Phase 2: Compare against expected behavior',
'Now compare your observations against the context below.',
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
''
)
if (isIssueContext) {
lines.push(
'## Issue Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video perform the reproduction steps described in the issue?',
'2. Did your Phase 1 observations show the reported bug behavior?',
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
''
)
} else {
lines.push(
'## PR Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video test the specific behavior the PR changes?',
'2. Did your Phase 1 observations show the expected before/after difference?',
'3. If the test was incomplete or inconclusive, say so honestly.',
''
)
}
}
lines.push(
`Review this QA session video for platform "${platformName}".`,
`Source video: ${toProjectRelativePath(videoPath)}.`,
'The video shows the full test session — analyze it chronologically.',
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible problems and avoid speculation.',
'If confidence is low, mark it explicitly.',
'',
'Return markdown with these sections exactly:',
'## Summary',
isIssueContext
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
: prContext
? '(Explain what the PR intended and whether the video confirms it works)'
: '',
'## Confirmed Issues',
'For each confirmed issue, use this exact format (one block per issue):',
'',
'### [Short issue title]',
'`HIGH` `01:03` `Confidence: High`',
'',
'[Description of the issue — what went wrong and what was expected]',
'',
'**Evidence:** [What you observed in the video at the given timestamp]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'Do NOT use a table for issues — use the block format above.',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the bug/behavior is clearly visible in the video',
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient'
)
return lines.filter(Boolean).join('\n')
}
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
async function readVideoFile(videoPath: string): Promise<Buffer> {
const fileStat = await stat(videoPath)
if (fileStat.size > MAX_VIDEO_BYTES) {
throw new Error(
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
)
}
return readFile(videoPath)
}
async function requestGeminiReview(options: {
apiKey: string
model: string
platformName: string
videoPath: string
beforeVideoPath: string
timeoutMs: number
prContext: string
}): Promise<string> {
const genAI = new GoogleGenerativeAI(options.apiKey)
const model = genAI.getGenerativeModel({ model: options.model })
const isComparative = options.beforeVideoPath.length > 0
const prompt = buildReviewPrompt({
platformName: options.platformName,
videoPath: options.videoPath,
prContext: options.prContext,
isComparative
})
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
if (isComparative) {
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
parts.push(
{ text: 'Video 1 — BEFORE (main branch):' },
{
inlineData: {
mimeType: getMimeType(options.beforeVideoPath),
data: beforeBuffer.toString('base64')
}
}
)
}
const afterBuffer = await readVideoFile(options.videoPath)
if (isComparative) {
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
}
parts.push({
inlineData: {
mimeType: getMimeType(options.videoPath),
data: afterBuffer.toString('base64')
}
})
const result = await model.generateContent(parts, {
timeout: options.timeoutMs
})
const response = result.response
const text = response.text()
if (!text || text.trim().length === 0) {
throw new Error('Gemini API returned no output text')
}
return text.trim()
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
function buildReportMarkdown(input: {
platformName: string
model: string
videoPath: string
videoSizeBytes: number
beforeVideoPath?: string
beforeVideoSizeBytes?: number
reviewText: string
targetUrl?: string
}): string {
const headerLines = [
`# ${input.platformName} QA Video Report`,
'',
`- Generated at: ${new Date().toISOString()}`,
`- Model: \`${input.model}\``
]
if (input.targetUrl) {
headerLines.push(`- Target: ${input.targetUrl}`)
}
if (input.beforeVideoPath) {
headerLines.push(
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
'- Mode: **Comparative (before/after)**'
)
} else {
headerLines.push(
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
`- Video size: ${formatBytes(input.videoSizeBytes)}`
)
}
headerLines.push('', '## AI Review', '')
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
}
async function reviewVideo(
video: VideoCandidate,
options: CliOptions,
apiKey: string
): Promise<void> {
let prContext = ''
if (options.prContext) {
try {
prContext = await readFile(options.prContext, 'utf-8')
process.stdout.write(
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
)
} catch {
process.stdout.write(
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
)
}
}
const beforeVideoPath = options.beforeVideo
? resolve(options.beforeVideo)
: ''
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
process.stdout.write(
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
)
}
process.stdout.write(
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
)
const reviewText = await requestGeminiReview({
apiKey,
model: options.model,
platformName: video.platformName,
videoPath: video.videoPath,
beforeVideoPath,
timeoutMs: options.requestTimeoutMs,
prContext
})
const videoStat = await stat(video.videoPath)
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
const outputPath = resolve(
options.outputDir,
`${video.platformName}${passSegment}-qa-video-report.md`
)
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
platformName: video.platformName,
model: options.model,
videoPath: video.videoPath,
videoSizeBytes: videoStat.size,
reviewText,
targetUrl: options.targetUrl || undefined
}
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
reportInput.beforeVideoPath = beforeVideoPath
reportInput.beforeVideoSizeBytes = beforeStat.size
}
const reportMarkdown = buildReportMarkdown(reportInput)
await mkdir(dirname(outputPath), { recursive: true })
await writeFile(outputPath, reportMarkdown, 'utf-8')
process.stdout.write(
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
async function main(): Promise<void> {
const options = parseCliOptions(process.argv.slice(2))
const candidates = await collectVideoCandidates(options.artifactsDir)
if (candidates.length === 0) {
process.stdout.write(
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
)
return
}
const selectedVideo = selectVideoCandidateByFile(candidates, {
artifactsDir: options.artifactsDir,
videoFile: options.videoFile
})
process.stdout.write(
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
)
if (options.dryRun) {
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
return
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
}
await reviewVideo(selectedVideo, options, apiKey)
}
if (isExecutedAsScript(import.meta.url)) {
void main().catch((error: unknown) => {
const message = errorToString(error)
process.stderr.write(`qa-video-review failed: ${message}\n`)
process.exit(1)
})
}

View File

@@ -9,13 +9,10 @@ import { captureException } from '@sentry/vue'
import BlockUI from 'primevue/blockui'
import { computed, onMounted, onUnmounted, watch } from 'vue'
import { useI18n } from 'vue-i18n'
import GlobalDialog from '@/components/dialog/GlobalDialog.vue'
import config from '@/config'
import { isDesktop } from '@/platform/distribution/types'
import { useSettingStore } from '@/platform/settings/settingStore'
import { useToastStore } from '@/platform/updates/common/toastStore'
import { app } from '@/scripts/app'
import { useWorkspaceStore } from '@/stores/workspaceStore'
import { electronAPI } from '@/utils/envUtil'
@@ -23,7 +20,6 @@ import { parsePreloadError } from '@/utils/preloadErrorUtil'
import { useDialogService } from '@/services/dialogService'
import { useConflictDetection } from '@/workbench/extensions/manager/composables/useConflictDetection'
const { t } = useI18n()
const workspaceStore = useWorkspaceStore()
app.extensionManager = useWorkspaceStore()
@@ -98,12 +94,17 @@ onMounted(() => {
}
})
}
useToastStore().add({
severity: 'error',
summary: t('g.preloadErrorTitle'),
detail: t('g.preloadError'),
life: 10000
})
// Disabled: Third-party custom node extensions frequently trigger this toast
// (e.g., bare "vue" imports, wrong relative paths to scripts/app.js, missing
// core dependencies). These are plugin bugs, not ComfyUI core failures, but
// the generic error message alarms users and offers no actionable guidance.
// The console.error above still logs the details for developers to debug.
// useToastStore().add({
// severity: 'error',
// summary: t('g.preloadErrorTitle'),
// detail: t('g.preloadError'),
// life: 10000
// })
})
// Capture resource load failures (CSS, scripts) in non-localhost distributions

View File

@@ -34,9 +34,7 @@
"CLEAR_BACKGROUND_COLOR": "#2b2f38",
"NODE_TITLE_COLOR": "#b2b7bd",
"NODE_SELECTED_TITLE_COLOR": "#FFF",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#AAA",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#2b2f38",
"NODE_DEFAULT_BGCOLOR": "#242730",
"NODE_DEFAULT_BOXCOLOR": "#6e7581",
@@ -45,7 +43,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.5)",
"DEFAULT_GROUP_FONT": 22,
"WIDGET_BGCOLOR": "#2b2f38",
"WIDGET_OUTLINE_COLOR": "#6e7581",
"WIDGET_TEXT_COLOR": "#DDD",

View File

@@ -25,10 +25,8 @@
"CLEAR_BACKGROUND_COLOR": "#141414",
"NODE_TITLE_COLOR": "#999",
"NODE_SELECTED_TITLE_COLOR": "#FFF",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#AAA",
"NODE_TEXT_HIGHLIGHT_COLOR": "#FFF",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#333",
"NODE_DEFAULT_BGCOLOR": "#353535",
"NODE_DEFAULT_BOXCOLOR": "#666",
@@ -37,7 +35,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.5)",
"DEFAULT_GROUP_FONT": 24,
"WIDGET_BGCOLOR": "#222",
"WIDGET_OUTLINE_COLOR": "#666",
"WIDGET_TEXT_COLOR": "#DDD",

View File

@@ -34,9 +34,7 @@
"CLEAR_BACKGROUND_COLOR": "#040506",
"NODE_TITLE_COLOR": "#999",
"NODE_SELECTED_TITLE_COLOR": "#e5eaf0",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#bcc2c8",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#161b22",
"NODE_DEFAULT_BGCOLOR": "#13171d",
"NODE_DEFAULT_BOXCOLOR": "#30363d",
@@ -45,7 +43,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.5)",
"DEFAULT_GROUP_FONT": 24,
"WIDGET_BGCOLOR": "#161b22",
"WIDGET_OUTLINE_COLOR": "#30363d",
"WIDGET_TEXT_COLOR": "#bcc2c8",

View File

@@ -26,10 +26,8 @@
"CLEAR_BACKGROUND_COLOR": "lightgray",
"NODE_TITLE_COLOR": "#222",
"NODE_SELECTED_TITLE_COLOR": "#000",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#444",
"NODE_TEXT_HIGHLIGHT_COLOR": "#1e293b",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#F7F7F7",
"NODE_DEFAULT_BGCOLOR": "#F5F5F5",
"NODE_DEFAULT_BOXCOLOR": "#CCC",
@@ -38,7 +36,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.1)",
"DEFAULT_GROUP_FONT": 24,
"WIDGET_BGCOLOR": "#D4D4D4",
"WIDGET_OUTLINE_COLOR": "#999",
"WIDGET_TEXT_COLOR": "#222",

View File

@@ -34,9 +34,7 @@
"CLEAR_BACKGROUND_COLOR": "#212732",
"NODE_TITLE_COLOR": "#999",
"NODE_SELECTED_TITLE_COLOR": "#e5eaf0",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#bcc2c8",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#2e3440",
"NODE_DEFAULT_BGCOLOR": "#161b22",
"NODE_DEFAULT_BOXCOLOR": "#545d70",
@@ -45,7 +43,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.5)",
"DEFAULT_GROUP_FONT": 24,
"WIDGET_BGCOLOR": "#2e3440",
"WIDGET_OUTLINE_COLOR": "#545d70",
"WIDGET_TEXT_COLOR": "#bcc2c8",

View File

@@ -19,9 +19,7 @@
"litegraph_base": {
"NODE_TITLE_COLOR": "#fdf6e3",
"NODE_SELECTED_TITLE_COLOR": "#A9D400",
"NODE_TEXT_SIZE": 14,
"NODE_TEXT_COLOR": "#657b83",
"NODE_SUBTEXT_SIZE": 12,
"NODE_DEFAULT_COLOR": "#094656",
"NODE_DEFAULT_BGCOLOR": "#073642",
"NODE_DEFAULT_BOXCOLOR": "#839496",
@@ -30,7 +28,6 @@
"NODE_BYPASS_BGCOLOR": "#FF00FF",
"NODE_ERROR_COLOUR": "#E00",
"DEFAULT_SHADOW_COLOR": "rgba(0,0,0,0.5)",
"DEFAULT_GROUP_FONT": 24,
"WIDGET_BGCOLOR": "#002b36",
"WIDGET_OUTLINE_COLOR": "#839496",
"WIDGET_TEXT_COLOR": "#fdf6e3",

View File

@@ -42,7 +42,6 @@ import type { StyleValue } from 'vue'
import { useIntersectionObserver } from '@/composables/useIntersectionObserver'
import { useMediaCache } from '@/services/mediaCacheService'
import type { ClassValue } from '@/utils/tailwindUtil'
const {
src,
@@ -54,8 +53,8 @@ const {
} = defineProps<{
src: string
alt?: string
containerClass?: ClassValue
imageClass?: ClassValue
containerClass?: string
imageClass?: string
imageStyle?: StyleValue
rootMargin?: string
}>()

View File

@@ -186,13 +186,13 @@ const toggleState = () => {
}
const signInWithGoogle = async () => {
if (await authActions.signInWithGoogle()) {
if (await authActions.signInWithGoogle({ isNewUser: !isSignIn.value })) {
onSuccess()
}
}
const signInWithGithub = async () => {
if (await authActions.signInWithGithub()) {
if (await authActions.signInWithGithub({ isNewUser: !isSignIn.value })) {
onSuccess()
}
}

View File

@@ -19,10 +19,7 @@ const props = defineProps<{
const queryString = computed(() => props.errorMessage + ' is:issue')
/**
* Open GitHub issues search and track telemetry.
*/
const openGitHubIssues = () => {
function openGitHubIssues() {
useTelemetry()?.trackUiButtonClicked({
button_id: 'error_dialog_find_existing_issues_clicked'
})

View File

@@ -49,7 +49,12 @@
<Button variant="muted-textonly" size="unset" @click="dismiss">
{{ t('g.dismiss') }}
</Button>
<Button variant="secondary" size="lg" @click="seeErrors">
<Button
variant="secondary"
size="lg"
data-testid="error-overlay-see-errors"
@click="seeErrors"
>
{{
appMode ? t('linearMode.error.goto') : t('errorOverlay.seeErrors')
}}

View File

@@ -195,6 +195,7 @@ import { forEachNode } from '@/utils/graphTraversalUtil'
import SelectionRectangle from './SelectionRectangle.vue'
import { isCloud } from '@/platform/distribution/types'
import { useFeatureFlags } from '@/composables/useFeatureFlags'
import { useCreateWorkspaceUrlLoader } from '@/platform/workspace/composables/useCreateWorkspaceUrlLoader'
import { useInviteUrlLoader } from '@/platform/workspace/composables/useInviteUrlLoader'
const { t } = useI18n()
@@ -455,8 +456,9 @@ useEventListener(
const comfyAppReady = ref(false)
const workflowPersistence = useWorkflowPersistence()
const { flags } = useFeatureFlags()
// Set up invite loader during setup phase so useRoute/useRouter work correctly
// Set up URL loaders during setup phase so useRoute/useRouter work correctly
const inviteUrlLoader = isCloud ? useInviteUrlLoader() : null
const createWorkspaceUrlLoader = isCloud ? useCreateWorkspaceUrlLoader() : null
useCanvasDrop(canvasRef)
useLitegraphSettings()
useNodeBadge()
@@ -576,6 +578,18 @@ onMounted(async () => {
await inviteUrlLoader.loadInviteFromUrl()
}
// Open create workspace dialog from URL if present (e.g., ?create_workspace=1)
if (createWorkspaceUrlLoader && flags.teamWorkspacesEnabled) {
try {
await createWorkspaceUrlLoader.loadCreateWorkspaceFromUrl()
} catch (error) {
console.error(
'[GraphCanvas] Failed to load create workspace from URL:',
error
)
}
}
// Initialize release store to fetch releases from comfy-api (fire-and-forget)
const { useReleaseStore } =
await import('@/platform/updates/common/releaseStore')

View File

@@ -84,7 +84,9 @@ watch(
pos: group.pos,
size: [group.size[0], group.titleHeight]
})
inputFontStyle.value = { fontSize: `${group.font_size * scale}px` }
inputFontStyle.value = {
fontSize: `${LiteGraph.GROUP_TEXT_SIZE * scale}px`
}
} else if (target instanceof LGraphNode) {
const node = target
const [x, y] = node.getBounding()

View File

@@ -0,0 +1,60 @@
import { mount } from '@vue/test-utils'
import { describe, expect, it } from 'vitest'
import { nextTick } from 'vue'
import { createI18n } from 'vue-i18n'
import MultiSelect from './MultiSelect.vue'
const i18n = createI18n({
legacy: false,
locale: 'en',
messages: {
en: {
g: {
multiSelectDropdown: 'Multi-select dropdown',
noResultsFound: 'No results found',
search: 'Search',
clearAll: 'Clear all',
itemsSelected: 'Items selected'
}
}
}
})
describe('MultiSelect', () => {
function createWrapper() {
return mount(MultiSelect, {
attachTo: document.body,
global: {
plugins: [i18n]
},
props: {
modelValue: [],
label: 'Category',
options: [
{ name: 'One', value: 'one' },
{ name: 'Two', value: 'two' }
]
}
})
}
it('keeps open-state border styling available while the dropdown is open', async () => {
const wrapper = createWrapper()
const trigger = wrapper.get('button[aria-haspopup="listbox"]')
expect(trigger.classes()).toContain(
'data-[state=open]:border-node-component-border'
)
expect(trigger.attributes('aria-expanded')).toBe('false')
await trigger.trigger('click')
await nextTick()
expect(trigger.attributes('aria-expanded')).toBe('true')
expect(trigger.attributes('data-state')).toBe('open')
wrapper.unmount()
})
})

View File

@@ -1,207 +1,215 @@
<template>
<!--
Note: Unlike SingleSelect, we don't need an explicit options prop because:
1. Our value template only shows a static label (not dynamic based on selection)
2. We display a count badge instead of actual selected labels
3. All PrimeVue props (including options) are passed via v-bind="$attrs"
option-label="name" is required because our option template directly accesses option.name
max-selected-labels="0" is required to show count badge instead of selected item labels
-->
<MultiSelect
<ComboboxRoot
v-model="selectedItems"
v-bind="{ ...$attrs, options: filteredOptions }"
option-label="name"
unstyled
:max-selected-labels="0"
:pt="{
root: ({ props }: MultiSelectPassThroughMethodOptions) => ({
class: cn(
'relative inline-flex cursor-pointer select-none',
size === 'md' ? 'h-8' : 'h-10',
'rounded-lg bg-secondary-background text-base-foreground',
'transition-all duration-200 ease-in-out',
'hover:bg-secondary-background-hover',
'border-[2.5px] border-solid',
selectedCount > 0 ? 'border-base-foreground' : 'border-transparent',
'focus-within:border-base-foreground',
props.disabled &&
'cursor-default opacity-30 hover:bg-secondary-background'
)
}),
labelContainer: {
class: cn(
'flex flex-1 items-center overflow-hidden py-2 whitespace-nowrap',
size === 'md' ? 'pl-3' : 'pl-4'
)
},
label: {
class: 'p-0'
},
dropdown: {
class: 'flex shrink-0 cursor-pointer items-center justify-center px-3'
},
header: () => ({
class:
showSearchBox || showSelectedCount || showClearButton
? 'block'
: 'hidden'
}),
// Overlay & list visuals unchanged
overlay: {
class: cn(
'mt-2 rounded-lg p-2',
'bg-base-background',
'text-base-foreground',
'border border-solid border-border-default'
)
},
listContainer: () => ({
style: { maxHeight: `min(${listMaxHeight}, 50vh)` },
class: 'scrollbar-custom'
}),
list: {
class: 'flex flex-col gap-0 p-0 m-0 list-none border-none text-sm'
},
// Option row hover and focus tone
option: ({ context }: MultiSelectPassThroughMethodOptions) => ({
class: cn(
'flex h-10 cursor-pointer items-center gap-2 rounded-lg px-2',
'hover:bg-secondary-background-hover',
// Add focus/highlight state for keyboard navigation
context?.focused &&
'bg-secondary-background-selected hover:bg-secondary-background-selected'
)
}),
// Hide built-in checkboxes entirely via PT (no :deep)
pcHeaderCheckbox: {
root: { class: 'hidden' },
style: { display: 'none' }
},
pcOptionCheckbox: {
root: { class: 'hidden' },
style: { display: 'none' }
},
emptyMessage: {
class: 'px-3 pb-4 text-sm text-muted-foreground'
}
}"
:aria-label="label || t('g.multiSelectDropdown')"
role="combobox"
:aria-expanded="false"
aria-haspopup="listbox"
:tabindex="0"
multiple
by="value"
:disabled
ignore-filter
:reset-search-term-on-select="false"
>
<template
v-if="showSearchBox || showSelectedCount || showClearButton"
#header
>
<div class="flex flex-col px-2 pt-2 pb-0">
<SearchInput
v-if="showSearchBox"
v-model="searchQuery"
:class="showSelectedCount || showClearButton ? 'mb-2' : ''"
:placeholder="searchPlaceholder"
size="sm"
/>
<div
v-if="showSelectedCount || showClearButton"
class="mt-2 flex items-center justify-between"
>
<span
v-if="showSelectedCount"
class="px-1 text-sm text-base-foreground"
>
{{
selectedCount > 0
? $t('g.itemsSelected', { selectedCount })
: $t('g.itemSelected', { selectedCount })
}}
</span>
<Button
v-if="showClearButton"
variant="textonly"
size="md"
@click.stop="selectedItems = []"
>
{{ $t('g.clearAll') }}
</Button>
</div>
<div class="my-4 h-px bg-border-default"></div>
</div>
</template>
<!-- Trigger value (keep text scale identical) -->
<template #value>
<span :class="size === 'md' ? 'text-xs' : 'text-sm'">
{{ label }}
</span>
<span
v-if="selectedCount > 0"
class="pointer-events-none absolute -top-2 -right-2 z-10 flex size-5 items-center justify-center rounded-full bg-base-foreground text-xs font-semibold text-base-background"
>
{{ selectedCount }}
</span>
</template>
<!-- Chevron size identical to current -->
<template #dropdownicon>
<i class="icon-[lucide--chevron-down] text-muted-foreground" />
</template>
<!-- Custom option row: square checkbox + label (unchanged layout/colors) -->
<template #option="slotProps">
<div
role="button"
class="flex cursor-pointer items-center gap-2"
:style="popoverStyle"
<ComboboxAnchor as-child>
<ComboboxTrigger
v-bind="$attrs"
:aria-label="label || t('g.multiSelectDropdown')"
:class="
cn(
'relative inline-flex cursor-pointer items-center select-none',
size === 'md' ? 'h-8' : 'h-10',
'rounded-lg bg-secondary-background text-base-foreground',
'transition-all duration-200 ease-in-out',
'hover:bg-secondary-background-hover',
'border-[2.5px] border-solid border-transparent',
selectedCount > 0
? 'border-base-foreground'
: 'focus-visible:border-node-component-border data-[state=open]:border-node-component-border',
disabled &&
'cursor-default opacity-30 hover:bg-secondary-background'
)
"
>
<div
class="flex size-4 shrink-0 items-center justify-center rounded-sm p-0.5 transition-all duration-200"
:class="
slotProps.selected
? 'bg-primary-background'
: 'bg-secondary-background'
cn(
'flex flex-1 items-center overflow-hidden py-2 whitespace-nowrap',
size === 'md' ? 'pl-3' : 'pl-4'
)
"
>
<i
v-if="slotProps.selected"
class="text-bold icon-[lucide--check] text-xs text-base-foreground"
/>
<span :class="size === 'md' ? 'text-xs' : 'text-sm'">
{{ label }}
</span>
<span
v-if="selectedCount > 0"
class="pointer-events-none absolute -top-2 -right-2 z-10 flex size-5 items-center justify-center rounded-full bg-base-foreground text-xs font-semibold text-base-background"
>
{{ selectedCount }}
</span>
</div>
<span>
{{ slotProps.option.name }}
</span>
</div>
</template>
</MultiSelect>
<div
class="flex shrink-0 cursor-pointer items-center justify-center px-3"
>
<i class="icon-[lucide--chevron-down] text-muted-foreground" />
</div>
</ComboboxTrigger>
</ComboboxAnchor>
<ComboboxPortal>
<ComboboxContent
position="popper"
:side-offset="8"
align="start"
:style="popoverStyle"
:class="
cn(
'z-3000 overflow-hidden',
'rounded-lg p-2',
'bg-base-background text-base-foreground',
'border border-solid border-border-default',
'shadow-md',
'data-[state=closed]:animate-out data-[state=open]:animate-in',
'data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
'data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95',
'data-[side=bottom]:slide-in-from-top-2'
)
"
@focus-outside="preventFocusDismiss"
>
<div
v-if="showSearchBox || showSelectedCount || showClearButton"
class="flex flex-col px-2 pt-2 pb-0"
>
<div
v-if="showSearchBox"
:class="
cn(
'flex items-center gap-2 rounded-lg border border-solid border-border-default px-3 py-1.5',
(showSelectedCount || showClearButton) && 'mb-2'
)
"
>
<i
class="icon-[lucide--search] shrink-0 text-sm text-muted-foreground"
/>
<ComboboxInput
v-model="searchQuery"
:placeholder="searchPlaceholder ?? t('g.search')"
class="w-full border-none bg-transparent text-sm outline-none"
/>
</div>
<div
v-if="showSelectedCount || showClearButton"
class="mt-2 flex items-center justify-between"
>
<span
v-if="showSelectedCount"
class="px-1 text-sm text-base-foreground"
>
{{ $t('g.itemsSelected', { count: selectedCount }) }}
</span>
<Button
v-if="showClearButton"
variant="textonly"
size="md"
@click.stop="selectedItems = []"
>
{{ $t('g.clearAll') }}
</Button>
</div>
<div class="my-4 h-px bg-border-default" />
</div>
<ComboboxViewport
:class="
cn(
'flex flex-col gap-0 p-0 text-sm',
'scrollbar-custom overflow-y-auto',
'min-w-(--reka-combobox-trigger-width)'
)
"
:style="{ maxHeight: `min(${listMaxHeight}, 50vh)` }"
>
<ComboboxItem
v-for="opt in filteredOptions"
:key="opt.value"
:value="opt"
:class="
cn(
'group flex h-10 shrink-0 cursor-pointer items-center gap-2 rounded-lg px-2 outline-none',
'hover:bg-secondary-background-hover',
'data-highlighted:bg-secondary-background-selected data-highlighted:hover:bg-secondary-background-selected'
)
"
>
<div
class="flex size-4 shrink-0 items-center justify-center rounded-sm transition-all duration-200 group-data-[state=checked]:bg-primary-background group-data-[state=unchecked]:bg-secondary-background [&>span]:flex"
>
<ComboboxItemIndicator>
<i
class="icon-[lucide--check] text-xs font-bold text-base-foreground"
/>
</ComboboxItemIndicator>
</div>
<span>{{ opt.name }}</span>
</ComboboxItem>
<ComboboxEmpty class="px-3 pb-4 text-sm text-muted-foreground">
{{ $t('g.noResultsFound') }}
</ComboboxEmpty>
</ComboboxViewport>
</ComboboxContent>
</ComboboxPortal>
</ComboboxRoot>
</template>
<script setup lang="ts">
import { useFuse } from '@vueuse/integrations/useFuse'
import type { UseFuseOptions } from '@vueuse/integrations/useFuse'
import type { MultiSelectPassThroughMethodOptions } from 'primevue/multiselect'
import MultiSelect from 'primevue/multiselect'
import { computed, useAttrs } from 'vue'
import type { FocusOutsideEvent } from 'reka-ui'
import {
ComboboxAnchor,
ComboboxContent,
ComboboxEmpty,
ComboboxInput,
ComboboxItem,
ComboboxItemIndicator,
ComboboxPortal,
ComboboxRoot,
ComboboxTrigger,
ComboboxViewport
} from 'reka-ui'
import { computed } from 'vue'
import { useI18n } from 'vue-i18n'
import SearchInput from '@/components/ui/search-input/SearchInput.vue'
import Button from '@/components/ui/button/Button.vue'
import { usePopoverSizing } from '@/composables/usePopoverSizing'
import { cn } from '@/utils/tailwindUtil'
import type { SelectOption } from './types'
type Option = SelectOption
defineOptions({
inheritAttrs: false
})
interface Props {
const {
label,
options = [],
size = 'lg',
disabled = false,
showSearchBox = false,
showSelectedCount = false,
showClearButton = false,
searchPlaceholder,
listMaxHeight = '28rem',
popoverMinWidth,
popoverMaxWidth
} = defineProps<{
/** Input label shown on the trigger button */
label?: string
/** Available options */
options?: SelectOption[]
/** Trigger size: 'lg' (40px, Interface) or 'md' (32px, Node) */
size?: 'lg' | 'md'
/** Disable the select */
disabled?: boolean
/** Show search box in the panel header */
showSearchBox?: boolean
/** Show selected count text in the panel header */
@@ -216,22 +224,9 @@ interface Props {
popoverMinWidth?: string
/** Maximum width of the popover (default: auto) */
popoverMaxWidth?: string
// Note: options prop is intentionally omitted.
// It's passed via $attrs to maximize PrimeVue API compatibility
}
const {
label,
size = 'lg',
showSearchBox = false,
showSelectedCount = false,
showClearButton = false,
searchPlaceholder = 'Search...',
listMaxHeight = '28rem',
popoverMinWidth,
popoverMaxWidth
} = defineProps<Props>()
}>()
const selectedItems = defineModel<Option[]>({
const selectedItems = defineModel<SelectOption[]>({
required: true
})
const searchQuery = defineModel<string>('searchQuery', { default: '' })
@@ -239,15 +234,16 @@ const searchQuery = defineModel<string>('searchQuery', { default: '' })
const { t } = useI18n()
const selectedCount = computed(() => selectedItems.value.length)
function preventFocusDismiss(event: FocusOutsideEvent) {
event.preventDefault()
}
const popoverStyle = usePopoverSizing({
minWidth: popoverMinWidth,
maxWidth: popoverMaxWidth
})
const attrs = useAttrs()
const originalOptions = computed(() => (attrs.options as Option[]) || [])
// Use VueUse's useFuse for better reactivity and performance
const fuseOptions: UseFuseOptions<Option> = {
const fuseOptions: UseFuseOptions<SelectOption> = {
fuseOptions: {
keys: ['name', 'value'],
threshold: 0.3,
@@ -256,23 +252,20 @@ const fuseOptions: UseFuseOptions<Option> = {
matchAllWhenSearchEmpty: true
}
const { results } = useFuse(searchQuery, originalOptions, fuseOptions)
const { results } = useFuse(searchQuery, () => options, fuseOptions)
// Filter options based on search, but always include selected items
const filteredOptions = computed(() => {
if (!searchQuery.value || searchQuery.value.trim() === '') {
return originalOptions.value
return options
}
// results.value already contains the search results from useFuse
const searchResults = results.value.map(
(result: { item: Option }) => result.item
(result: { item: SelectOption }) => result.item
)
// Include selected items that aren't in search results
const selectedButNotInResults = selectedItems.value.filter(
(item) =>
!searchResults.some((result: Option) => result.value === item.value)
!searchResults.some((result: SelectOption) => result.value === item.value)
)
return [...selectedButNotInResults, ...searchResults]

View File

@@ -1,21 +1,12 @@
<template>
<!--
Note: We explicitly pass options here (not just via $attrs) because:
1. Our custom value template needs options to look up labels from values
2. PrimeVue's value slot only provides 'value' and 'placeholder', not the selected item's label
3. We need to maintain the icon slot functionality in the value template
option-label="name" is required because our option template directly accesses option.name
-->
<Select
v-model="selectedItem"
v-bind="$attrs"
:options="options"
option-label="name"
option-value="value"
unstyled
:pt="{
root: ({ props }: SelectPassThroughMethodOptions<SelectOption>) => ({
class: cn(
<SelectRoot v-model="selectedItem" :disabled>
<SelectTrigger
v-bind="$attrs"
:aria-label="label || t('g.singleSelectDropdown')"
:aria-busy="loading || undefined"
:aria-invalid="invalid || undefined"
:class="
cn(
'relative inline-flex cursor-pointer items-center select-none',
size === 'md' ? 'h-8' : 'h-10',
'rounded-lg',
@@ -23,121 +14,107 @@
'transition-all duration-200 ease-in-out',
'hover:bg-secondary-background-hover',
'border-[2.5px] border-solid',
invalid
? 'border-destructive-background'
: 'border-transparent focus-within:border-node-component-border',
props.disabled &&
'cursor-default opacity-30 hover:bg-secondary-background'
invalid ? 'border-destructive-background' : 'border-transparent',
'focus:border-node-component-border focus:outline-none',
'disabled:cursor-default disabled:opacity-30 disabled:hover:bg-secondary-background'
)
}),
label: {
class: cn(
'flex flex-1 items-center py-2 whitespace-nowrap outline-hidden',
size === 'md' ? 'pl-3' : 'pl-4'
)
},
dropdown: {
class:
// Right chevron touch area
'flex shrink-0 items-center justify-center px-3 py-2'
},
overlay: {
class: cn(
'mt-2 rounded-lg p-2',
'bg-base-background text-base-foreground',
'border border-solid border-border-default'
)
},
listContainer: () => ({
style: `max-height: min(${listMaxHeight}, 50vh)`,
class: 'scrollbar-custom'
}),
list: {
class:
// Same list tone/size as MultiSelect
'flex flex-col gap-0 p-0 m-0 list-none border-none text-sm'
},
option: ({ context }: SelectPassThroughMethodOptions<SelectOption>) => ({
class: cn(
// Row layout
'flex items-center justify-between gap-3 rounded-sm px-2 py-3',
'hover:bg-secondary-background-hover',
// Add focus state for keyboard navigation
context.focused && 'bg-secondary-background-hover',
// Selected state + check icon
context.selected &&
'bg-secondary-background-selected hover:bg-secondary-background-selected'
)
}),
optionLabel: {
class: 'truncate'
},
optionGroupLabel: {
class: 'px-3 py-2 text-xs uppercase tracking-wide text-muted-foreground'
},
emptyMessage: {
class: 'px-3 py-2 text-sm text-muted-foreground'
}
}"
:aria-label="label || t('g.singleSelectDropdown')"
:aria-busy="loading || undefined"
:aria-invalid="invalid || undefined"
role="combobox"
:aria-expanded="false"
aria-haspopup="listbox"
:tabindex="0"
>
<!-- Trigger value -->
<template #value="slotProps">
"
>
<div
:class="
cn('flex items-center gap-2', size === 'md' ? 'text-xs' : 'text-sm')
cn(
'flex flex-1 items-center gap-2 overflow-hidden py-2',
size === 'md' ? 'pl-3 text-xs' : 'pl-4 text-sm'
)
"
>
<i
v-if="loading"
class="icon-[lucide--loader-circle] animate-spin text-muted-foreground"
class="icon-[lucide--loader-circle] shrink-0 animate-spin text-muted-foreground"
/>
<slot v-else name="icon" />
<span
v-if="slotProps.value !== null && slotProps.value !== undefined"
class="text-base-foreground"
>
{{ getLabel(slotProps.value) }}
</span>
<span v-else class="text-base-foreground">
{{ label }}
</span>
<SelectValue :placeholder="label" class="truncate" />
</div>
</template>
<!-- Trigger caret (hidden when loading) -->
<template #dropdownicon>
<i
v-if="!loading"
class="icon-[lucide--chevron-down] text-muted-foreground"
/>
</template>
<!-- Option row -->
<template #option="{ option, selected }">
<div
class="flex w-full items-center justify-between gap-3"
:style="optionStyle"
class="flex shrink-0 cursor-pointer items-center justify-center px-3"
>
<span class="truncate">{{ option.name }}</span>
<i v-if="selected" class="icon-[lucide--check] text-base-foreground" />
<i class="icon-[lucide--chevron-down] text-muted-foreground" />
</div>
</template>
</Select>
</SelectTrigger>
<SelectPortal>
<SelectContent
position="popper"
:side-offset="8"
align="start"
:style="optionStyle"
:class="
cn(
'z-3000 overflow-hidden',
'rounded-lg p-2',
'bg-base-background text-base-foreground',
'border border-solid border-border-default',
'shadow-md',
'min-w-(--reka-select-trigger-width)',
'data-[state=closed]:animate-out data-[state=open]:animate-in',
'data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
'data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95',
'data-[side=bottom]:slide-in-from-top-2'
)
"
>
<SelectViewport
:style="{ maxHeight: `min(${listMaxHeight}, 50vh)` }"
class="scrollbar-custom w-full"
>
<SelectItem
v-for="opt in options"
:key="opt.value"
:value="opt.value"
:class="
cn(
'relative flex w-full cursor-pointer items-center justify-between select-none',
'gap-3 rounded-sm px-2 py-3 text-sm outline-none',
'hover:bg-secondary-background-hover',
'focus:bg-secondary-background-hover',
'data-[state=checked]:bg-secondary-background-selected',
'data-[state=checked]:hover:bg-secondary-background-selected'
)
"
>
<SelectItemText class="truncate">
{{ opt.name }}
</SelectItemText>
<SelectItemIndicator
class="flex shrink-0 items-center justify-center"
>
<i
class="icon-[lucide--check] text-base-foreground"
aria-hidden="true"
/>
</SelectItemIndicator>
</SelectItem>
</SelectViewport>
</SelectContent>
</SelectPortal>
</SelectRoot>
</template>
<script setup lang="ts">
import type { SelectPassThroughMethodOptions } from 'primevue/select'
import Select from 'primevue/select'
import { computed } from 'vue'
import {
SelectContent,
SelectItem,
SelectItemIndicator,
SelectItemText,
SelectPortal,
SelectRoot,
SelectTrigger,
SelectValue,
SelectViewport
} from 'reka-ui'
import { useI18n } from 'vue-i18n'
import { usePopoverSizing } from '@/composables/usePopoverSizing'
import { cn } from '@/utils/tailwindUtil'
import type { SelectOption } from './types'
@@ -152,16 +129,12 @@ const {
size = 'lg',
invalid = false,
loading = false,
disabled = false,
listMaxHeight = '28rem',
popoverMinWidth,
popoverMaxWidth
} = defineProps<{
label?: string
/**
* Required for displaying the selected item's label.
* Cannot rely on $attrs alone because we need to access options
* in getLabel() to map values to their display names.
*/
options?: SelectOption[]
/** Trigger size: 'lg' (40px, Interface) or 'md' (32px, Node) */
size?: 'lg' | 'md'
@@ -169,6 +142,8 @@ const {
invalid?: boolean
/** Show loading spinner instead of chevron */
loading?: boolean
/** Disable the select */
disabled?: boolean
/** Maximum height of the dropdown panel (default: 28rem) */
listMaxHeight?: string
/** Minimum width of the popover (default: auto) */
@@ -181,26 +156,8 @@ const selectedItem = defineModel<string | undefined>({ required: true })
const { t } = useI18n()
/**
* Maps a value to its display label.
* Necessary because PrimeVue's value slot doesn't provide the selected item's label,
* only the raw value. We need this to show the correct text when an item is selected.
*/
const getLabel = (val: string | null | undefined) => {
if (val == null) return label ?? ''
if (!options) return label ?? ''
const found = options.find((o) => o.value === val)
return found ? found.name : (label ?? '')
}
// Extract complex style logic from template
const optionStyle = computed(() => {
if (!popoverMinWidth && !popoverMaxWidth) return undefined
const styles: string[] = []
if (popoverMinWidth) styles.push(`min-width: ${popoverMinWidth}`)
if (popoverMaxWidth) styles.push(`max-width: ${popoverMaxWidth}`)
return styles.join('; ')
const optionStyle = usePopoverSizing({
minWidth: popoverMinWidth,
maxWidth: popoverMaxWidth
})
</script>

View File

@@ -14,6 +14,11 @@ const meta: Meta<typeof Loader> = {
control: 'select',
options: ['sm', 'md', 'lg'],
description: 'Spinner size: sm (16px), md (32px), lg (48px)'
},
variant: {
control: 'select',
options: ['loader', 'loader-circle'],
description: 'The type of loader displayed'
}
}
}

View File

@@ -9,12 +9,15 @@ import TabList from '@/components/tab/TabList.vue'
import Button from '@/components/ui/button/Button.vue'
import { useGraphHierarchy } from '@/composables/graph/useGraphHierarchy'
import { st } from '@/i18n'
import { app } from '@/scripts/app'
import { getActiveGraphNodeIds } from '@/utils/graphTraversalUtil'
import { SubgraphNode } from '@/lib/litegraph/src/litegraph'
import type { LGraphNode } from '@/lib/litegraph/src/litegraph'
import { useSettingStore } from '@/platform/settings/settingStore'
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import { useMissingModelStore } from '@/platform/missingModel/missingModelStore'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useMissingNodesErrorStore } from '@/platform/nodeReplacement/missingNodesErrorStore'
import { useRightSidePanelStore } from '@/stores/workspace/rightSidePanelStore'
import type { RightSidePanelTab } from '@/stores/workspace/rightSidePanelStore'
import { resolveNodeDisplayName } from '@/utils/nodeTitleUtil'
@@ -38,12 +41,21 @@ import TabErrors from './errors/TabErrors.vue'
const canvasStore = useCanvasStore()
const executionErrorStore = useExecutionErrorStore()
const missingModelStore = useMissingModelStore()
const missingNodesErrorStore = useMissingNodesErrorStore()
const rightSidePanelStore = useRightSidePanelStore()
const settingStore = useSettingStore()
const { t } = useI18n()
const { hasAnyError, allErrorExecutionIds, activeMissingNodeGraphIds } =
storeToRefs(executionErrorStore)
const { hasAnyError, allErrorExecutionIds } = storeToRefs(executionErrorStore)
const activeMissingNodeGraphIds = computed<Set<string>>(() => {
if (!app.isGraphReady) return new Set()
return getActiveGraphNodeIds(
app.rootGraph,
canvasStore.currentGraph ?? app.rootGraph,
missingNodesErrorStore.missingAncestorExecutionIds
)
})
const { activeMissingModelGraphIds } = storeToRefs(missingModelStore)

View File

@@ -237,6 +237,11 @@ describe('ErrorNodeCard.vue', () => {
// Report is still generated with fallback log message
expect(mockGenerateErrorReport).toHaveBeenCalledOnce()
expect(mockGenerateErrorReport).toHaveBeenCalledWith(
expect.objectContaining({
serverLogs: 'Failed to retrieve server logs'
})
)
expect(wrapper.text()).toContain('ComfyUI Error Report')
})

View File

@@ -90,6 +90,7 @@
variant="secondary"
size="sm"
class="h-8 w-2/3 justify-center gap-1 rounded-lg text-xs"
data-testid="error-card-find-on-github"
@click="handleCheckGithub(error)"
>
{{ t('g.findOnGithub') }}
@@ -99,6 +100,7 @@
variant="secondary"
size="sm"
class="h-8 w-1/3 justify-center gap-1 rounded-lg text-xs"
data-testid="error-card-copy"
@click="handleCopyError(idx)"
>
{{ t('g.copy') }}
@@ -125,12 +127,10 @@
import { useI18n } from 'vue-i18n'
import Button from '@/components/ui/button/Button.vue'
import { useExternalLink } from '@/composables/useExternalLink'
import { useTelemetry } from '@/platform/telemetry'
import { useCommandStore } from '@/stores/commandStore'
import { cn } from '@/utils/tailwindUtil'
import type { ErrorCardData, ErrorItem } from './types'
import { useErrorActions } from './useErrorActions'
import { useErrorReport } from './useErrorReport'
const {
@@ -154,10 +154,8 @@ const emit = defineEmits<{
}>()
const { t } = useI18n()
const telemetry = useTelemetry()
const { staticUrls } = useExternalLink()
const commandStore = useCommandStore()
const { displayedDetailsMap } = useErrorReport(() => card)
const { findOnGitHub, contactSupport: handleGetHelp } = useErrorActions()
function handleLocateNode() {
if (card.nodeId) {
@@ -178,23 +176,6 @@ function handleCopyError(idx: number) {
}
function handleCheckGithub(error: ErrorItem) {
telemetry?.trackUiButtonClicked({
button_id: 'error_tab_find_existing_issues_clicked'
})
const query = encodeURIComponent(error.message + ' is:issue')
window.open(
`${staticUrls.githubIssues}?q=${query}`,
'_blank',
'noopener,noreferrer'
)
}
function handleGetHelp() {
telemetry?.trackHelpResourceClicked({
resource_type: 'help_feedback',
is_external: true,
source: 'error_dialog'
})
commandStore.execute('Comfy.ContactSupport')
findOnGitHub(error.message)
}
</script>

View File

@@ -1,7 +1,5 @@
import { mount } from '@vue/test-utils'
import { createTestingPinia } from '@pinia/testing'
import PrimeVue from 'primevue/config'
import { ref } from 'vue'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { createI18n } from 'vue-i18n'
@@ -42,23 +40,25 @@ vi.mock('@/stores/systemStatsStore', () => ({
})
}))
const mockApplyChanges = vi.fn()
const mockIsRestarting = ref(false)
const mockApplyChanges = vi.hoisted(() => vi.fn())
const mockIsRestarting = vi.hoisted(() => ({ value: false }))
vi.mock('@/workbench/extensions/manager/composables/useApplyChanges', () => ({
useApplyChanges: () => ({
isRestarting: mockIsRestarting,
get isRestarting() {
return mockIsRestarting.value
},
applyChanges: mockApplyChanges
})
}))
const mockIsPackInstalled = vi.fn(() => false)
const mockIsPackInstalled = vi.hoisted(() => vi.fn(() => false))
vi.mock('@/workbench/extensions/manager/stores/comfyManagerStore', () => ({
useComfyManagerStore: () => ({
isPackInstalled: mockIsPackInstalled
})
}))
const mockShouldShowManagerButtons = { value: false }
const mockShouldShowManagerButtons = vi.hoisted(() => ({ value: false }))
vi.mock('@/workbench/extensions/manager/composables/useManagerState', () => ({
useManagerState: () => ({
shouldShowManagerButtons: mockShouldShowManagerButtons
@@ -128,7 +128,7 @@ function mountCard(
...props
},
global: {
plugins: [createTestingPinia({ createSpy: vi.fn }), PrimeVue, i18n],
plugins: [createTestingPinia({ createSpy: vi.fn }), i18n],
stubs: {
DotSpinner: { template: '<span role="status" aria-label="loading" />' }
}

View File

@@ -209,12 +209,9 @@ describe('TabErrors.vue', () => {
}
})
// Find the copy button by text (rendered inside ErrorNodeCard)
const copyButton = wrapper
.findAll('button')
.find((btn) => btn.text().includes('Copy'))
expect(copyButton).toBeTruthy()
await copyButton!.trigger('click')
const copyButton = wrapper.find('[data-testid="error-card-copy"]')
expect(copyButton.exists()).toBe(true)
await copyButton.trigger('click')
expect(mockCopy).toHaveBeenCalledWith('Test message\n\nTest details')
})
@@ -245,5 +242,9 @@ describe('TabErrors.vue', () => {
// Should render in the dedicated runtime error panel, not inside accordion
const runtimePanel = wrapper.find('[data-testid="runtime-error-panel"]')
expect(runtimePanel.exists()).toBe(true)
// Verify the error message appears exactly once (not duplicated in accordion)
expect(
wrapper.text().match(/RuntimeError: Out of memory/g) ?? []
).toHaveLength(1)
})
})

View File

@@ -53,6 +53,7 @@
<PropertiesAccordionItem
v-for="group in filteredGroups"
:key="group.title"
:data-testid="'error-group-' + group.type.replaceAll('_', '-')"
:collapse="isSectionCollapsed(group.title) && !isSearching"
class="border-b border-interface-stroke"
:size="getGroupSize(group)"
@@ -209,12 +210,9 @@
import { computed, ref, watch } from 'vue'
import { useI18n } from 'vue-i18n'
import { useCommandStore } from '@/stores/commandStore'
import { useCopyToClipboard } from '@/composables/useCopyToClipboard'
import { useFocusNode } from '@/composables/canvas/useFocusNode'
import { useExternalLink } from '@/composables/useExternalLink'
import { useSettingStore } from '@/platform/settings/settingStore'
import { useTelemetry } from '@/platform/telemetry'
import { useRightSidePanelStore } from '@/stores/workspace/rightSidePanelStore'
import { useManagerState } from '@/workbench/extensions/manager/composables/useManagerState'
import { ManagerTab } from '@/workbench/extensions/manager/types/comfyManagerTypes'
@@ -238,6 +236,7 @@ import Button from '@/components/ui/button/Button.vue'
import DotSpinner from '@/components/common/DotSpinner.vue'
import { usePackInstall } from '@/workbench/extensions/manager/composables/nodePack/usePackInstall'
import { useMissingNodes } from '@/workbench/extensions/manager/composables/nodePack/useMissingNodes'
import { useErrorActions } from './useErrorActions'
import { useErrorGroups } from './useErrorGroups'
import type { SwapNodeGroup } from './useErrorGroups'
import type { ErrorGroup } from './types'
@@ -246,7 +245,7 @@ import { useNodeReplacement } from '@/platform/nodeReplacement/useNodeReplacemen
const { t } = useI18n()
const { copyToClipboard } = useCopyToClipboard()
const { focusNode, enterSubgraph } = useFocusNode()
const { staticUrls } = useExternalLink()
const { openGitHubIssues, contactSupport } = useErrorActions()
const settingStore = useSettingStore()
const rightSidePanelStore = useRightSidePanelStore()
const { shouldShowManagerButtons, shouldShowInstallButton, openManager } =
@@ -372,13 +371,13 @@ watch(
if (!graphNodeId) return
const prefix = `${graphNodeId}:`
for (const group of allErrorGroups.value) {
const hasMatch =
group.type === 'execution' &&
group.cards.some(
(card) =>
card.graphNodeId === graphNodeId ||
(card.nodeId?.startsWith(prefix) ?? false)
)
if (group.type !== 'execution') continue
const hasMatch = group.cards.some(
(card) =>
card.graphNodeId === graphNodeId ||
(card.nodeId?.startsWith(prefix) ?? false)
)
setSectionCollapsed(group.title, !hasMatch)
}
rightSidePanelStore.focusedErrorNodeId = null
@@ -418,20 +417,4 @@ function handleReplaceAll() {
function handleEnterSubgraph(nodeId: string) {
enterSubgraph(nodeId, errorNodeCache.value)
}
function openGitHubIssues() {
useTelemetry()?.trackUiButtonClicked({
button_id: 'error_tab_github_issues_clicked'
})
window.open(staticUrls.githubIssues, '_blank', 'noopener,noreferrer')
}
async function contactSupport() {
useTelemetry()?.trackHelpResourceClicked({
resource_type: 'help_feedback',
is_external: true,
source: 'error_dialog'
})
useCommandStore().execute('Comfy.ContactSupport')
}
</script>

View File

@@ -47,7 +47,7 @@ vi.mock('@/utils/executableGroupNodeDto', () => ({
isGroupNode: vi.fn(() => false)
}))
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useMissingNodesErrorStore } from '@/platform/nodeReplacement/missingNodesErrorStore'
import { useErrorGroups } from './useErrorGroups'
function makeMissingNodeType(
@@ -80,8 +80,7 @@ describe('swapNodeGroups computed', () => {
})
function getSwapNodeGroups(nodeTypes: MissingNodeType[]) {
const store = useExecutionErrorStore()
store.surfaceMissingNodes(nodeTypes)
useMissingNodesErrorStore().surfaceMissingNodes(nodeTypes)
const searchQuery = ref('')
const t = (key: string) => key

View File

@@ -0,0 +1,39 @@
import { useCommandStore } from '@/stores/commandStore'
import { useExternalLink } from '@/composables/useExternalLink'
import { useTelemetry } from '@/platform/telemetry'
export function useErrorActions() {
const telemetry = useTelemetry()
const commandStore = useCommandStore()
const { staticUrls } = useExternalLink()
function openGitHubIssues() {
telemetry?.trackUiButtonClicked({
button_id: 'error_tab_github_issues_clicked'
})
window.open(staticUrls.githubIssues, '_blank', 'noopener,noreferrer')
}
function contactSupport() {
telemetry?.trackHelpResourceClicked({
resource_type: 'help_feedback',
is_external: true,
source: 'error_dialog'
})
void commandStore.execute('Comfy.ContactSupport')
}
function findOnGitHub(errorMessage: string) {
telemetry?.trackUiButtonClicked({
button_id: 'error_tab_find_existing_issues_clicked'
})
const query = encodeURIComponent(errorMessage + ' is:issue')
window.open(
`${staticUrls.githubIssues}?q=${query}`,
'_blank',
'noopener,noreferrer'
)
}
return { openGitHubIssues, contactSupport, findOnGitHub }
}

View File

@@ -58,6 +58,7 @@ vi.mock(
)
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useMissingNodesErrorStore } from '@/platform/nodeReplacement/missingNodesErrorStore'
import { useErrorGroups } from './useErrorGroups'
function makeMissingNodeType(
@@ -126,8 +127,9 @@ describe('useErrorGroups', () => {
})
it('groups non-replaceable nodes by cnrId', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('NodeA', { cnrId: 'pack-1' }),
makeMissingNodeType('NodeB', { cnrId: 'pack-1', nodeId: '2' }),
makeMissingNodeType('NodeC', { cnrId: 'pack-2', nodeId: '3' })
@@ -146,8 +148,9 @@ describe('useErrorGroups', () => {
})
it('excludes replaceable nodes from missingPackGroups', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('OldNode', {
isReplaceable: true,
replacement: { new_node_id: 'NewNode' }
@@ -164,8 +167,9 @@ describe('useErrorGroups', () => {
})
it('groups nodes without cnrId under null packId', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('UnknownNode', { nodeId: '1' }),
makeMissingNodeType('AnotherUnknown', { nodeId: '2' })
])
@@ -177,8 +181,9 @@ describe('useErrorGroups', () => {
})
it('sorts groups alphabetically with null packId last', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('NodeA', { cnrId: 'zebra-pack' }),
makeMissingNodeType('NodeB', { nodeId: '2' }),
makeMissingNodeType('NodeC', { cnrId: 'alpha-pack', nodeId: '3' })
@@ -190,8 +195,9 @@ describe('useErrorGroups', () => {
})
it('sorts nodeTypes within each group alphabetically by type then nodeId', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('NodeB', { cnrId: 'pack-1', nodeId: '2' }),
makeMissingNodeType('NodeA', { cnrId: 'pack-1', nodeId: '3' }),
makeMissingNodeType('NodeA', { cnrId: 'pack-1', nodeId: '1' })
@@ -206,8 +212,9 @@ describe('useErrorGroups', () => {
})
it('handles string nodeType entries', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
'StringGroupNode' as unknown as MissingNodeType
])
await nextTick()
@@ -224,8 +231,9 @@ describe('useErrorGroups', () => {
})
it('includes missing_node group when missing nodes exist', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('NodeA', { cnrId: 'pack-1' })
])
await nextTick()
@@ -237,8 +245,9 @@ describe('useErrorGroups', () => {
})
it('includes swap_nodes group when replaceable nodes exist', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('OldNode', {
isReplaceable: true,
replacement: { new_node_id: 'NewNode' }
@@ -253,8 +262,9 @@ describe('useErrorGroups', () => {
})
it('includes both swap_nodes and missing_node when both exist', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('OldNode', {
isReplaceable: true,
replacement: { new_node_id: 'NewNode' }
@@ -272,8 +282,9 @@ describe('useErrorGroups', () => {
})
it('swap_nodes has lower priority than missing_node', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('OldNode', {
isReplaceable: true,
replacement: { new_node_id: 'NewNode' }
@@ -533,13 +544,18 @@ describe('useErrorGroups', () => {
})
it('includes missing node group title as message', async () => {
const { store, groups } = createErrorGroups()
store.setMissingNodeTypes([
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
makeMissingNodeType('NodeA', { cnrId: 'pack-1' })
])
await nextTick()
expect(groups.groupedErrorMessages.value.length).toBeGreaterThan(0)
const missingGroup = groups.allErrorGroups.value.find(
(g) => g.type === 'missing_node'
)
expect(missingGroup).toBeDefined()
expect(groups.groupedErrorMessages.value).toContain(missingGroup!.title)
})
})

View File

@@ -5,6 +5,7 @@ import type { IFuseOptions } from 'fuse.js'
import { useMissingModelStore } from '@/platform/missingModel/missingModelStore'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useMissingNodesErrorStore } from '@/platform/nodeReplacement/missingNodesErrorStore'
import { useComfyRegistryStore } from '@/stores/comfyRegistryStore'
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import { app } from '@/scripts/app'
@@ -195,12 +196,8 @@ function searchErrorGroups(groups: ErrorGroup[], query: string) {
cardIndex: ci,
searchableNodeId: card.nodeId ?? '',
searchableNodeTitle: card.nodeTitle ?? '',
searchableMessage: card.errors
.map((e: ErrorItem) => e.message)
.join(' '),
searchableDetails: card.errors
.map((e: ErrorItem) => e.details ?? '')
.join(' ')
searchableMessage: card.errors.map((e) => e.message).join(' '),
searchableDetails: card.errors.map((e) => e.details ?? '').join(' ')
})
}
}
@@ -240,6 +237,7 @@ export function useErrorGroups(
t: (key: string) => string
) {
const executionErrorStore = useExecutionErrorStore()
const missingNodesStore = useMissingNodesErrorStore()
const missingModelStore = useMissingModelStore()
const canvasStore = useCanvasStore()
const { inferPackFromNodeName } = useComfyRegistryStore()
@@ -285,7 +283,7 @@ export function useErrorGroups(
const missingNodeCache = computed(() => {
const map = new Map<string, LGraphNode>()
const nodeTypes = executionErrorStore.missingNodesError?.nodeTypes ?? []
const nodeTypes = missingNodesStore.missingNodesError?.nodeTypes ?? []
for (const nodeType of nodeTypes) {
if (typeof nodeType === 'string') continue
if (nodeType.nodeId == null) continue
@@ -407,7 +405,7 @@ export function useErrorGroups(
const asyncResolvedIds = ref<Map<string, string | null>>(new Map())
const pendingTypes = computed(() =>
(executionErrorStore.missingNodesError?.nodeTypes ?? []).filter(
(missingNodesStore.missingNodesError?.nodeTypes ?? []).filter(
(n): n is Exclude<MissingNodeType, string> =>
typeof n !== 'string' && !n.cnrId
)
@@ -448,6 +446,8 @@ export function useErrorGroups(
for (const r of results) {
if (r.status === 'fulfilled') {
final.set(r.value.type, r.value.packId)
} else {
console.warn('Failed to resolve pack ID:', r.reason)
}
}
// Clear any remaining RESOLVING markers for failed lookups
@@ -459,8 +459,18 @@ export function useErrorGroups(
{ immediate: true }
)
// Evict stale entries when missing nodes are cleared
watch(
() => missingNodesStore.missingNodesError,
(error) => {
if (!error && asyncResolvedIds.value.size > 0) {
asyncResolvedIds.value = new Map()
}
}
)
const missingPackGroups = computed<MissingPackGroup[]>(() => {
const nodeTypes = executionErrorStore.missingNodesError?.nodeTypes ?? []
const nodeTypes = missingNodesStore.missingNodesError?.nodeTypes ?? []
const map = new Map<
string | null,
{ nodeTypes: MissingNodeType[]; isResolving: boolean }
@@ -522,7 +532,7 @@ export function useErrorGroups(
})
const swapNodeGroups = computed<SwapNodeGroup[]>(() => {
const nodeTypes = executionErrorStore.missingNodesError?.nodeTypes ?? []
const nodeTypes = missingNodesStore.missingNodesError?.nodeTypes ?? []
const map = new Map<string, SwapNodeGroup>()
for (const nodeType of nodeTypes) {
@@ -546,7 +556,7 @@ export function useErrorGroups(
/** Builds an ErrorGroup from missingNodesError. Returns [] when none present. */
function buildMissingNodeGroups(): ErrorGroup[] {
const error = executionErrorStore.missingNodesError
const error = missingNodesStore.missingNodesError
if (!error) return []
const groups: ErrorGroup[] = []

View File

@@ -2,6 +2,8 @@ import { computed, onMounted, onUnmounted, reactive, toValue } from 'vue'
import type { MaybeRefOrGetter } from 'vue'
import { until } from '@vueuse/core'
import { api } from '@/scripts/api'
import { app } from '@/scripts/app'
import { useSystemStatsStore } from '@/stores/systemStatsStore'
@@ -40,24 +42,33 @@ export function useErrorReport(cardSource: MaybeRefOrGetter<ErrorCardData>) {
if (runtimeErrors.length === 0) return
if (!systemStatsStore.systemStats) {
try {
await systemStatsStore.refetchSystemStats()
} catch {
return
if (systemStatsStore.isLoading) {
await until(systemStatsStore.isLoading).toBe(false)
} else {
try {
await systemStatsStore.refetchSystemStats()
} catch (e) {
console.warn('Failed to fetch system stats for error report:', e)
return
}
}
}
if (cancelled || !systemStatsStore.systemStats) return
let logs: string
try {
logs = await api.getLogs()
} catch {
logs = 'Failed to retrieve server logs'
}
if (!systemStatsStore.systemStats || cancelled) return
const logs = await api
.getLogs()
.catch(() => 'Failed to retrieve server logs')
if (cancelled) return
const workflow = app.rootGraph.serialize()
const workflow = (() => {
try {
return app.rootGraph.serialize()
} catch (e) {
console.warn('Failed to serialize workflow for error report:', e)
return null
}
})()
if (!workflow) return
for (const { error, idx } of runtimeErrors) {
try {
@@ -72,8 +83,8 @@ export function useErrorReport(cardSource: MaybeRefOrGetter<ErrorCardData>) {
workflow
})
enrichedDetails[idx] = report
} catch {
// Fallback: keep original error.details
} catch (e) {
console.warn('Failed to generate error report:', e)
}
}
})

View File

@@ -65,8 +65,7 @@ describe('DefaultThumbnail', () => {
const classString = Array.isArray(imageClass)
? imageClass.join(' ')
: imageClass
expect(classString).toContain('w-full')
expect(classString).toContain('h-full')
expect(classString).toContain('size-full')
expect(classString).toContain('object-cover')
})

View File

@@ -3,12 +3,14 @@
<LazyImage
:src="src"
:alt="alt"
:image-class="[
'transform-gpu transition-transform duration-300 ease-out',
isVideoType
? 'w-full h-full object-cover'
: 'max-w-full max-h-64 object-contain'
]"
:image-class="
cn(
'transform-gpu transition-transform duration-300 ease-out',
isVideoType
? 'size-full object-cover'
: 'max-h-64 max-w-full object-contain'
)
"
:image-style="
isHovered ? { transform: `scale(${1 + hoverZoom / 100})` } : undefined
"
@@ -19,6 +21,7 @@
<script setup lang="ts">
import LazyImage from '@/components/common/LazyImage.vue'
import BaseThumbnail from '@/components/templates/thumbnails/BaseThumbnail.vue'
import { cn } from '@/utils/tailwindUtil'
const { src, isVideo } = defineProps<{
src: string

View File

@@ -140,13 +140,19 @@ export const useFirebaseAuthActions = () => {
return result
}, reportError)
const signInWithGoogle = wrapWithErrorHandlingAsync(async () => {
return await authStore.loginWithGoogle()
}, reportError)
const signInWithGoogle = wrapWithErrorHandlingAsync(
async (options?: { isNewUser?: boolean }) => {
return await authStore.loginWithGoogle(options)
},
reportError
)
const signInWithGithub = wrapWithErrorHandlingAsync(async () => {
return await authStore.loginWithGithub()
}, reportError)
const signInWithGithub = wrapWithErrorHandlingAsync(
async (options?: { isNewUser?: boolean }) => {
return await authStore.loginWithGithub(options)
},
reportError
)
const signInWithEmail = wrapWithErrorHandlingAsync(
async (email: string, password: string) => {

View File

@@ -315,6 +315,45 @@ describe('installErrorClearingHooks lifecycle', () => {
cleanup()
expect(graph.onNodeAdded).toBe(originalHook)
})
it('restores original node callbacks when a node is removed', () => {
const graph = new LGraph()
const node = new LGraphNode('test')
node.addInput('clip', 'CLIP')
node.addWidget('number', 'steps', 20, () => undefined, {})
const originalOnConnectionsChange = vi.fn()
const originalOnWidgetChanged = vi.fn()
node.onConnectionsChange = originalOnConnectionsChange
node.onWidgetChanged = originalOnWidgetChanged
graph.add(node)
installErrorClearingHooks(graph)
// Callbacks should be chained (not the originals)
expect(node.onConnectionsChange).not.toBe(originalOnConnectionsChange)
expect(node.onWidgetChanged).not.toBe(originalOnWidgetChanged)
// Simulate node removal via the graph hook
graph.onNodeRemoved!(node)
// Original callbacks should be restored
expect(node.onConnectionsChange).toBe(originalOnConnectionsChange)
expect(node.onWidgetChanged).toBe(originalOnWidgetChanged)
})
it('does not double-wrap callbacks when installErrorClearingHooks is called twice', () => {
const graph = new LGraph()
const node = new LGraphNode('test')
node.addInput('clip', 'CLIP')
graph.add(node)
installErrorClearingHooks(graph)
const chainedAfterFirst = node.onConnectionsChange
// Install again on the same graph — should be a no-op for existing nodes
installErrorClearingHooks(graph)
expect(node.onConnectionsChange).toBe(chainedAfterFirst)
})
})
describe('clearWidgetRelatedErrors parameter routing', () => {

View File

@@ -35,10 +35,22 @@ function resolvePromotedExecId(
const hookedNodes = new WeakSet<LGraphNode>()
type OriginalCallbacks = {
onConnectionsChange: LGraphNode['onConnectionsChange']
onWidgetChanged: LGraphNode['onWidgetChanged']
}
const originalCallbacks = new WeakMap<LGraphNode, OriginalCallbacks>()
function installNodeHooks(node: LGraphNode): void {
if (hookedNodes.has(node)) return
hookedNodes.add(node)
originalCallbacks.set(node, {
onConnectionsChange: node.onConnectionsChange,
onWidgetChanged: node.onWidgetChanged
})
node.onConnectionsChange = useChainCallback(
node.onConnectionsChange,
function (type, slotIndex, isConnected) {
@@ -82,6 +94,15 @@ function installNodeHooks(node: LGraphNode): void {
)
}
function restoreNodeHooks(node: LGraphNode): void {
const originals = originalCallbacks.get(node)
if (!originals) return
node.onConnectionsChange = originals.onConnectionsChange
node.onWidgetChanged = originals.onWidgetChanged
originalCallbacks.delete(node)
hookedNodes.delete(node)
}
function installNodeHooksRecursive(node: LGraphNode): void {
installNodeHooks(node)
if (node.isSubgraphNode?.()) {
@@ -91,6 +112,15 @@ function installNodeHooksRecursive(node: LGraphNode): void {
}
}
function restoreNodeHooksRecursive(node: LGraphNode): void {
restoreNodeHooks(node)
if (node.isSubgraphNode?.()) {
for (const innerNode of node.subgraph._nodes ?? []) {
restoreNodeHooksRecursive(innerNode)
}
}
}
export function installErrorClearingHooks(graph: LGraph): () => void {
for (const node of graph._nodes ?? []) {
installNodeHooksRecursive(node)
@@ -102,7 +132,17 @@ export function installErrorClearingHooks(graph: LGraph): () => void {
originalOnNodeAdded?.call(this, node)
}
const originalOnNodeRemoved = graph.onNodeRemoved
graph.onNodeRemoved = function (node: LGraphNode) {
restoreNodeHooksRecursive(node)
originalOnNodeRemoved?.call(this, node)
}
return () => {
for (const node of graph._nodes ?? []) {
restoreNodeHooksRecursive(node)
}
graph.onNodeAdded = originalOnNodeAdded || undefined
graph.onNodeRemoved = originalOnNodeRemoved || undefined
}
}

View File

@@ -197,14 +197,16 @@ describe('Widget slotMetadata reactivity on link disconnect', () => {
const subgraphNode = createTestSubgraphNode(subgraph, { id: 123 })
// Create a PromotedWidgetView with displayName="value" (subgraph input
// Create a PromotedWidgetView with identityName="value" (subgraph input
// slot name) and sourceWidgetName="prompt" (interior widget name).
// PromotedWidgetView.name returns "value", but safeWidgetMapper sets
// SafeWidgetData.name to sourceWidgetName ("prompt").
// PromotedWidgetView.name returns "value" (identity), safeWidgetMapper
// sets SafeWidgetData.name to sourceWidgetName ("prompt").
const promotedView = createPromotedWidgetView(
subgraphNode,
'10',
'prompt',
'value',
undefined,
'value'
)

View File

@@ -92,6 +92,10 @@ export interface SafeWidgetData {
* execution ID (e.g. `"65:42"` vs the host node's `"65"`).
*/
sourceExecutionId?: string
/** Tooltip text from the resolved widget. */
tooltip?: string
/** For promoted widgets, the display label from the subgraph input slot. */
promotedLabel?: string
}
export interface VueNodeData {
@@ -352,7 +356,8 @@ function safeWidgetMapper(
sourceNode && app.rootGraph
? (getExecutionIdByNode(app.rootGraph, sourceNode) ?? undefined)
: undefined,
tooltip: widget.tooltip
tooltip: widget.tooltip,
promotedLabel: isPromotedWidgetView(widget) ? widget.label : undefined
}
} catch (error) {
console.warn(
@@ -803,6 +808,8 @@ export function useGraphNodeManager(graph: LGraph): GraphNodeManager {
if (slotLabelEvent.slotType !== NodeSlotType.INPUT && nodeRef.outputs) {
nodeRef.outputs = [...nodeRef.outputs]
}
// Re-extract widget data so promotedLabel reflects the rename
vueNodeData.set(nodeId, extractVueNodeData(nodeRef))
}
}

View File

@@ -0,0 +1,111 @@
import type { Ref } from 'vue'
import { computed, watch } from 'vue'
import type { LGraph, LGraphNode } from '@/lib/litegraph/src/litegraph'
import type { useMissingModelStore } from '@/platform/missingModel/missingModelStore'
import { useSettingStore } from '@/platform/settings/settingStore'
import { app } from '@/scripts/app'
import type { NodeError } from '@/schemas/apiSchema'
import { getParentExecutionIds } from '@/types/nodeIdentification'
import { forEachNode, getNodeByExecutionId } from '@/utils/graphTraversalUtil'
function setNodeHasErrors(node: LGraphNode, hasErrors: boolean): void {
if (node.has_errors === hasErrors) return
const oldValue = node.has_errors
node.has_errors = hasErrors
node.graph?.trigger('node:property:changed', {
type: 'node:property:changed',
nodeId: node.id,
property: 'has_errors',
oldValue,
newValue: hasErrors
})
}
/**
* Single-pass reconciliation of node error flags.
* Collects the set of nodes that should have errors, then walks all nodes
* once, setting each flag exactly once. This avoids the redundant
* true→false→true transition (and duplicate events) that a clear-then-apply
* approach would cause.
*/
function reconcileNodeErrorFlags(
rootGraph: LGraph,
nodeErrors: Record<string, NodeError> | null,
missingModelExecIds: Set<string>
): void {
// Collect nodes and slot info that should be flagged
// Includes both error-owning nodes and their ancestor containers
const flaggedNodes = new Set<LGraphNode>()
const errorSlots = new Map<LGraphNode, Set<string>>()
if (nodeErrors) {
for (const [executionId, nodeError] of Object.entries(nodeErrors)) {
const node = getNodeByExecutionId(rootGraph, executionId)
if (!node) continue
flaggedNodes.add(node)
const slotNames = new Set<string>()
for (const error of nodeError.errors) {
const name = error.extra_info?.input_name
if (name) slotNames.add(name)
}
if (slotNames.size > 0) errorSlots.set(node, slotNames)
for (const parentId of getParentExecutionIds(executionId)) {
const parentNode = getNodeByExecutionId(rootGraph, parentId)
if (parentNode) flaggedNodes.add(parentNode)
}
}
}
for (const execId of missingModelExecIds) {
const node = getNodeByExecutionId(rootGraph, execId)
if (node) flaggedNodes.add(node)
}
forEachNode(rootGraph, (node) => {
setNodeHasErrors(node, flaggedNodes.has(node))
if (node.inputs) {
const nodeSlotNames = errorSlots.get(node)
for (const slot of node.inputs) {
slot.hasErrors = !!nodeSlotNames?.has(slot.name)
}
}
})
}
export function useNodeErrorFlagSync(
lastNodeErrors: Ref<Record<string, NodeError> | null>,
missingModelStore: ReturnType<typeof useMissingModelStore>
): () => void {
const settingStore = useSettingStore()
const showErrorsTab = computed(() =>
settingStore.get('Comfy.RightSidePanel.ShowErrorsTab')
)
const stop = watch(
[
lastNodeErrors,
() => missingModelStore.missingModelNodeIds,
showErrorsTab
],
() => {
if (!app.isGraphReady) return
// Legacy (LGraphNode) only: suppress missing-model error flags when
// the Errors tab is hidden, since legacy nodes lack the per-widget
// red highlight that Vue nodes use to indicate *why* a node has errors.
// Vue nodes compute hasAnyError independently and are unaffected.
reconcileNodeErrorFlags(
app.rootGraph,
lastNodeErrors.value,
showErrorsTab.value
? missingModelStore.missingModelAncestorExecutionIds
: new Set()
)
},
{ flush: 'post' }
)
return stop
}

View File

@@ -86,6 +86,24 @@ function useVueNodeLifecycleIndividual() {
() => !shouldRenderVueNodes.value,
() => {
disposeNodeManagerAndSyncs()
// Force arrange() on all nodes so input.pos is computed before
// the first legacy drawConnections frame (which may run before
// drawNode on the foreground canvas).
const graph = comfyApp.canvas?.graph
if (!graph) {
comfyApp.canvas?.setDirty(true, true)
return
}
for (const node of graph._nodes) {
if (node.flags.collapsed) continue
try {
node.arrange()
} catch {
/* skip nodes not fully initialized */
}
}
comfyApp.canvas?.setDirty(true, true)
}
)

View File

@@ -0,0 +1,65 @@
import { describe, expect, it } from 'vitest'
import {
exceedsClickThreshold,
useClickDragGuard
} from '@/composables/useClickDragGuard'
describe('exceedsClickThreshold', () => {
it('returns false when distance is within threshold', () => {
expect(exceedsClickThreshold({ x: 0, y: 0 }, { x: 2, y: 2 }, 5)).toBe(false)
})
it('returns true when distance exceeds threshold', () => {
expect(exceedsClickThreshold({ x: 0, y: 0 }, { x: 3, y: 5 }, 5)).toBe(true)
})
it('returns false when distance exactly equals threshold', () => {
expect(exceedsClickThreshold({ x: 0, y: 0 }, { x: 3, y: 4 }, 5)).toBe(false)
})
it('handles negative deltas', () => {
expect(exceedsClickThreshold({ x: 10, y: 10 }, { x: 4, y: 2 }, 5)).toBe(
true
)
})
})
describe('useClickDragGuard', () => {
it('reports no drag when pointer has not moved', () => {
const guard = useClickDragGuard(5)
guard.recordStart({ clientX: 100, clientY: 200 })
expect(guard.wasDragged({ clientX: 100, clientY: 200 })).toBe(false)
})
it('reports no drag when movement is within threshold', () => {
const guard = useClickDragGuard(5)
guard.recordStart({ clientX: 100, clientY: 200 })
expect(guard.wasDragged({ clientX: 103, clientY: 204 })).toBe(false)
})
it('reports drag when movement exceeds threshold', () => {
const guard = useClickDragGuard(5)
guard.recordStart({ clientX: 100, clientY: 200 })
expect(guard.wasDragged({ clientX: 106, clientY: 200 })).toBe(true)
})
it('returns false when no start has been recorded', () => {
const guard = useClickDragGuard(5)
expect(guard.wasDragged({ clientX: 100, clientY: 200 })).toBe(false)
})
it('returns false after reset', () => {
const guard = useClickDragGuard(5)
guard.recordStart({ clientX: 100, clientY: 200 })
guard.reset()
expect(guard.wasDragged({ clientX: 200, clientY: 300 })).toBe(false)
})
it('respects custom threshold', () => {
const guard = useClickDragGuard(3)
guard.recordStart({ clientX: 0, clientY: 0 })
expect(guard.wasDragged({ clientX: 3, clientY: 0 })).toBe(false)
expect(guard.wasDragged({ clientX: 4, clientY: 0 })).toBe(true)
})
})

Some files were not shown because too many files have changed in this diff Show More