Compare commits

..

177 Commits

Author SHA1 Message Date
snomiao
04bf4cbe95 feat: badge shows reproduction method (E2E test / video / both)
- Add reproducedBy field to ResearchResult and done() tool
- Agent reports how bug was proven: e2e_test, video, both, or none
- Badge shows '1 via E2E test' instead of generic '1 reproduced'
- Deploy script reads reproducedBy from research-log.json
- Test code (reproduce.spec.ts) now deployed to report page

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 13:45:21 +00:00
snomiao
705d10bc8d fix: prevent trivial assertions from false REPRODUCED + deploy test code
- Add rule: assertions must be specific to the bug (not just > 0)
- If no bug-specific assertion possible, verdict must be NOT_REPRODUCIBLE
- Copy reproduce.spec.ts to deployed report for transparency
- Addresses #10307 false REPRODUCED (test only asserted node count > 0)

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 13:35:38 +00:00
snomiao
765b5dbf1d fix: remove oversized files before CF Pages deploy (25MB limit)
Large mp4 videos (37-76MB) caused wrangler to silently fail the deploy,
leaving stale ANALYZING badges. Now strips files >25MB before deploying.

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 11:12:57 +00:00
snomiao
8ac9a9843b fix: remove PREPARING/ANALYZING intermediate badge deploys
Root cause: Cloudflare Pages serves stale deployments when multiple
deploys race to the same branch. The ANALYZING placeholder deployed
seconds before the final report would sometimes 'win' the race.

Fix: Only deploy once — the final report with the real badge. No more
intermediate PREPARING or ANALYZING placeholders.

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 10:37:19 +00:00
snomiao
3351f79c27 fix: log wrangler output and always use fallback deploy URL
Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 10:01:02 +00:00
snomiao
64c4bedc0d fix: stop iterating after test passes, ban waitForTimeout in QA tests
- Add explicit instruction to call done() immediately after test passes
- Inject warning message in runTest response when test passes
- Ban page.waitForTimeout() in system prompt (use retrying assertions)
- Instruct agent to write ONE focused test, not multiple

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 08:19:53 +00:00
snomiao
854f1c7da0 feat: readFixture/readTest tools, ANTHROPIC_API_KEY_QA, fix TS errors
- Add readFixture and readTest tools to qa-agent for fixture API discovery
- Enrich system prompt with comprehensive ComfyPage fixture API reference
- Switch CI to ANTHROPIC_API_KEY_QA secret
- Fix all TS errors in qa-agent.ts, qa-record.ts, qa-reproduce.ts
- Better error handling for API credit exhaustion
- Rewrite SKILL.md to reflect three-phase pipeline

Amp-Thread-ID: https://ampcode.com/threads/T-019d4786-eb5f-7115-a10e-5b086c921800
Co-authored-by: Amp <amp@ampcode.com>
2026-04-01 06:44:34 +00:00
snomiao
ed96aaafc6 fix: auto-complete when test passes but agent doesn't call done()
Claude sometimes keeps iterating after a test passes, exhausting
the time budget without calling done(). Now: when runTest() returns
TEST PASSED, the test code is saved. If the agent loop ends without
done(), auto-sets verdict=REPRODUCED with the passing test.

Fixes #8532 (17 calls, test passed twice, but INCONCLUSIVE verdict).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 13:25:07 +00:00
snomiao
4ba526b859 fix: remaining pipefail crashes in deploy script
Added || true to all grep/sed pipe chains that could exit non-zero:
- grep on pr-context.txt (line 149)
- sed/grep on pr-context description (line 158-159)
- grep -oiP on RISK_FIRST (line 331)
- wrangler deploy | grep URL (line 355)

All tested under set -euo pipefail with empty inputs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 12:23:32 +00:00
snomiao
a3dd897823 fix: pipefail crash in deploy script with empty video-reviews
sed on video-reviews/*.md returns exit code 2 when no files match,
killing the script under set -euo pipefail. Added || true to all
potentially empty glob pipelines.

Affects 13/20 QA runs that had successful research but failed deploy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:08 +00:00
snomiao
979124109a fix: use comfyPage.page.waitForTimeout for delay injection
The test uses comfyPageFixture, not bare page. Also match
firstNode await calls for node interaction pauses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:08 +00:00
snomiao
b0f8e69dc2 feat: inject 800ms pauses between test actions for readable videos
Regex inserts await page.waitForTimeout(800) before every
comfyPage/topbar/page/canvas/expect await call in the Phase 2
test code. Adds ~5-8s to a 10-step test (negligible vs 10min research).

Default playback changed to 0.5x (was 0.25x) since pauses provide
natural breathing room. A 15s video at 0.5x = 30s viewing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:08 +00:00
snomiao
377cbf775f feat: default playback 0.25x + cursor overlay in E2E test videos
- Report player defaults to 0.25x speed (was 0.5x) — 5s test videos
  play in 20s, much more watchable
- Phase 2 injects cursor overlay via addInitScript into the test code
  before running — white SVG arrow follows mousemove events

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:08 +00:00
snomiao
4e261fe5c6 fix: set PLAYWRIGHT_LOCAL=1 for Phase 2 to enable video recording
Playwright config only records video when PLAYWRIGHT_LOCAL is set.
In CI, this env var was missing so Phase 2 produced no video.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:08 +00:00
snomiao
629dfc9d37 fix: don't overwrite Phase 2 test video with idle research video
After context.close(), renameLatestWebm would overwrite the Phase 2
test execution video with the idle research browser recording.
Now skips the rename if qa-session.webm already exists from Phase 2.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
8752276945 fix: Phase 2 records test execution video, copies to qa-session.webm
The old video showed an idle screen (research browser doing nothing).
Now Phase 2 runs the test with --video=on from browser_tests/tests/,
finds the recorded .webm, and copies it to qa-session.webm where
the deploy script expects it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
4d6fe47e01 fix: runTest uses project Playwright config + fixtures
- Copy test to browser_tests/tests/ where Playwright config expects it
- System prompt teaches Claude the project's test fixtures:
  comfyPageFixture, comfyPage.menu.topbar, comfyPage.workflow, etc.
- Increased time budget to 10 min for write→run→fix iterations
- Increased max turns to 50

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
4de56e2696 fix: broaden research-log.json search paths in deploy script
Also search qa-artifacts/before/*/research/ for the research log
since artifacts are downloaded with that nested structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
e037244d49 feat: Claude writes E2E tests to reproduce bugs instead of driving browser
Phase 1: Claude reads issue + a11y tree, writes a Playwright .spec.ts
test that asserts the bug exists. Runs the test, reads errors, iterates
until the test passes (proving the bug) or determines NOT_REPRODUCIBLE.

Phase 2: Run the passing test with --video=on for clean recording.

This replaces interactive browser driving with deterministic test code.
Claude Sonnet 4.6 excels at writing Playwright tests — much more
reliable than real-time browser interaction.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
e8ce53c92b fix: use ariaSnapshot() instead of removed page.accessibility API
page.accessibility.snapshot() was removed in Playwright 1.49+.
Use page.locator('body').ariaSnapshot() which returns a text
representation of the accessibility tree.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
30f4976621 feat: deploy research-log.json + use it as primary verdict source
- Copy research-log.json to deploy dir (accessible at /research-log.json)
- Read verdict from research log first (a11y-verified ground truth)
- Fall back to video review verdict only if no research log exists
- Research log is uploaded as part of QA artifacts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
b156ecd493 feat: three-phase QA pipeline — Research → Reproduce → Report
Phase 1 (qa-agent.ts): Claude investigates via a11y API only.
  - No video, no Gemini vision — only page.accessibility.snapshot()
  - Every action logged with a11y before/after state
  - done() requires evidence citing inspect() results
  - Outputs reproduction plan for Phase 2

Phase 2 (qa-reproduce.ts): Deterministic replay of research plan.
  - Executes each step with a11y assertions
  - Gemini describes visual changes (narration for humans)
  - Clean focused video with subtitles

Phase 3: Report job reads research-log.json for verdict (ground truth),
  narration-log.json for descriptions, video for visuals.
  Gemini formats logs into report — never determines verdict.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
5541c9aaea fix: prevent AI lies — assertion-based verdicts + blind reviewer
Agent: MUST use inspect() after every action, verdict based on DOM
state not opinions. "NEVER claim REPRODUCED unless inspect() confirms."

Reviewer: Two-phase prompt — Phase 1 describes what it SEES (blind,
no context). Phase 2 compares observations against issue/PR context.
Anti-hallucination rules: "describe ONLY what you observe, NEVER infer."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
ec29d5a828 feat: Agent SDK auto-detects Claude Code session — no API key needed locally
ANTHROPIC_API_KEY is optional: Agent SDK uses Claude Code OAuth
session when running locally (detects CLAUDE_CODE_SSE_PORT).
In CI, ANTHROPIC_API_KEY from secrets is used.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
a96f836a00 refactor: require ANTHROPIC_API_KEY, remove Gemini-only fallback
The Gemini-only agentic loop had ~47% success rate — too low to be
useful as a fallback. Now ANTHROPIC_API_KEY is required for issue
reproduction. Fails clearly if missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
80db4a1ac6 fix: inject cursor overlay via addScriptTag after login, not addInitScript
addInitScript runs before page load — Vue's app mount destroys the
cursor div when it takes over the DOM. Using addScriptTag after login
ensures the cursor persists in the stable DOM.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
9db6463b96 fix: cursor overlay on locator clicks (clickByText, menu items)
Locator.click/hover bypasses our page.mouse monkey-patch. Now
clickByText, hoverMenuItem, clickSubmenuItem get the element
bounding box and update cursor overlay manually.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
6761c391b7 fix: verdict JSON grep pattern — capture value without closing quote
The grep \{"verdict":\s*"[^"]+ captures up to but not including the
closing quote. The second grep for "[A-Z_]+"$ then fails because
there's no closing quote. Fixed: match "verdict":\s*"[A-Z_]+ then
extract [A-Z_]+$ (no quotes needed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
bfc0931260 fix: grep pipefail crash + add QA troubleshooting doc
- Add || true to all grep pipelines in deploy script (grep returns 1
  on no match, pipefail kills script)
- Add docs/qa/TROUBLESHOOTING.md covering all failures encountered:
  __name errors, zod/v4 imports, model IDs, badge mismatches, cursor,
  loadDefaultWorkflow, pressKey timing, agent behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
be92dee046 feat: structured JSON verdict from AI reviewer, light-first theme
- Video review prompt now requests a ## Verdict JSON block:
  {"verdict": "REPRODUCED|NOT_REPRODUCIBLE|INCONCLUSIVE", "risk": "low|medium|high"}
- Deploy script reads JSON verdict first, falls back to grep
- Eliminates all regex-matching false positives permanently
- Theme: light mode is default, dark via prefers-color-scheme:dark
- Cards use solid backgrounds, grain overlay only in dark mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
6e60706656 feat: report site follows system light/dark theme
Add prefers-color-scheme:light media query with light palette.
Replace hardcoded dark oklch values with CSS variables.
Light mode: white surfaces, dark text, subtle borders, no grain overlay.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
40b5b0ca4a fix: loadDefaultWorkflow uses API instead of menu, pressKey uses instant press
- loadDefaultWorkflow now calls app.resetToDefaultWorkflow() via JS API
  instead of navigating File → Load Default menu (menu item name varies)
- pressKey reverted to instant press() — the 400ms hold via down/up
  prevented Escape from propagating to parent dialog (#10397 BEFORE video
  showed wrong behavior because hold intercepted the event)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
0a9ca0c0ba fix: monkey-patch page.mouse for universal cursor overlay
Instead of manually calling moveCursorOverlay in each action,
patch page.mouse.move/click/dblclick/down/up globally. Now EVERY
mouse operation shows the cursor — text clicks, menu hovers, etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
159fdee471 fix: use gemini-3-flash-preview in hybrid agent (not 2.5 preview)
Gemini 2.5 preview models return 404. Always use gemini-3+ models.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
eac491c6b1 fix: add 'could not be confirmed' to negative verdict patterns
"could not be confirmed" contains "confirmed" which matched the
positive reproduc|confirm check. Now caught by the negative check first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
4bfb059696 fix: correct Claude model ID — claude-sonnet-4-6 (not dated suffix)
The Agent SDK returned "model not found" for claude-sonnet-4-6-20250514.
Correct ID is claude-sonnet-4-6.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
99343cdef8 fix: cursor overlay now controlled via __moveCursor, not DOM events
Headless Chrome's Playwright CDP doesn't trigger DOM mousemove events
reliably. Now executeAction calls __moveCursor(x,y) directly after
every mouse.move/click/drag. Cursor is an SVG arrow (white + outline).
Click state shown via scale animation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
17776a850f feat: badge label includes QA date — #10397 QA0327
Shows when the QA was run so stale results are obvious at a glance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
00dc89dad7 fix: make key presses visible in video — hold + subtitle
pressKey now uses keyboard.down/up with 400ms hold instead of
instant press(). Shows subtitle "⌨ Escape" and the keyboard HUD
catches the held state for video frame capture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
0984ec2706 fix: use zod instead of zod/v4 — project zod doesn't export /v4 subpath
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
742216ad18 fix: add claude-agent-sdk to workspace catalog
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:07 +00:00
snomiao
6db13752a0 feat: control/test comparison strategy + QA backlog doc
- Agent system prompt now instructs Claude to demonstrate BOTH working
  (control) and broken (test) states when bug is triggered by a setting
- Added docs/qa/backlog.md with future improvements: Type B/C comparisons,
  TTS, pre-seeding, cost optimization, environment-dependent issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
9809d52ac6 feat: all badges use vertical box style
Drop horizontal badges. Universal box badge shows:
  ┌──────────────────┐
  │    #7414 QA       │
  │ ✓ 1 reproduced   │
  │ ⚙ Fix: APPROVED  │  ← only for PRs
  └──────────────────┘

Issues show repro/not-repro/inconclusive rows.
PRs add a fix quality row (APPROVED/MINOR/MAJOR).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
1a25756290 feat: show QA pipeline commit hash + timing on report site
- Shows "QA @ abc1234" linking to the pipeline code commit
- Shows start time → deploy time in header
- Helps trace which version of QA scripts generated each report

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
83204b9a67 feat: hybrid QA agent — Claude Sonnet 4.6 brain + Gemini vision
Architecture:
- Claude Sonnet 4.6 plans and reasons (via Claude Agent SDK)
- Gemini 2.5 Flash watches video buffer and describes what it sees
- 4 tools: observe(), inspect(), perform(), done()

observe(seconds, focus): builds video clip from screenshot buffer,
  sends to Gemini with Claude's focused question.
inspect(selector): searches a11y tree for specific element state.
perform(action, params): executes Playwright action.
done(verdict, summary): signals completion.

Falls back to Gemini-only loop if ANTHROPIC_API_KEY not set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
d78388c893 feat: pass OPENAI_API_KEY to recording step for TTS narration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
eeaedf9854 feat: subtitle overlay + OpenAI TTS narration on reproduce videos
- Agent reasoning shown as subtitle bar at bottom of video during recording
- After recording, generates TTS audio via OpenAI API (tts-1, nova voice)
- Merges audio clips at correct timestamps into the video with ffmpeg
- Requires OPENAI_API_KEY env var; gracefully skips if not set
- No-sandbox + disable-dev-shm-usage for headless Chrome compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
3f1a580d35 feat: show test requirements from QA guide on report site
- Download QA guide artifact in report job
- Extract prerequisites, test focus, and steps from guide JSON
- Display below the purpose description: focus → prerequisites → steps
- Separated by a subtle divider with smaller font

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
6bd6e08195 feat: purpose description on report + multi-pass video link fix
- Report site shows "PR #N aims to..." or "Issue #N reports..." block
  above the video cards, extracted from pr-context.txt
- Multi-pass video links fall back to pass1 when qa-{os}.mp4 is 404
- More negative verdict patterns: "does not demonstrate", "never tested"
- Risk uses first word of Overall Risk (avoids "high confidence" match)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
8994f74b2f fix: verdict detection — more negative patterns, risk uses first word
- Add "does not demonstrate", "steps were not performed", "never tested"
  to NOT_REPRO patterns (fixes #9101 false positive)
- Risk detection uses first word of Overall Risk section instead of
  grepping entire text (fixes "high confidence" matching HIGH)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
72e8dedb9e fix: use addScriptTag for keyboard HUD to avoid tsx __name issue
tsx compiles arrow functions with __name helpers that don't exist in
browser context. Using addScriptTag with plain JS string avoids this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
9e16b16c55 fix: remove TS type annotation from page.evaluate (browser context)
Set<string>() in page.evaluate causes __name ReferenceError in browser.
Use untyped Set() since browser JS doesn't support TS generics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
6c284c00eb debug: add verdict count logging to deploy script 2026-03-31 07:54:06 +00:00
snomiao
6dc97d765d fix: check negative verdicts before positive in per-report classification
"fails to reproduce" contains "reproduce" — must check negatives first
within each report. Across reports, REPRODUCED still wins (multi-pass).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
11e4de9c45 feat: keyboard HUD overlay shows pressed keys in video
Injects a persistent overlay in bottom-right corner that displays
currently held keys (e.g. "⌨ Space", "⌨ CTRL+C"). Makes keyboard
interactions visible in the recording for both human and AI reviewers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
4f5e92d03b feat: vertical box badge for multi-pass with breakdown
Multi-pass issues show a stacked box badge:
  ┌──────────────┐
  │  #7806 QA    │
  │ ✓ 1 reproduced    │
  │ ⚠ 1 inconclusive  │
  └──────────────┘

Single-pass issues keep the standard horizontal badge.
Badge colors: blue=reproduced, gray=not-repro, yellow=inconclusive.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
f6de96580e feat: show pass counts in badge for multi-pass reports (X/Y REPRODUCED)
When multiple report files exist, badge shows "2/3 REPRODUCED" instead
of just "REPRODUCED". Single-pass issues still show plain verdict.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
7835eda6d4 fix: REPRODUCED wins over INCONCLUSIVE in multi-pass badge
When multiple passes exist and one confirms while another is
inconclusive, the badge should show REPRODUCED. Previously
INCONCLUSIVE was checked first, hiding successful reproductions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
09d6ab881b fix: cloneNode uses Ctrl+C/V instead of right-click Clone menu
The "Clone" context menu item doesn't exist in Nodes 2.0 mode.
Using Ctrl+C/Ctrl+V works in both legacy and Nodes 2.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
ff91c8a414 fix: preflight performs actual repro steps, not just setup
- #10307: preflight clones KSampler node, hint says drag to overlap
- #7414: preflight clicks numeric widget, hint says drag to change value
- #7806: preflight takes baseline screenshot, hint gives exact coords
  for holdKeyAndDrag with spacebar
- Hints now reference "Preflight already did X, NOW do Y" pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
36e77cbaf4 fix: tell agent what preflight already did to prevent repeated actions
Agent was wasting turns re-doing loadDefaultWorkflow and setSetting
that preflight already executed. Now the system prompt includes
"Already Done" section listing preflight actions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
83aa03b3e5 fix: preflight actions + badge false-positive pattern
- Auto-execute prerequisite actions (enable Nodes 2.0, load default
  workflow) BEFORE the agentic loop starts. Agent model ignores prompt
  hints but preflight guarantees nodes are on canvas.
- Add "fails to reproduce" to NOT REPRODUCIBLE badge patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
1f9b1e2d6c feat: qa-issue label trigger + labels in issue context
- Add issues:[labeled] trigger and qa-issue label support
- Resolve github.event.issue.number for issue-triggered runs
- Include issue labels in context (feeds keyword matcher for hints)
- Remove qa-issue label after run completes (same as qa-changes/qa-full)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
dd45547d48 fix: keyword-driven action hints for agent issue reproduction
Scan issue context for keywords (clone, copy-paste, spacebar, resize,
sidebar, scroll, middle-click, node shape, Nodes 2.0, etc.) and inject
specific MUST-follow action steps into the agentic system prompt.

Addresses 9 INCONCLUSIVE issues where agent had actions available
but didn't know when to use them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
7538101ea3 fix: restyle QA annotations to avoid misleading AI reviewer
- Annotations now use cyan dashed border + monospace "QA:" prefix
  instead of red solid labels that look like UI error messages
- Video review prompts explicitly tell reviewer to ignore QA annotations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
8cd6ed4fbc fix: badge mismatch, multi-pass report overwrite, agent node creation
P1: Filter out QA bot's own comments from pr-context (INCONCLUSIVE loop)
P2: Grep only ## Summary section for verdict (false REPRODUCED fix)
P3: Strip markdown bold before matching Overall Risk section
P4: Deploy full placeholder page with spinner during CI
P5: Pass #NUM QA label to PREPARING/ANALYZING badges
P6: Add copyPaste, holdKeyAndDrag, resizeNode, middleClick actions
P7: preload=auto + custom seekbar (already deployed)
P8: Deploy FAILED badge on report job failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:06 +00:00
snomiao
6d086d7062 fix: video seeking — preload=auto, custom seekbar, _headers
- Change preload=metadata to preload=auto for full video download
- Add _headers file with Accept-Ranges for Cloudflare Pages
- Add custom seekbar (range input + buffer indicator) that works
  even without server HTTP range request support
- Seekbar shows buffered progress and allows dragging to any point

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
0227ef70bc fix: strengthen after-mode prompt to test PR-specific behavior
Previous prompt said "test the specific behavior" which was too vague,
leading to generic UI walkthroughs instead of targeted tests.

New prompt: explicitly instructs to read the diff, trigger the exact
scenario the PR fixes, and avoid generic menu screenshots.

Also added reload action to before/after prompt for state persistence tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
70252000c6 fix: badge mismatch, multi-pass report overwrite, agent node creation
- Fix quality badge now reads "## Overall Risk" section only
- Prevents false MAJOR ISSUES from severity labels or negated phrases
- "Low" risk → APPROVED, "High" → MAJOR ISSUES, "Medium" → MINOR ISSUES

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
93110911ef fix: install pnpm before building PR frontend in sno-qa-* triggers
setup-frontend must run first to install node/pnpm, then rebuild
with PR code. Also re-install sno-skills deps after switching back
so QA scripts' dependencies are available.

Also gitignore .claude/scheduled_tasks.lock.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
3b4a07cba1 fix: restore sno-skills scripts after building PR frontend
When triggered via sno-qa-* push, the workflow checks out the PR code
to build its frontend, but this replaces qa-record.ts which only
exists on sno-skills. Fix: build PR frontend, then checkout back to
sno-skills so QA scripts remain available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
ea603bd210 fix: use array subshell instead of mapfile for macOS compat
mapfile is not available on macOS default bash.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
d51d749bbd fix: resolve remaining shellcheck warnings in qa-deploy-pages.sh
- SC2231: quote glob expansions in for loop
- SC2002: use sed directly instead of cat | sed
- SC2086: quote variable in echo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
0d556e9d9f feat: clickable badges with #NUM label and copy button
- Badge generators accept optional label param (#NUM QA)
- Badge in PR/issue comments links to report site
- Report site shows badge with copy-to-clipboard button
- Copy button produces markdown: [![QA Badge](url/badge.svg)](url/)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
aec47a6087 fix: resolve CI failures — test, shellcheck, format
- Update DefaultThumbnail test to match size-full class change
- Fix shellcheck warnings in qa-batch.sh (SC2001, SC2207)
- Fix shellcheck warnings in qa-deploy-pages.sh (SC2034, SC2235, SC2231, SC2002)
- Add qa-report-template.html to oxfmt ignore (minified, not formattable)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:54:05 +00:00
snomiao
f1a37110f6 fix: badge pattern too narrow/broad, multi-pass video discovery
- "confirmed" didn't match "confirms"/"reproducible" — use "reproduc|confirm" stem
- "partial" matched unrelated text — require "partially reproduced" specifically
- collectVideoCandidates now finds qa-session-*.mp4 for multi-pass reviews

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
e3ca8a5ff5 fix: badge mismatch, multi-pass report overwrite, agent node creation
- Check INCONCLUSIVE before reproduced/confirmed in badge detection
- Exclude markdown headings from reproduced grep match
- Add --pass-label to qa-video-review.ts for unique multi-pass filenames
- Pass pass label from workflow YAML when reviewing numbered sessions
- Collect all pass-specific reports in deploy script HTML
- Add addNode/cloneNode convenience actions to qa-record agent
- Improve strategy hints for visual/rendering bug reproduction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
89aa30cd5f chore: tidy PR for merge — resolve TODOs, fix misplaced import
- Remove push trigger (was for dev testing only)
- Restore concurrency group (was commented out for dev)
- Move misplaced import in qa-analyze-pr.ts to top of file

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
42787a1c71 fix: extract deploy step into script to fix expression length limit
The Cloudflare Pages deploy step exceeded GitHub Actions' 21000 char
expression limit due to inline HTML/CSS/JS. Extract to
scripts/qa-deploy-pages.sh + scripts/qa-report-template.html.
2026-03-31 07:26:31 +00:00
snomiao
b11d054e8a fix: enforce menu navigation pattern + add CI job link to report
- Strengthen prompt: MUST use openMenu → hoverMenuItem → clickMenuItem
  in that order. Previous runs skipped openMenu causing silent failures.
- Add CI Job link to the QA report site header for quick navigation
  to the GitHub Actions run that generated the report.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
8db85802df fix: split dual badge generator into separate step to fix expression length
GitHub Actions has a 21000 char limit per expression. The combined
badge setup step exceeded this after adding the dual badge generator.
Split into its own step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
29ae28577b chore: retrigger QA pipeline 2026-03-31 07:26:31 +00:00
snomiao
c03d0436bc fix: remove Bug: prefix from PR badge, just show REPRODUCED directly
Badge now reads: QA Bot | REPRODUCED | Fix: APPROVED
Not all issues are bugs — could be feature requests too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:31 +00:00
snomiao
28798a658b fix: use blue for REPRODUCED badge (success, not failure)
Reproducing a bug is a successful outcome for the QA bot.
Blue (#2196f3) = bot succeeded. Red = bot found problems with the fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
b1895fe068 feat: combine PR bug+fix into single dual-segment badge
PRs now show one badge with three segments:
  QA Bot | Bug: REPRODUCED | Fix: APPROVED

Instead of two separate badges. Uses gen-badge-dual.sh which
renders label + bug status + fix status in one SVG.

Issues still use single two-segment badge:
  QA Bot | FINISHED: REPRODUCED

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
aff609c558 feat: add visible cursor overlay and annotation action to QA recorder
- Inject fake cursor (red dot with click animation) via addInitScript
  since headless Chrome doesn't render the system cursor in video
- Add hover-before-click delay to clickByText and canvas clicks
  so viewers can see where the cursor moves before clicking
- Add 'annotate' action: shows a floating label at (x,y) for N ms
  so AI can draw viewer attention to important UI state changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
d353cba489 fix: feed QA guide + issue context to agentic reproduce loop
Root cause: runAgenticLoop never read the QA guide — agent saw
"No issue context provided" for issues. Now reads qaGuideFile,
parses structured fields, and injects into system prompt.

Also: fetch issue body via gh issue view in workflow, increase
budget to 120s/30 turns, add focus reminders, smarter stuck
detection (50px grid normalization + action-type frequency),
reject invalid click targets, add loadDefaultWorkflow and
openSettings convenience actions, strategy hints in prompt.

Fix pre-existing typecheck error in eslint.config.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
fe9d145ead feat: dual badges for PRs — bug reproduction + fix quality
PRs now get two separate badges:
- Bug: REPRODUCED / NOT REPRODUCIBLE / PARTIAL (before branch)
- Fix: APPROVED / MAJOR ISSUES / MINOR ISSUES (after branch)

Issues keep a single badge: FINISHED: REPRODUCED / etc.

Both badge-bug.svg and badge-fix.svg served from the deploy site.
PR comment shows all three: ![badge] ![bug] ![fix]

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
928db261f4 fix: badge FINISHED state includes result sub-state
FINISHED is not standalone — always shows result:
- FINISHED: REPRODUCED / NOT REPRODUCIBLE / PARTIAL (issues)
- FINISHED: APPROVED / MAJOR ISSUES / MINOR ISSUES (PRs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
861cb7a782 feat: add SVG status badge to QA report site
Badge shows QA pipeline status, deployed at each stage:
- PREPARING (blue) — setting up artifacts
- ANALYZING (orange) — running video review
- Final status with color:
  - Issues: REPRODUCED (red) / NOT REPRODUCIBLE (gray) / PARTIAL (yellow)
  - PRs: APPROVED (green) / MAJOR ISSUES (red) / MINOR ISSUES (yellow)

Badge served as /badge.svg from the same Cloudflare Pages site.
Included in PR comment as ![QA Badge](url/badge.svg).

Also restore @ts-expect-error for import-x plugin type incompatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
GitHub Action
f98a11e473 [automated] Apply ESLint and Oxfmt fixes 2026-03-31 07:26:30 +00:00
snomiao
b67bdaaa86 fix: regenerate pnpm-lock.yaml after rebase
The rebase introduced a duplicated mapping key in the lockfile,
causing ERR_PNPM_BROKEN_LOCKFILE in CI.
2026-03-31 07:26:30 +00:00
snomiao
6024d05d04 fix: use correct flash model name for agentic loop 2026-03-31 07:26:30 +00:00
snomiao
aa0c020e89 feat: agentic screenshot feedback loop + multi-pass recording
Replace single-shot step generation in reproduce mode with an agentic
loop where Gemini sees the screen after each action and decides what
to do next. For multi-bug issues, decompose into sub-issues and run
separate recording passes.

- Extract executeAction() from executeSteps() for reuse
- Add reload and done action types
- Add captureScreenshotForGemini() (JPEG q50, ~50KB)
- Add runAgenticLoop() with sliding window history (3 screenshots)
- Add decomposeIssue() for multi-pass recording (1-3 sub-issues)
- Update workflow to handle numbered session videos (qa-session-1, etc.)
2026-03-31 07:26:30 +00:00
snomiao
7170918fd7 feat: add frame-by-frame video controls to QA report player
- Add custom video controls below each video with frame stepping
- Frame back/forward buttons (1 frame at 30fps, 10 frames skip)
- Speed selector: 0.1x, 0.25x, 0.5x (default), 1x, 1.5x, 2x
- Keyboard shortcuts: arrow keys for frame step, space for play/pause
- SMPTE-style timecode display (m:ss.ms)
- Default 0.5x speed since AI operates UI faster than humans
- Videos no longer autoplay (pause on load for inspection)
- Zero external dependencies (pure HTML5 video API)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
22807604e4 fix: cap reproduce video at 5min, skip env setup in Phase 4
- Reproduce video must be max 5 minutes (short, focused demo)
- Phase 4 reuses the environment from Phase 3 (no re-setup)
- Use video-start/video-stop commands (not --save-video flag)
- Start recording right before steps, stop immediately after

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
13acd91e33 feat: add reproduce-issue skill with two-video architecture
New Claude agent-driven issue reproduction skill that:
- Phase 1-2: Research issue and set up environment (custom nodes, workflows, settings)
- Phase 3: Record research video while exploring interactively via playwright-cli
- Phase 4: Record clean reproduce video with only the minimal repro steps
- Phase 5: Generate structured reproduction report

Key difference from the old approach: Claude agent explores and adapts
instead of blindly executing a Gemini-generated static plan.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
ac8ff65cd9 fix: add default workflow node positions to QA prompts
Gemini was right-clicking empty canvas instead of nodes because it
didn't know where the default workflow nodes are positioned. Now the
prompt includes approximate coordinates for all 7 default nodes and
clarifies the difference between node context menu vs canvas menu.

Also fixes TS2352 in page.evaluate by using double-cast through unknown.
2026-03-31 07:26:30 +00:00
snomiao
cc7ec6fd32 feat: upgrade QA pipeline to Gemini 3.x models
- qa-record.ts, qa-analyze-pr.ts: gemini-2.5-flash/pro → gemini-3.1-pro-preview
- qa-video-review.ts, qa-generate-test.ts: gemini-2.5-flash → gemini-3-flash-preview
- pr-qa.yaml: update hardcoded model reference
- Add docs/qa/models.md with model comparison and rationale
2026-03-31 07:26:30 +00:00
snomiao
6d5110364a feat: add loadWorkflow and setSetting actions to QA recorder
- Add loadWorkflow action to load workflow JSON from URL
- Add setSetting action to configure ComfyUI settings
- Improve reproduce mode prompt to emphasize establishing prerequisites
  (save workflow first, make it dirty, add needed nodes, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
fe107d1f20 feat: add clickable issue/PR URLs to QA reports
- Add --target-url CLI option to qa-video-review.ts
- Include target URL in generated markdown reports
- Add clickable issue/PR link in deployed HTML report header
- Workflow passes the target URL automatically

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
5e354b118c fix: resolve pre-existing typecheck errors
- Remove unused @ts-expect-error directives in eslint.config.ts
- Simplify LazyImage prop types from ClassValue to string
- Fix DialogInstance to avoid infinitely deep type instantiation
- Use cn() in DefaultThumbnail for class merging

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
32fbe17e68 fix: improve fillDialog and clickSubmenuItem for litegraph UI components
- fillDialog now tries: PrimeVue dialog → node search box → focused input → keyboard fallback
- clickSubmenuItem now tries: PrimeVue tiered menu → litegraph context menu → role menuitem
- Fixes double-click-to-add-node flow and right-click context menu clicks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
fb9229620b fix: add step-level error resilience and click timeout in QA recording
- Wrap each step in try/catch so failed steps don't abort the recording
- Add 5s timeout to clickByText to prevent 30s hangs on disabled elements

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
b15097a44f feat: expand QA action set and improve issue reproduction depth
- Add new canvas actions: rightClick, doubleClick, clickCanvas,
  rightClickCanvas, dragCanvas, scrollCanvas for node graph interactions
- Increase reproduce mode step limit from 3-6 to 8-15 steps
- Add ComfyUI UI context to prompts (canvas layout, node interactions)
- Add anti-hallucination instructions to video review for issue mode
- Improve issue analysis prompt with detailed action descriptions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:30 +00:00
snomiao
2c08a98117 fix: use pulls API instead of gh pr view for PR/issue detection
gh pr view can't distinguish PRs from issues — it succeeds for both.
Use the REST API endpoint repos/{owner}/{repo}/pulls/{number} which
returns 404 for issues.
2026-03-31 07:26:29 +00:00
snomiao
836492e3dc fix: address Copilot review feedback on QA scripts
- Enforce requestTimeoutMs via Gemini SDK requestOptions
- Add 100MB video size check before base64 encoding
- Sanitize screenshot filenames to prevent path traversal
- Sort video files by mtime for reliable rename
- Validate --mode arg against allowed values
- Add Content-Length pre-check in downloadMedia
- Add GitHub domain allowlist for media downloads (SSRF mitigation)
- Add contents:write permission and git config for report job
- Update Node.js requirement in SKILL.md from 18+ to 22+
2026-03-31 07:26:29 +00:00
snomiao
f78f44fea1 fix: gracefully skip invalid pressKey values in QA recording
Instead of crashing the entire recording session when Gemini generates
an invalid key name (e.g. "mouseWheelDown"), catch the error and
continue with remaining steps.
2026-03-31 07:26:29 +00:00
snomiao
0332f706fa feat: add behavior changes summary table to QA video review
Add a "Behavior Changes" table (Behavior, Before, After, Verdict)
alongside the existing timeline comparison. This gives reviewers a
quick high-level view of all behavioral differences before diving
into the frame-by-frame timeline.
2026-03-31 07:26:29 +00:00
snomiao
84a7b8ed62 fix: format before/after comparison as table in QA video review
Instruct Gemini to output the Before vs After section as a markdown
table with Time, Type, Severity, Before, After columns for easier
comparison. Update HTML template table styles with fixed layout and
column widths optimized for the 5-column comparison format.
2026-03-31 07:26:29 +00:00
snomiao
dbae35db0c fix: handle array response from Gemini in analyze-pr
Gemini Pro with responseMimeType: 'application/json' returns a JSON
array [before, after] instead of {before, after}. Handle both shapes.
2026-03-31 07:26:29 +00:00
snomiao
1adcd2aaa2 fix: make analyze-pr non-blocking and log Gemini response
- Log raw Gemini response for debugging when parsing fails
- Handle possible wrapper keys in response
- Make qa-before/qa-after run even if analyze-pr fails (only gate
  on resolve-matrix success)
2026-03-31 07:26:29 +00:00
snomiao
d28949b2b9 fix: extract PR number from sno-qa-<number> branch name
When running on push events for sno-qa-* branches without an open PR,
extract the PR number from the branch name so analyze-pr can fetch
the full PR thread for analysis.
2026-03-31 07:26:29 +00:00
snomiao
7375f16afe feat: add analyze-pr job to QA pipeline
Add Gemini Pro-powered PR analysis that generates targeted QA guides
from the full PR thread (description, comments, screenshots, diff).
The analyze-pr job runs on lightweight ubuntu before recordings start,
producing qa-guide-before.json and qa-guide-after.json that are
downloaded by recording jobs to produce more focused test steps.

Graceful fallback: if analysis fails, recordings proceed without guides.
2026-03-31 07:26:29 +00:00
snomiao
280ff17c15 fix: download before/after artifacts into separate directories
download-artifact@v7 merges all files flat regardless of
merge-multiple setting. Use separate path dirs (before/after)
and copy all files into the report directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:29 +00:00
snomiao
cd9bbdae4c fix: set merge-multiple false for download-artifact v7
download-artifact@v7 defaults merge-multiple to true, which puts all
files flat in qa-artifacts/ instead of per-artifact subdirectories.
The merge step expects qa-artifacts/qa-before-{os}-{run}/ subdirs,
so the report directory never gets created and video review finds
no files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:29 +00:00
snomiao
36ce1fa406 refactor: split QA into parallel before/after jobs
Instead of running before/after sequentially in a single job with
fragile git stash/checkout gymnastics, split into two independent
parallel jobs on separate runners:

  resolve-matrix → qa-before (main) ─┐
                 → qa-after  (PR)   ─┴→ report

- qa-before: uses git worktree for clean main branch build
- qa-after: normal PR build via setup-frontend
- report: downloads both artifact sets, merges, runs Gemini review

Benefits:
- Clean workspace isolation (no git checkout origin/main -- .)
- ~2x faster (parallel execution)
- Each job gets its own ComfyUI server (no shared state)
- Eliminates entire class of workspace contamination bugs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:29 +00:00
snomiao
096f617f3b fix: select existing user from dropdown instead of re-creating
The Pre-seed step creates qa-ci via API, so the "New user" form
shows "already exists" error. Fix by selecting the existing user
from the dropdown first, falling back to a unique username.
2026-03-31 07:26:29 +00:00
snomiao
5ced3a72bf fix: use login page directly instead of localStorage bypass
The localStorage userId bypass doesn't work because the server
validates user IDs and rejects the simple 'qa-ci' string. Instead,
detect the login page by its input fields and create a user via the
"New user" text input, which is how real users would log in.
2026-03-31 07:26:29 +00:00
snomiao
0bf8dbb130 fix: switch from Firefox to Chromium for WebGL canvas support
Firefox headless doesn't support WebGL, causing "getCanvas: canvas is
null" errors. Switch to Chromium which has full headless WebGL support.
Also fix login flow to wait for async router guard to settle and
create user via text input as fallback.
2026-03-31 07:26:29 +00:00
snomiao
5a4b9d923d fix: add debug screenshot and fallback for menu button click
Add coordinate fallback when .comfy-menu-button-wrapper selector isn't
found, and capture a debug screenshot after login to diagnose what the
page looks like when the editor UI fails to render.
2026-03-31 07:26:29 +00:00
snomiao
2a3f68333b fix: use proper CSS selectors for menu interactions in QA recording
The openComfyMenu was clicking at hardcoded coordinates (20, 67) which
missed the menu button. Now uses .comfy-menu-button-wrapper selector
matching the browser tests. Also fixes menu item hover/click selectors
to use .p-menubar-item-label and .p-tieredmenu-item classes, and adds
a wait for the editor UI to fully load before executing test steps.
2026-03-31 07:26:29 +00:00
snomiao
441263dc64 fix: bypass login in QA recordings with localStorage pre-seeding
The QA recordings were stuck on the user selection screen because CI
has no existing users. Fix by pre-seeding localStorage with userId,
userName, and TutorialCompleted before navigation, plus creating a
qa-ci user via API as a fallback.
2026-03-31 07:26:29 +00:00
snomiao
641aee03c0 feat: autoplay and loop videos on QA dashboard 2026-03-31 07:26:29 +00:00
snomiao
d63a9058fb fix: handle flat artifact layout when no report.md exists
The normalize step couldn't create the qa-report-* subdir because it
only looked for *-report.md files. Add fallback to detect webm files.
2026-03-31 07:26:29 +00:00
snomiao
9613fb9e93 fix: exclude known video names when renaming Playwright recordings
The AFTER step was renaming qa-before-session.webm instead of the new
recording. Filter out already-named files before picking the latest.
2026-03-31 07:26:29 +00:00
snomiao
d5f22fcb1e fix: dismiss dropdown overlay before clicking Next in QA login flow
The user dropdown shows "No available options" in CI, and the overlay
blocks the Next button. Dismiss with Escape before attempting click.
2026-03-31 07:26:29 +00:00
snomiao
8402c63f99 fix: install main branch deps before building, then reinstall PR deps
Main branch imports @vueuse/router which isn't in PR's node_modules.
Need both: main deps for building main, PR deps for running QA scripts.
2026-03-31 07:26:29 +00:00
snomiao
588f19a774 fix: reinstall PR deps after main branch build to restore @google/generative-ai
The main branch build step was running pnpm install with main's lockfile,
which removed @google/generative-ai from node_modules. Move the reinstall
to after restoring PR files so the QA recording script can find its deps.
2026-03-31 07:26:29 +00:00
snomiao
e5f358057c fix: temporarily disable concurrency group to unstick QA runs 2026-03-31 07:26:28 +00:00
snomiao
e8e1e0eb32 fix: restore PR files with git checkout HEAD instead of git checkout -
git checkout - uses @{-1} which requires a previous branch switch.
Since we use 'git checkout origin/main -- .' (file checkout, not branch
switch), there is no @{-1} ref. Use HEAD to restore from current branch.

Also restore proper concurrency group.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
2f59c85469 fix: use unique concurrency group to unstick QA runs 2026-03-31 07:26:28 +00:00
snomiao
a8cd483d83 ci: trigger QA run 2026-03-31 07:26:28 +00:00
snomiao
9f4b14ab9b feat: integrate comfy-qa skill test plan into QA recording pipeline
Pass the comprehensive test plan from .claude/skills/comfy-qa/SKILL.md
to Gemini when generating test steps. This gives Gemini knowledge of all
12 QA categories (canvas, menus, sidebar, settings, etc.) so it picks
the most relevant tests for each PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
f560eb8fb4 fix: use vite build directly to skip nx typecheck dependency
nx build runs typecheck as a prerequisite (via @nx/vite/plugin config).
Use vite build directly for the main branch comparison build.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
3311d12546 fix: skip typecheck when building main branch for QA comparison
Main branch may have transient TS errors when built with the PR
branch's lockfile. Since we only need the dist for visual comparison,
run nx build directly instead of pnpm build (which includes typecheck).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
4e5f683185 refactor: replace Codex with direct Playwright recording in QA pipeline
Replace the unreliable codex exec approach with a Playwright script
(qa-record.ts) that uses Gemini to generate targeted test steps from
the PR diff, then executes them deterministically via Playwright's API.

Key changes:
- New scripts/qa-record.ts: Gemini generates JSON test actions, Playwright
  executes them with reliable helper functions (menu nav, dialog fill, etc.)
- Remove codex CLI and playwright-cli dependencies
- Remove 150+ lines of prompt templates from pr-qa.yaml
- Firefox headless with video recording (same approach proven locally)
- Fallback steps if Gemini fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
6993a7ad5f feat: auto-generate regression tests from QA reports
- Tighten BEFORE prompt to 15s snapshot (show old state only)
- Add qa-generate-test.ts: Gemini-powered Playwright test generator
- New workflow step: generate .spec.ts and push to {branch}-add-qa-test
- Tests assert UIUX behavior (tab names, dirty state, visibility)
2026-03-31 07:26:28 +00:00
snomiao
eb0ce5ed4e feat: before/after video comparison for QA pipeline
- Build both main (dist-before/) and PR (dist/) frontends in focused mode
- Run QA twice: BEFORE on main branch frontend, AFTER on PR branch
- Send both videos to Gemini in one request for comparative analysis
- Side-by-side dashboard layout with Before (main) / After (PR) panels
- Comparative prompt evaluates whether before confirms old behavior
  and after proves the fix works
- Falls back to single-video mode when no before video available
2026-03-31 07:26:28 +00:00
snomiao
06e4a512f0 fix: make QA videos seekable with faststart and frequent keyframes
moov atom was at end of file (8.6MB offset) — browser had to download
the entire video before seeking. Keyframes were only every 10 seconds.

Add -movflags +faststart (moov before mdat) and -g 60 (keyframe every
2.4s at 25fps) to ffmpeg conversion.
2026-03-31 07:26:28 +00:00
snomiao
5ad94cfaef fix: issue cards instead of dense table, rename to comfy-qa.pages.dev
- Replace 6-column confirmed issues table with vertical card blocks
  using colored severity/timestamp/confidence badges
- Rename Cloudflare Pages project from comfyui-qa-videos to comfy-qa
2026-03-31 07:26:28 +00:00
snomiao
7d34ffe6d4 fix: seekable video, hide empty cards, PR-aware video review
- Remove autoplay/loop so video timeline is seekable
- Skip card generation for platforms without recordings
- Add --pr-context flag to qa-video-review.ts so Gemini evaluates
  against PR purpose instead of just describing what happened
- Workflow now builds pr-context.txt from PR title/body/diff
2026-03-31 07:26:28 +00:00
snomiao
d0dc720698 feat: redesign QA dashboard with modern frontend design
OKLCH color tokens, liquid glass card surfaces, Inter + JetBrains Mono
typography, grain texture overlay, staggered fade-up animations, pill
action buttons with SVG icons, and improved report table styling.
2026-03-31 07:26:28 +00:00
snomiao
1a7ea13660 fix: make settings pre-seed non-fatal and try both API endpoints
The /api/settings endpoint returned 4xx in CI. Try both /api/settings
and /settings endpoints, and don't fail the job if neither works.
2026-03-31 07:26:28 +00:00
snomiao
cac0660de4 fix: pre-seed Comfy.TutorialCompleted to skip template gallery in QA
The Codex agent was spending 35s browsing the "Getting Started" template
gallery instead of testing the PR's changes. Pre-seeding this setting
via the ComfyUI API ensures the agent lands directly in the graph editor.
2026-03-31 07:26:28 +00:00
snomiao
421bf2ac9e fix: tighten focused QA prompt to only test PR-specific behavior
The Codex agent was spending time on login flow, template browsing,
and general smoke testing instead of testing the PR's actual changes.

Changes:
- Add 30-second time budget for video recording
- Move video-start AFTER login and editor verification
- Explicitly prohibit template browsing and sidebar exploration
- Reduce test steps to 3-6 targeted actions
- Restructure prompt with clear Instructions/Rules sections
2026-03-31 07:26:28 +00:00
snomiao
6bb2f0346f fix: render markdown in QA reports with marked.js
Replace crude sed-based markdown conversion with client-side
rendering via marked.js CDN. Adds proper table, list, and
code styling for the report section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
74dbf84e4b fix: run report job on workflow_dispatch events
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
76c6efcc50 refactor: replace GPT frame extraction with Gemini native video analysis
Replace the OpenAI GPT-based frame extraction approach (ffmpeg + screenshots)
with Gemini 2.5 Flash's native video understanding. This eliminates false
positives from frame-based analysis (e.g. "black screen = critical bug" during
page transitions) and produces dramatically better QA reviews.

Changes:
- Remove ffmpeg frame extraction, ffprobe duration detection, and all related
  logic (~365 lines removed)
- Add @google/generative-ai SDK for native video/mp4 upload to Gemini
- Update CLI: remove --max-frames, --min-interval-seconds, --keep-frames flags
- Update env: OPENAI_API_KEY → GEMINI_API_KEY
- Update workflow: swap API key secret and model in pr-qa.yaml
- Update report: replace "Frames analyzed" with "Video size"
- Add note in prompt that brief black frames during transitions are normal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 07:26:28 +00:00
snomiao
e787340c16 fix: use fill+click for quick login before video recording
playwright-cli doesn't support 'evaluate' command. Instead, instruct
Codex to quickly fill the username input and click Next on user-select
page BEFORE starting video recording, so the video only shows actual
QA testing.
2026-03-31 07:26:28 +00:00
snomiao
ae7c423a7e fix: use evaluate to set localStorage before video recording
storageState config doesn't work with playwright-cli. Instead, use
evaluate to set Comfy.userId/userName after opening the page, then
navigate back. This skips user-select before video-start so the
recording only shows actual QA testing.
2026-03-31 07:26:28 +00:00
snomiao
a98893915b fix: pre-seed localStorage to skip user-select in QA runs
Write a Playwright storageState JSON with Comfy.userId/userName pre-set
so the app loads directly to the graph editor. Saves ~40s per QA run
that was wasted on navigating the user-select page.
2026-03-31 07:26:28 +00:00
snomiao
161dccd1ed fix: prefer explicit qa-session.webm over corrupt auto-recorded videos
The convert step was using find which picked up a 0-byte file from
playwright's videos/ directory instead of the valid qa-session.webm.
Now prefers qa-session.webm explicitly and skips empty files.
2026-03-31 07:26:27 +00:00
snomiao
bdcb28c298 fix: improve focused QA prompt to test PR-specific behavior, not random walk 2026-03-31 07:26:27 +00:00
snomiao
c75531a11e fix: re-add push trigger for sno-skills and sno-qa-* branches 2026-03-31 07:26:27 +00:00
snomiao
c3fc430cec fix: also install ffprobe for GPT video review frame extraction 2026-03-31 07:26:27 +00:00
snomiao
5859cdc588 fix: use sudo for ffmpeg static binary extraction to /usr/local/bin 2026-03-31 07:26:27 +00:00
snomiao
ac6ee64074 fix: use static ffmpeg binary instead of apt-get (avoids dpkg lock hang) 2026-03-31 07:26:27 +00:00
snomiao
8d71d273b9 fix: add ffmpeg install back (not pre-installed on GH runners) 2026-03-31 07:26:27 +00:00
snomiao
c9363e93e4 fix: normalize flat artifact download into expected subdirectory 2026-03-31 07:26:27 +00:00
snomiao
ab8ad12382 fix: pre-install chromium and clarify prompt for codex
Codex was using pnpm dlx instead of the global playwright-cli.
Pre-install chromium in setup step and make prompt explicit about
using the global command directly without pnpm/npx.
2026-03-31 07:26:27 +00:00
snomiao
a70e9b8f78 fix: add debug output to video convert step 2026-03-31 07:26:27 +00:00
snomiao
a1d1372097 fix: use danger-full-access sandbox for codex on GH Actions 2026-03-31 07:26:27 +00:00
snomiao
2bfbf8d83e fix: use correct codex model name gpt-5.4-mini 2026-03-31 07:26:27 +00:00
snomiao
cb885b24ef feat: switch QA from Claude Code to OpenAI Codex CLI
Replace claude --print with codex exec for cheaper QA runs.
Uses codex-mini-latest model ($1.50/$6 vs Sonnet $3/$15).
Uses existing OPENAI_API_KEY secret (no new secrets needed).
2026-03-31 07:26:27 +00:00
snomiao
ac62d0a57a fix: default to linux-only QA, full 3-OS only via qa-full label
Reduces per-run cost from ~$10-16 to ~$2.50 by defaulting to
Linux-only. Use qa-full label or workflow_dispatch for 3-OS runs.
2026-03-31 07:26:27 +00:00
snomiao
d5050ffaa4 fix: use explicit video-start/stop, remove ffmpeg install, use gpt-4.1-mini
- Replace saveVideo config (didn't produce video) with explicit
  playwright-cli video-start/video-stop commands in QA prompt
- Remove apt-get install ffmpeg step (pre-installed on GH runners)
- Switch video review model from gpt-4o to gpt-4.1-mini
2026-03-31 07:26:27 +00:00
snomiao
6522e74999 fix: use auto video recording and show GPT reports on QA site
- Enable saveVideo in playwright-cli config for real video recording
- Replace screenshot stitching with webm→mp4 conversion
- Move video review step before deploy so reports are included
- Add GPT video review reports inline on the Cloudflare Pages site
- Each video card now has expandable "GPT Video Review" section
2026-03-31 07:26:27 +00:00
snomiao
a4f1588a15 fix: configure playwright-cli outputDir and improve artifact collection
- Set .playwright/cli.config.json with outputDir pointing to screenshots/
- This way bare 'playwright-cli screenshot' auto-saves to the right place
- Create screenshot directory before Claude runs (don't rely on Claude)
- Collect step now searches working directory for stray PNGs
- Simplified prompt: no --filename needed, just 'playwright-cli screenshot'
2026-03-31 07:26:27 +00:00
snomiao
178f469948 fix: stitch screenshots from correct directory and simplify prompt
Screenshots were saved to artifact root but stitch looked in frames/.
Now: prompt tells Claude to save to screenshots/ dir with numbered names,
collect step consolidates PNGs there, stitch step globs from screenshots/.
Removed video-start/video-stop (Claude doesn't use them).
2026-03-31 07:26:27 +00:00
snomiao
41c9fe29dc fix: use playwright-cli video recording and collect default output
- Add playwright-cli config with outputDir and saveVideo
- Use video-start/video-stop instead of relying on screenshot frames
- Add fallback artifact collection from .playwright-cli/ default dir
- Simplify prompts to focus on video recording workflow
2026-03-31 07:26:27 +00:00
snomiao
a04d53a905 fix: resolve QA_ARTIFACTS path in prompt so Claude gets the literal path
The escaped \$QA_ARTIFACTS in the heredoc produced literal text
'$QA_ARTIFACTS' in the prompt. Claude's Bash tool didn't reliably
expand this env var, so no screenshots or reports were saved.
Remove the escapes so the heredoc expands the variable to the actual
path (e.g. /home/runner/work/_temp/qa-artifacts).
2026-03-31 07:26:27 +00:00
snomiao
8d933077ea fix: escape backticks in QA prompt heredoc to prevent command substitution
Backtick-wrapped playwright-cli examples in the unquoted heredoc were
being interpreted as bash command substitution, producing empty prompts.
Replace backtick syntax with plain "Run:" prefixed commands.
2026-03-31 07:26:27 +00:00
snomiao
0f51308196 fix: reorganize QA CI — remove screen recording, merge video-review into report
- Remove all Xvfb/ffmpeg screen recording infrastructure from qa job
  (captured blank display since playwright-cli runs headless)
- Add screenshot instructions to QA prompts: Claude saves sequential
  frames to $QA_ARTIFACTS/frames/ after every interaction
- Stitch screenshots into video via ffmpeg in report job (2fps)
- Merge video-review job into report job (4 jobs → 3 jobs)
- Unified PR comment with video links + video review in <details> collapse
- Clean up stale QA_VIDEO_REVIEW_COMMENT markers from prior runs
2026-03-31 07:26:27 +00:00
GitHub Action
d0dd211f89 [automated] Apply ESLint and Oxfmt fixes 2026-03-31 07:26:27 +00:00
snomiao
c98de44e98 fix: harden setup-comfyui-server against shell injection
Move extra_server_params input to env var to prevent shell injection
from untrusted input. Replace wait-for-it pip dependency with a
cross-platform curl polling loop.
2026-03-31 07:26:26 +00:00
snomiao
4622d0728c feat: add comfy-qa skill and automated QA CI pipeline
Add Claude Code skills and a label-triggered QA workflow:

- .claude/skills/comfy-qa/SKILL.md: 12-category QA test plan using
  playwright-cli for browser automation
- .github/workflows/pr-qa.yaml: CI workflow triggered by qa-changes
  (focused, Linux) or qa-full (3-OS matrix) labels. Records screen via
  ffmpeg, runs Claude CLI with playwright-cli, deploys video gallery to
  Cloudflare Pages, posts PR comment with GIF thumbnails, and runs
  OpenAI vision-based video review
- scripts/qa-video-review.ts: frame extraction + GPT-4o analysis
- scripts/qa-video-review.test.ts: unit tests for video review
- knip.config.ts: resolve knip errors for ingest-types package
2026-03-31 07:26:26 +00:00
2101 changed files with 41159 additions and 197156 deletions

View File

@@ -1,86 +1,9 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"if": "Bash(tsc *)",
"command": "echo 'Use `pnpm typecheck` instead of running tsc directly.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(vue-tsc *)",
"command": "echo 'Use `pnpm typecheck` instead of running vue-tsc directly.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx tsc *)",
"command": "echo 'Use `pnpm typecheck` instead of running tsc via npx.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpx tsc *)",
"command": "echo 'Use `pnpm typecheck` instead of running tsc via pnpx.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpm exec tsc *)",
"command": "echo 'Use `pnpm typecheck` instead of `pnpm exec tsc`.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx vitest *)",
"command": "echo 'Use `pnpm test:unit` (or `pnpm test:unit -- <path>`) instead of npx vitest.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpx vitest *)",
"command": "echo 'Use `pnpm test:unit` (or `pnpm test:unit -- <path>`) instead of pnpx vitest.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx eslint *)",
"command": "echo 'Use `pnpm lint` or `pnpm lint:fix` instead of npx eslint.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpx eslint *)",
"command": "echo 'Use `pnpm lint` or `pnpm lint:fix` instead of pnpx eslint.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx prettier *)",
"command": "echo 'This project uses oxfmt, not prettier. Use `pnpm format` or `pnpm format:check`.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpx prettier *)",
"command": "echo 'This project uses oxfmt, not prettier. Use `pnpm format` or `pnpm format:check`.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx oxlint *)",
"command": "echo 'Use `pnpm oxlint` instead of npx oxlint.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx stylelint *)",
"command": "echo 'Use `pnpm stylelint` instead of npx stylelint.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(npx knip *)",
"command": "echo 'Use `pnpm knip` instead of npx knip.' >&2 && exit 2"
},
{
"type": "command",
"if": "Bash(pnpx knip *)",
"command": "echo 'Use `pnpm knip` instead of pnpx knip.' >&2 && exit 2"
}
]
}
"permissions": {
"allow": [
"Bash(pnpx vitest run --testPathPattern=\"draftCacheV2.property\")",
"Bash(pnpx vitest run \"draftCacheV2.property\")",
"Bash(node -e \"const fc = require\\(''fast-check''\\); console.log\\(Object.keys\\(fc\\).filter\\(k => k.includes\\(''string''\\)\\).join\\('', ''\\)\\)\")"
]
}
}

View File

@@ -1,173 +0,0 @@
---
name: add-model-page
description: 'add, update, or remove a model page entry on the comfy org website. creates a PR to Comfy-Org/ComfyUI_frontend apps/website folder with the change and posts a Vercel preview link back to Slack.'
---
# add-model-page
add, update, or remove model pages in the ComfyUI website.
## Trigger phrases
- `Add a model page for <model-name>`
- `Update the model page for <model-name>`
- `Remove <model-name> from model pages`
## Phase 1 — Parse the request
Extract:
- **action**: `add` | `update` | `remove`
- **model-name**: raw string (e.g. `flux1-schnell`, `flux1_dev.safetensors`)
Normalize to a slug: lowercase, replace `_` and `.` with `-`, strip file extensions.
Example: `flux1_dev.safetensors``flux1-dev`
## Architecture overview
Models come from two sources merged at build time:
| File | Purpose |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `apps/website/src/config/generated-models.json` | Auto-generated from workflow_templates (slug, name, directory, huggingFaceUrl, workflowCount, displayName, thumbnailUrl, docsUrl) |
| `apps/website/src/config/model-metadata.ts` | Hand-curated overrides (docsUrl, blogUrl, featured) — only add entries that need overrides |
| `apps/website/src/config/models.ts` | Merges the two above; exports typed `Model[]` |
To regenerate the JSON from workflow_templates:
```bash
pnpm tsx apps/website/scripts/generate-models.ts
```
This writes `apps/website/src/config/generated-models.json` directly.
Thumbnails are populated from local `.webp` files in `workflow_templates/templates/` — no network access needed.
---
## Phase 2 — Gather model data (ADD / UPDATE)
Run the generator to get fresh data, then find the model:
```bash
pnpm tsx apps/website/scripts/generate-models.ts
jq '.[] | select(.slug | contains("MODEL_SLUG"))' \
apps/website/src/config/generated-models.json
```
The JSON fields are:
- `slug` — URL slug
- `name` — exact filename or display name for partner nodes
- `huggingFaceUrl` — download URL (empty for partner nodes)
- `directory``diffusion_models` | `loras` | … | `partner_nodes`
- `workflowCount` — integer
- `displayName` — human-readable name
If no match and it is a known API/partner model, add it to `API_PROVIDER_MAP` in
`generate-models.ts` and re-run. Otherwise tell the user.
---
## Phase 3 — Check for existing entry
```bash
jq --arg slug "${SLUG}" '.[] | select(.slug == $slug)' \
apps/website/src/config/generated-models.json
```
- Match found + action is `add` → switch to UPDATE flow automatically
- No match + action is `update` → stop and tell the user
---
## Phase 4A — ADD: new partner/API model not in workflow_templates
For partner nodes (no local file), add an entry to `API_PROVIDER_MAP` in
`apps/website/scripts/generate-models.ts`:
```typescript
mymodel: { name: 'My Model', slug: 'my-model' },
```
Then re-run `pnpm tsx apps/website/scripts/generate-models.ts` — it will appear
in `generated-models.json` automatically.
If you also want a `docsUrl`, `blogUrl`, or a link to the hub model page, add an entry to `model-metadata.ts`:
```typescript
'my-model': {
docsUrl: 'https://docs.comfy.org/tutorials/...',
blogUrl: 'https://blog.comfy.org/...',
hubSlug: 'my-model', // slug at comfy.org/workflows/model/{hubSlug} — only set if the page exists (returns 200)
featured: true
}
```
No changes to `models.ts` or `translations.ts` are needed.
---
## Phase 4B — UPDATE: edit existing entry
Only `model-metadata.ts` needs editing for most updates (docsUrl, blogUrl,
featured). For `displayName` or `directory` changes, edit the entry directly in
`generated-models.json` (until the next generator run would overwrite it — then
fix the source in `generate-models.ts`).
---
## Phase 4C — REMOVE: delete entry
Remove the entry from `generated-models.json` (or mark it with `canonicalSlug`
pointing to the replacement). No translation file changes needed.
---
## Phase 5 — Verify TypeScript
```bash
pnpm typecheck 2>&1 | grep -E "error|warning" | head -20
```
Fix any type errors before proceeding. Common issues:
- `ModelDirectory` type not matching a new `directory` value — add it to the union
- JSON import shape mismatch — `generated-models.json` must match `OutputModel`
---
## Phase 6 — Create PR
```bash
BRANCH="add-model-page-MODEL-SLUG" # or update- / remove-
git checkout -b $BRANCH
git add apps/website/src/config/generated-models.json \
apps/website/scripts/generate-models.ts \
apps/website/src/config/model-metadata.ts
git commit -m "feat(models): add model page for MODEL-SLUG"
git push -u origin $BRANCH
gh pr create \
--title "Add model page: MODEL-SLUG" \
--body "$(cat <<'EOF'
Adds a new model page entry for MODEL-SLUG.
## Changes
- `generated-models.json`: regenerated with new entry (workflowCount N, directory DIRECTORY)
- `model-metadata.ts`: editorial overrides (docsUrl, featured) if needed
EOF
)"
```
For UPDATE use branch `update-model-page-MODEL-SLUG`.
For REMOVE use `remove-model-page-MODEL-SLUG`.
---
## Error states
| Situation | Response |
| ------------------------------- | ---------------------------------------------------------------- |
| Model not in workflow templates | Ask user to verify spelling or add it manually as a partner node |
| Slug already exists (add) | Switch to update flow automatically |
| Slug not found (update/remove) | Stop and ask user to confirm |
| Typecheck fails | Fix the error before pushing |

View File

@@ -9,18 +9,12 @@ Cherry-pick backport management for Comfy-Org/ComfyUI_frontend stable release br
## Quick Start
1. **Discover** — Collect candidates from Slack bot + git log gap, then **reconcile both lists** (`reference/discovery.md`)
2. **Pre-filter by path** — Auto-skip PRs whose changed files are entirely under `apps/website/`, `browser_tests/`, `.github/`, `packages/design-system/`, `packages/{cloud,registry}-types/`, `.claude/`, `docs/`. Don't read PR bodies for these — they don't ship to core ComfyUI users (`reference/analysis.md`)
3. **Verify target file existence** — For each surviving candidate, run `git cat-file -e origin/$TARGET:$path` for primary changed files. If they don't exist on the target, auto-mark SKIP with reason `feature-not-on-branch`
4. **Tiered triage** — Bucket into **Tier 1 (core editor must-haves)**, **Tier 2 (cloud-distribution only)**, **Tier 3 (skip)** before reviewing individually (`reference/analysis.md`)
5. **Analyze**Categorize remaining MUST/SHOULD, check deps (`reference/analysis.md`)
6. **Human Review** — Present candidates in batches for interactive approval, with tier context attached (see Interactive Approval Flow)
7. **Plan** — Order by dependency (leaf fixes first), group into waves per branch
8. **Test-then-resolve dry-run** — Classify clean vs conflict before committing time (`reference/execution.md`)
9. **Execute** — Label-driven automation for clean PRs → worktree fallback for conflicts (`reference/execution.md`)
10. **Public-API conflict review** — If conflict resolution touches a public LiteGraph callback, extension API, or `node.*` method, consult oracle for compat-regression review BEFORE pushing (`reference/execution.md`)
11. **Verify** — Per-PR validation (typecheck + targeted tests + lint on changed files) AND per-wave verification (full typecheck + test:unit on branch HEAD)
12. **Log & Report** — Generate session report + author accountability report + Slack status update (`reference/logging.md`)
1. **Discover** — Collect candidates from Slack bot + git log gap (`reference/discovery.md`)
2. **Analyze** — Categorize MUST/SHOULD/SKIP, check deps (`reference/analysis.md`)
3. **Plan** — Order by dependency (leaf fixes first), group into waves per branch
4. **Execute** — Label-driven automation → worktree fallback for conflicts (`reference/execution.md`)
5. **Verify**After each wave, verify branch integrity before proceeding
6. **Log & Report** — Generate session report with mermaid diagram (`reference/logging.md`)
## System Context
@@ -43,29 +37,16 @@ Cherry-pick backport management for Comfy-Org/ComfyUI_frontend stable release br
**Critical: Match PRs to the correct target branches.**
| Branch prefix | Scope | Example |
| ------------- | ------------------------------ | ------------------------------------------------- |
| `cloud/*` | Cloud-hosted ComfyUI only | Team workspaces, cloud queue, cloud-only login |
| `core/*` | Local/self-hosted ComfyUI only | Core editor, local workflows, node system |
| Both | Shared infrastructure | App mode, Firebase auth (API nodes), payment URLs |
| Branch prefix | Scope | Example |
| ------------- | ------------------------------ | ----------------------------------------- |
| `cloud/*` | Cloud-hosted ComfyUI only | App mode, cloud auth, cloud-specific UI |
| `core/*` | Local/self-hosted ComfyUI only | Core editor, local workflows, node system |
### What Goes Where
**⚠️ NEVER backport cloud-only PRs to `core/*` branches.** Cloud-only changes (app mode, cloud auth, cloud billing UI, cloud-specific API calls) are irrelevant to local users and waste effort. Before backporting any PR to a `core/*` branch, check:
**Both core + cloud:**
- **App mode** PRs — app mode is NOT cloud-only
- **Firebase auth** PRs — Firebase auth is on core for API nodes
- **Payment redirect** PRs — payment infrastructure shared
- **Bug fixes** touching shared components
**Cloud-only (skip for core):**
- Team workspaces
- Cloud queue virtualization
- Hide API key login
- Cloud-specific UI behind cloud feature flags
**⚠️ NEVER backport cloud-only PRs to `core/*` branches.** But do NOT assume "app mode" or "Firebase" = cloud-only. Check the actual files changed.
- Does the PR title/description mention "app mode", "cloud", or cloud-specific features?
- Does the PR only touch files like `appModeStore.ts`, cloud auth, or cloud-specific components?
- If yes → skip for `core/*` branches (may still apply to `cloud/*` branches)
## ⚠️ Gotchas (Learn from Past Sessions)
@@ -86,61 +67,6 @@ The `pr-backport.yaml` action reports more conflicts than reality. `git cherry-p
12 or 27 conflicting files can be trivial (snapshots, new files). **Categorize conflicts first**, then decide. See Conflict Triage below.
### Accept-Theirs Can Produce Broken Hybrids
When a PR **rewrites a component** (e.g., PrimeVue → Reka UI), the accept-theirs regex produces a broken mix of old and new code. The template may reference new APIs while the script still has old imports, or vice versa.
**Detection:** Content conflicts with 4+ conflict markers in a single `.vue` file, especially when imports change between component libraries.
**Fix:** Instead of accept-theirs regex, use `git show MERGE_SHA:path/to/file > path/to/file` to get the complete correct version from the merge commit on main. This bypasses the conflict entirely.
### Cherry-Picks Can Reference Missing Dependencies
When PR A on main depends on code introduced by PR B (which was merged before A), cherry-picking A brings in code that references B's additions. The cherry-pick succeeds but the branch is broken.
**Common pattern:** Composables, component files, or type definitions introduced by an earlier PR and used by the cherry-picked PR.
**Detection:** `pnpm typecheck` fails with "Cannot find module" or "is not defined" errors after cherry-pick.
**Fix:** Use `git show MERGE_SHA:path/to/missing/file > path/to/missing/file` to bring the missing files from main. Always verify with typecheck.
### Use `--no-verify` for Worktree Pushes
Husky hooks fail in worktrees (can't find lint-staged config). Always use `git push --no-verify` and `git commit --no-verify` when working in `/tmp/` worktrees.
### Automation Success Varies Wildly by Branch
In the 2026-04-06 session: core/1.42 got 18/26 auto-PRs, cloud/1.42 got only 1/25. The cloud branch has more divergence. **Always plan for manual fallback** — don't assume automation will handle most PRs.
### Cherry-Picked Tests Can Reference Files Added By Earlier Unbackported PRs
A common conflict: PR A on main modifies a test file that was _added_ on main by an earlier PR B (not backported to the target). The cherry-pick of A reports "modify/delete" on B's test file because the file doesn't exist on the target. Adding the new file would smuggle in B's test scaffolding without B's runtime changes.
**Detection:** Conflict says `deleted in HEAD and modified in <PR>`. Verify with:
```bash
git log --diff-filter=A --oneline origin/main -- path/to/test.ts
```
If the introducing commit is **not** on the target branch, the test file isn't a real prerequisite for the runtime fix.
**Fix:** `git rm` the test file (drop it from the backport). Document in the commit body which PR introduced it on main and why dropping it is safe. The runtime fix itself usually doesn't depend on these tests — coverage exists at the integration layer.
### Backport-Only Compatibility Shims
When a PR's _mechanism_ relies on changes upstream that aren't on the older branch, a literal cherry-pick can recreate the original bug for any consumer still using the old contract. This is most dangerous for **public LiteGraph callbacks, extension APIs, and `node.*` methods** that custom-node packages depend on.
**Real example (#11541, core/1.43 backport):** The PR removed `LGraphNode.vue`'s legacy `handled === true` sync-return check from `handleDrop`, replacing it with `await node.onDragDrop(event, true)`. Safe on `main` because all in-repo `onDragDrop` handlers had migrated to participate in the new `claimEvent` flag. On `core/1.43`, `onDragDrop` is a public callback — custom-node packages with synchronous `onDragDrop` returning `true` would no longer have their event claimed, recreating the duplicate-node-creation bug the PR was fixing.
**Detection:** The PR's diff modifies a file that is part of a public extension API surface. Look for:
- `node.onXxx` callback assignments
- Methods on `LGraphNode`, `LGraphCanvas`, `LGraph`, `Subgraph`
- Public exports from `src/lib/litegraph/`
- Type changes affecting `litegraph-augmentation.d.ts`
**Fix:** Add a backport-only compatibility shim that preserves the old contract while keeping the new fix. Document it explicitly in the commit body under a `## Backport-only compatibility fix` heading. Consult oracle for review before pushing — a bad shim is worse than no fix.
## Conflict Triage
**Always categorize before deciding to skip. High conflict count ≠ hard conflicts.**
@@ -151,8 +77,6 @@ When a PR's _mechanism_ relies on changes upstream that aren't on the older bran
| **Modify/delete (new file)** | PR introduces files not on target | `git add $FILE` — keep the new file |
| **Modify/delete (removed)** | Target removed files the PR modifies | `git rm $FILE` — file no longer relevant |
| **Content conflicts** | Marker-based (`<<<<<<<`) | Accept theirs via python regex (see below) |
| **Component rewrites** | 4+ markers in `.vue`, library change | Use `git show SHA:path > path` — do NOT accept-theirs |
| **Import-only conflicts** | Only import lines differ | Keep both imports if both used; remove unused after |
| **Add/add** | Both sides added same file | Accept theirs, verify no logic conflict |
| **Locale/JSON files** | i18n key additions | Accept theirs, validate JSON after |
@@ -179,27 +103,7 @@ Skip these without discussion:
- **Test-only / lint rule changes** — Not user-facing
- **Revert pairs** — If PR A reverted by PR B, skip both. If fixed version (PR C) exists, backport only C.
- **Features not on target branch** — e.g., Painter, GLSLShader, appModeStore on core/1.40
- **Cloud-only PRs on core/\* branches** — Team workspaces, cloud queue, cloud-only login. (Note: app mode and Firebase auth are NOT cloud-only — see Branch Scope Rules)
### Path Pre-Filter (run BEFORE reading PR bodies)
For 50+ candidate PRs, classify by changed paths first to skip the unproductive ones without spending time on triage. Run `git show --stat $SHA` (or `gh pr view --json files`) and bucket:
| Path prefix | Bucket | Reason |
| ---------------------------------------------- | ---------------------- | ------------------------------------------------ |
| `apps/website/` | SKIP | Marketing/platform site, not core ComfyUI bundle |
| `apps/desktop-ui/` | SKIP for `core/*` | Desktop app, separate release cadence |
| `browser_tests/` only (no `src/`) | SKIP | Test-only |
| `.github/workflows/` only | SKIP | CI/release infra |
| `packages/design-system/` only | SKIP | Design tokens, not core |
| `packages/{cloud,registry,ingest}-types/` only | SKIP | Generated types |
| `.claude/`, `.agents/`, `docs/` | SKIP | Agent / documentation |
| `*.stories.ts` only | SKIP | Storybook only |
| `src/` (core editor) | KEEP — analyze further | Runtime/editor code that requires full triage |
A PR touches multiple paths? Keep it if **any** changed file is under `src/` (or other core paths) and run normal analysis. Auto-skip is conservative — only skip when _all_ paths match the SKIP buckets.
This filter alone removes ~30-50% of candidates in a typical session, leaving only the PRs that need real triage.
- **Cloud-only PRs on core/\* branches** — App mode, cloud auth, cloud billing. These only affect cloud-hosted ComfyUI.
## Wave Verification
@@ -218,18 +122,6 @@ git worktree remove /tmp/verify-TARGET --force
If typecheck or tests fail, stop and investigate before continuing. A broken branch after wave N means all subsequent waves will compound the problem.
### Fix PRs Are Normal
Expect to create 1 fix PR per branch after verification. Common issues:
1. **Component rewrite hybrids** — accept-theirs produced broken `.vue` files. Fix: overwrite with correct version from merge commit via `git show SHA:path > path`
2. **Missing dependency files** — cherry-pick brought in code referencing composables/components not on the branch. Fix: add missing files from merge commit
3. **Missing type properties** — cherry-picked code uses interface properties not yet on the branch (e.g., `key` on `ConfirmDialogOptions`). Fix: add the property to the interface
4. **Unused imports** — conflict resolution kept imports that the branch doesn't use. Fix: remove unused imports
5. **Wrong types from conflict resolution** — e.g., `{ top: number; right: number }` vs `{ top: number; left: number }`. Fix: match the return type of the actual function
Create a fix PR on a branch from the target, verify typecheck passes, then merge with `--squash --admin`.
### Never Admin-Merge Without CI
In a previous bulk session, all 69 backport PRs were merged with `gh pr merge --squash --admin`, bypassing required CI checks. This shipped 3 test failures to a release branch. **Lesson: `--admin` skips all branch protection, including required status checks.** Only use `--admin` after confirming CI has passed (e.g., `gh pr checks $PR` shows all green), or rely on auto-merge (`--auto --squash`) which waits for CI by design.
@@ -243,43 +135,6 @@ Large backport sessions (50+ PRs) are expensive and error-prone. Prefer continuo
- Reserve session-style bulk backporting for catching up after gaps
- When a release branch is created, immediately start the continuous process
## Interactive Approval Flow
After analysis, present ALL candidates (MUST, SHOULD, and borderline) to the human for interactive review before execution. Do not write a static decisions.md — collect approvals in conversation.
### Batch Presentation
Present PRs in batches of 5-10, grouped by theme (visual bugs, interaction bugs, cloud/auth, data correctness, etc.). Use this table format:
```
# | PR | Title | Target | Rec | Context
----+--------+------------------------------------------+---------------+------+--------
1 | #12345 | fix: broken thing | core+cloud/42 | Y | Description here. Why it matters. Agent reasoning.
2 | #12346 | fix: another issue | core/42 | N | Only affects removed feature. Not on target branch.
```
Each row includes:
- PR number and title
- Target branches
- Agent recommendation: `Rec: Y` or `Rec: N` with brief reasoning
- 2-3 sentence context: what the PR does, why it matters (or doesn't)
### Human Response Format
- `Y` — approve for backport
- `N` — skip
- `?` — investigate (agent shows PR description, files changed, detailed take, then re-asks)
- Any freeform question or comment triggers discussion before moving on
- Bulk responses accepted (e.g. `1 Y, 2 Y, 3 N, 4 ?`)
### Rules
- ALL candidates are reviewed, not just MUST items
- When human responds `?`, show the PR description, files changed, and agent's detailed analysis, then re-ask for their decision
- When human asks a question about a PR, answer with context and recommendation, then wait for their decision
- Do not proceed to execution until all batches are reviewed and every candidate has a Y or N
## Quick Reference
### Label-Driven Automation (default path)
@@ -295,96 +150,13 @@ gh api repos/Comfy-Org/ComfyUI_frontend/issues/$PR/labels \
```bash
git worktree add /tmp/backport-$BRANCH origin/$BRANCH
cd /tmp/backport-$BRANCH
# For each PR:
git fetch origin $BRANCH
git checkout -b backport-$PR-to-$BRANCH origin/$BRANCH
git cherry-pick -m 1 $MERGE_SHA
# Resolve conflicts (see Conflict Triage)
git push origin backport-$PR-to-$BRANCH --no-verify
gh pr create --base $BRANCH --head backport-$PR-to-$BRANCH \
--title "[backport $BRANCH] $TITLE (#$PR)" \
--body "Backport of #$PR. [conflict notes]"
gh pr merge $NEW_PR --squash --admin
sleep 25
# Resolve conflicts, push, create PR, merge
```
### Efficient Batch: Test-Then-Resolve Pattern
When many PRs need manual cherry-pick (e.g., cloud branches), test all first:
```bash
cd /tmp/backport-$BRANCH
for pr in "${ORDER[@]}"; do
git checkout -b test-$pr origin/$BRANCH
if git cherry-pick -m 1 $SHA 2>/dev/null; then
echo "CLEAN: $pr"
else
echo "CONFLICT: $pr"
git cherry-pick --abort
fi
git checkout --detach HEAD
git branch -D test-$pr
done
```
Then process clean PRs in a batch loop, conflicts individually.
### PR Title Convention
```
[backport TARGET_BRANCH] Original Title (#ORIGINAL_PR)
```
## Final Deliverables (Slack-Compatible)
After execution completes, generate two files in `~/temp/backport-session/`. Both must be **Slack-compatible plain text** — no emojis, no markdown tables, no headers (`#`), no bold (`**`), no inline code. Use plain dashes, indentation, and line breaks only.
### 1. Author Accountability Report
File: `backport-author-accountability.md`
Lists all backported PRs grouped by original author (via `gh pr view $PR --json author`). Surfaces who should be self-labeling.
```
Backport Session YYYY-MM-DD -- PRs that should have been labeled by authors
- author-login
- #1234 fix: short title
- #5678 fix: another title
- other-author
- #9012 fix: some other fix
```
Authors sorted alphabetically, 4-space indent for nested items.
### 2. Slack Status Update
File: `slack-status-update.md`
A shareable summary of the session. Structure:
```
Backport session complete -- YYYY-MM-DD
[1-sentence summary: N PRs backported to which branches. All pass typecheck.]
Branches updated:
- core/X.XX: N PRs + N fix PRs (N auto, N manual)
- cloud/X.XX: N PRs + N fix PRs (N auto, N manual)
- ...
N total PRs created and merged (N backports + N fix PRs).
Notable fixes included:
- [category]: [list of fixes]
- ...
Conflict patterns encountered:
- [pattern and how it was resolved]
- ...
N authors had PRs backported. See author accountability list for details.
```
No emojis, no tables, no bold, no headers. Plain text that pastes cleanly into Slack.

View File

@@ -23,10 +23,10 @@ For SHOULD items with conflicts: if conflict resolution requires more than trivi
**Before categorizing, filter by branch scope:**
| Target branch | Skip if PR is... |
| ------------- | ----------------------------------------------------------------------------------------------------------------- |
| `core/*` | Cloud-only (team workspaces, cloud queue, cloud-only login). Note: app mode and Firebase auth are NOT cloud-only. |
| `cloud/*` | Local-only features not present on cloud branch |
| Target branch | Skip if PR is... |
| ------------- | ------------------------------------------------------------------- |
| `core/*` | Cloud-only (app mode, cloud auth, cloud billing, cloud-specific UI) |
| `cloud/*` | Local-only features not present on cloud branch |
Cloud-only PRs backported to `core/*` are wasted effort — `core/*` branches serve local/self-hosted users who never see cloud features. Check PR titles, descriptions, and files changed for cloud-specific indicators.
@@ -39,89 +39,6 @@ Check before backporting — these don't exist on older branches:
- **App builder** — check per branch
- **appModeStore.ts** — not on core/1.40
### Verify Target File Existence (Run Before Cherry-Pick)
Before cherry-picking any PR, confirm the files it modifies actually exist on the target branch. If they don't, the PR's runtime fix is for a feature that hasn't been added yet — skip cleanly without attempting cherry-pick:
```bash
# For each file the PR changes
for f in $(gh pr view $PR --json files --jq '.files[].path' | grep -v "^browser_tests/\|\.test\." ); do
if ! git cat-file -e origin/$TARGET:$f 2>/dev/null; then
echo "MISSING on $TARGET: $f"
fi
done
```
If the _primary_ changed files (the runtime ones, not tests) are missing, mark the PR `SKIP / feature-not-on-branch`. This is faster than letting cherry-pick fail with modify/delete conflicts and gives a clean signal.
This check is the first thing that runs after the path pre-filter and BEFORE you spend time reading PR descriptions.
## Tiered Triage (Recommended for 30+ Candidates)
Before the interactive Y/N approval flow, bucket all surviving candidates into three tiers. This surfaces release-engineering decisions that a flat MUST/SHOULD list obscures:
### Tier 1 — Core Editor Must-Haves
User-facing bugs, crashes, data corruption, or security issues in code paths that exist on the target branch. These are the strongest backport candidates.
Indicators:
- `fix:` prefix and the bug is reproducible on the target branch
- Crash guards, runtime null checks, race-condition fixes
- Data-loss bugs (state not persisted, duplicates, drops)
- Security hardening (CSRF, XSS, auth)
- Vue Nodes 2.0 regression cluster (if the target ships Vue Nodes 2.0)
- Subgraph correctness fixes
- Public-API extension callback fixes
Recommend `Y` to user.
### Tier 2 — Cloud-Distribution Only
Bugs that only manifest on cloud-hosted distributions (Secrets panel, subscription flows, cloud signup, workspace tracking, etc.). Whether to backport depends on whether cloud ships from the target `core/*` branch in your release matrix.
Indicators:
- Files under `src/platform/secrets/`, `src/platform/subscription/`, signup flows
- PR description mentions cloud staging issues
- Fix gated behind cloud feature flags
Default: ask the cloud release rotation owner. If unsure, defer.
### Tier 3 — Skip
Path pre-filter caught most of these. The rest are PRs where the diff _touches_ `src/` but the practical impact is non-user-facing or scoped to features the target doesn't ship.
Indicators:
- All changes in test files even if the PR touched `src/` test files
- Storybook stories only
- Lint config / lint rule additions
- Documentation comments
- Internal refactors with no behavior change
### Presentation Format
When showing tier results to the user, format as:
```text
Tier 1 (N PRs) — strong backport candidates
- #11541 fix: stop duplicate node creation when dropping image on Vue nodes
Why: Vue Nodes 2.0 regression — async onDragDrop bypassed handled-check, drops bubble to document, spawns extra LoadImage nodes
- #10849 fix: store promoted widget values per SubgraphNode instance
Why: Multiple instances overwriting each other's promoted widget values — data loss
Tier 2 (N PRs) — cloud-distribution release rotation should decide
- #11636 fix: enable Chrome password autofill on signup form
- ...
Tier 3 (N PRs) — skip recommended
- #11586 fix: website polish (apps/website/ only)
- ...
```
Then run interactive Y/N over Tier 1 and Tier 2; Tier 3 gets confirmed-skip without per-PR review.
## Dep Refresh PRs
Always SKIP on stable branches. Risk of transitive dependency regressions outweighs audit cleanup benefit. If a specific CVE fix is needed, cherry-pick that individual fix instead.
@@ -144,6 +61,8 @@ done
## Human Review Checkpoint
Use the Interactive Approval Flow (see SKILL.md) to review all candidates interactively. Do not write a static decisions.md for the human to edit — instead, present batches of 5-10 PRs with context and recommendations, and collect Y/N/? responses in conversation.
Present decisions.md before execution. Include:
All candidates must be reviewed (MUST, SHOULD, and borderline items), not just a subset.
1. All MUST/SHOULD/SKIP categorizations with rationale
2. Questions for human (feature existence, scope, deps)
3. Estimated effort per branch

View File

@@ -1,11 +1,5 @@
# Discovery — Candidate Collection
**Run all sources, then reconcile.** No single source is authoritative:
- Slack bot may flag PRs that have already been backported (false positive)
- Git gap may include PRs that don't need backport (test-only, design-system, website)
- Bot can also miss PRs that landed without the right labels
## Source 1: Slack Backport-Checker Bot
Use `slackdump` skill to export `#frontend-releases` channel (C09K9TPU2G7):
@@ -42,43 +36,7 @@ gh pr view $PR --json mergeCommit,title --jq '"Title: \(.title)\nMerge: \(.merge
gh pr view $PR --json files --jq '.files[].path'
```
## Source 4: Already-Backported PRs (cross-reference)
When the target branch already has some cherry-picks on it (e.g., partway through a release window), extract the originals to avoid re-backporting:
```bash
# Get all original PR numbers already backported to TARGET since the last release tag
git log --format="%H%n%B" $LAST_TAG..origin/$TARGET \
| grep -oiE "(backport of|cherry.picked) #?[0-9]+" \
| grep -oE "[0-9]+" \
| sort -un > /tmp/already-backported.txt
```
Subtract this list from your candidates.
## Reconciliation Workflow
```bash
# 1. Slack bot list (parse from export)
# /tmp/bot-flagged.txt — one PR# per line, sorted
# 2. Git gap fix/perf only
MB=$(git merge-base origin/main origin/$TARGET)
git log --format="%h|%s" $MB..origin/main \
| grep -iE "^[a-f0-9]+\|(fix|perf)" \
| grep -oE "#[0-9]+\)" | grep -oE "[0-9]+" \
| sort -un > /tmp/gap-fixes.txt
# 3. Already backported (Source 4 above)
# 4. Candidates = (gap-fixes bot-flagged) already-backported
sort -u /tmp/gap-fixes.txt /tmp/bot-flagged.txt > /tmp/union.txt
comm -23 /tmp/union.txt /tmp/already-backported.txt > /tmp/candidates.txt
```
The result is the input to the path pre-filter (`SKILL.md` Quick Start step 2).
## Output: candidate_list.md
Table per target branch:
| PR# | Title | Source (bot/gap/both) | Path bucket | Tier | Decision |
| PR# | Title | Category | Flagged by Bot? | Decision |

View File

@@ -6,43 +6,6 @@
2. Medium gap next (quick win)
3. Largest gap last (main effort)
## Step 0: Test-Then-Resolve Pre-Pass (Recommended)
Before triggering label-driven automation, run a dry-run cherry-pick loop to classify candidates. This is much faster than discovering conflicts after-the-fact across automation, manual cherry-picks, and CI failures.
```bash
git fetch origin TARGET_BRANCH
git worktree add /tmp/dryrun-TARGET origin/TARGET_BRANCH
cd /tmp/dryrun-TARGET
CLEAN=()
CONFLICT=()
for pr in "${CANDIDATES[@]}"; do
SHA=$(gh pr view $pr --json mergeCommit --jq '.mergeCommit.oid')
git checkout -b dryrun-$pr origin/TARGET_BRANCH 2>/dev/null
if git cherry-pick -m 1 $SHA 2>/dev/null; then
CLEAN+=($pr)
else
CONFLICT+=($pr)
git cherry-pick --abort
fi
git checkout --detach HEAD 2>/dev/null
git branch -D dryrun-$pr 2>/dev/null
done
echo "CLEAN (${#CLEAN[@]}): ${CLEAN[*]}"
echo "CONFLICT (${#CONFLICT[@]}): ${CONFLICT[*]}"
cd -
git worktree remove /tmp/dryrun-TARGET --force
```
Use the result to:
- Send CLEAN PRs through label-driven automation (Step 1) — they'll typically self-merge
- Reserve manual worktree time (Step 3) for CONFLICT PRs only
- Surface PRs likely to need backport-only compat shims (CONFLICT files in `src/lib/litegraph/` or `src/scripts/app.ts`)
## Step 1: Label-Driven Automation (Batch)
```bash
@@ -110,55 +73,14 @@ for PR in ${CONFLICT_PRS[@]}; do
git cherry-pick -m 1 $MERGE_SHA
# If conflict — NEVER skip based on file count alone!
# Categorize conflicts first: binary PNGs, modify/delete, content, add/add, component rewrites
# Categorize conflicts first: binary PNGs, modify/delete, content, add/add
# See SKILL.md Conflict Triage table for resolution per type.
# For component rewrites (4+ markers in a .vue file, library migration):
# DO NOT use accept-theirs regex — it produces broken hybrids.
# Instead, use the complete file from the merge commit:
# git show $MERGE_SHA:path/to/file > path/to/file
# For simple content conflicts, accept theirs:
# python3 -c "import re; ..."
# Resolve all conflicts, then:
git add .
GIT_EDITOR=true git cherry-pick --continue
# ── Public-API conflict review (REQUIRED for extension-API surfaces) ──
# If the conflict resolution touched any of these surfaces, consult oracle
# BEFORE pushing. A bad shim is worse than no fix:
# - node.onXxx callback assignments (onDragDrop, onConnectionsChange, onRemoved, onConfigure, etc.)
# - Methods on LGraphNode, LGraphCanvas, LGraph, Subgraph
# - Public exports from src/lib/litegraph/
# - Type changes in litegraph-augmentation.d.ts
# If a public callback's signature/contract changed: add a backport-only
# compatibility shim that preserves the OLD contract while keeping the
# new fix. Document it in the commit body under
# "## Backport-only compatibility fix". See SKILL.md gotcha section.
# ───────────────────────────────────────────────────────────────────────
# Per-PR validation BEFORE push (catches issues earlier than wave verification).
# Guard each targeted command against empty file lists — running `pnpm test:unit -- run`
# with no arg matchers would run the full suite, and `pnpm exec eslint` with no args errors.
pnpm typecheck
mapfile -t TEST_FILES < <(git diff --name-only HEAD~1 | grep -E '\.test\.ts$' || true)
if [ ${#TEST_FILES[@]} -gt 0 ]; then
pnpm test:unit -- run "${TEST_FILES[@]}"
else
echo "No changed test files — skipping targeted unit tests"
fi
mapfile -t CODE_FILES < <(git diff --name-only HEAD~1 | grep -E '\.(ts|vue)$' || true)
if [ ${#CODE_FILES[@]} -gt 0 ]; then
pnpm exec eslint "${CODE_FILES[@]}"
pnpm exec oxfmt --check "${CODE_FILES[@]}"
else
echo "No changed ts/vue files — skipping targeted lint/format"
fi
git push origin backport-$PR-to-TARGET --no-verify
git push origin backport-$PR-to-TARGET
NEW_PR=$(gh pr create --base TARGET_BRANCH --head backport-$PR-to-TARGET \
--title "[backport TARGET] TITLE (#$PR)" \
--body "Backport of #$PR..." | grep -oP '\d+$')
@@ -192,30 +114,7 @@ source ~/.nvm/nvm.sh && nvm use 24 && pnpm install && pnpm typecheck && pnpm tes
git worktree remove /tmp/verify-TARGET --force
```
If verification fails, **do not skip** — create a fix PR:
```bash
# Stay in the verify worktree
git checkout -b fix-backport-TARGET origin/TARGET_BRANCH
# Common fixes:
# 1. Component rewrite hybrids: overwrite with merge commit version
git show MERGE_SHA:path/to/Component.vue > path/to/Component.vue
# 2. Missing dependency files
git show MERGE_SHA:path/to/missing.ts > path/to/missing.ts
# 3. Missing type properties: edit the interface
# 4. Unused imports: delete the import lines
git add -A
git commit --no-verify -m "fix: resolve backport typecheck issues on TARGET"
git push origin fix-backport-TARGET --no-verify
gh pr create --base TARGET --head fix-backport-TARGET --title "fix: resolve backport typecheck issues on TARGET" --body "..."
gh pr merge $PR --squash --admin
```
Do not proceed to the next branch until typecheck passes.
If verification fails, stop and fix before proceeding to the next wave. Do not compound problems across waves.
## Conflict Resolution Patterns
@@ -243,35 +142,7 @@ git rm $FILE
git checkout --theirs $FILE && git add $FILE
```
### 4. Component Rewrites (DO NOT accept-theirs)
When a PR completely rewrites a component (e.g., PrimeVue → Reka UI), accept-theirs produces
a broken hybrid with mismatched template/script sections.
```bash
# Use the complete correct file from the merge commit instead:
git show $MERGE_SHA:src/components/input/MultiSelect.vue > src/components/input/MultiSelect.vue
git show $MERGE_SHA:src/components/input/SingleSelect.vue > src/components/input/SingleSelect.vue
git add src/components/input/MultiSelect.vue src/components/input/SingleSelect.vue
```
**Detection:** 4+ conflict markers in a single `.vue` file, imports changing between component
libraries (PrimeVue → Reka UI, etc.), template structure completely different on each side.
### 5. Missing Dependencies After Cherry-Pick
Cherry-picks can succeed but leave the branch broken because the PR's code on main
references composables/components introduced by an earlier PR.
```bash
# Add the missing file from the merge commit:
git show $MERGE_SHA:src/composables/queue/useJobDetailsHover.ts > src/composables/queue/useJobDetailsHover.ts
git show $MERGE_SHA:src/components/builder/BuilderSaveDialogContent.vue > src/components/builder/BuilderSaveDialogContent.vue
```
**Detection:** `pnpm typecheck` fails with "Cannot find module" or "X is not defined" after cherry-pick succeeds cleanly.
### 6. Locale Files
### 4. Locale Files
Usually adding new i18n keys — accept theirs, validate JSON:
@@ -305,17 +176,8 @@ gh pr checks $PR --watch --fail-fast && gh pr merge $PR --squash --admin
8. **Always validate JSON** after resolving locale file conflicts
9. **Dep refresh PRs** — skip on stable branches. Risk of transitive dep regressions outweighs audit cleanup. Cherry-pick individual CVE fixes instead.
10. **Verify after each wave** — run `pnpm typecheck && pnpm test:unit` on the target branch after merging a batch. Catching breakage early prevents compounding errors.
11. **App mode and Firebase auth are NOT cloud-only** — they go to both core and cloud branches. Only team workspaces, cloud queue, and cloud-specific login are cloud-only.
11. **Cloud-only PRs don't belong on core/\* branches** — app mode, cloud auth, and cloud-specific UI changes are irrelevant to local users. Always check PR scope against branch scope before backporting.
12. **Never admin-merge without CI**`--admin` bypasses all branch protections including required status checks. A bulk session of 69 admin-merges shipped 3 test failures. Always wait for CI to pass first, or use `--auto --squash` which waits by design.
13. **Accept-theirs regex breaks component rewrites** — when a PR migrates between component libraries (PrimeVue → Reka UI), the regex produces a broken hybrid. Use `git show SHA:path > path` to get the complete correct version instead.
14. **Cherry-picks can silently bring in missing-dependency code** — if PR A references a composable introduced by PR B, cherry-picking A succeeds but typecheck fails. Always run typecheck after each wave and add missing files from the merge commit.
15. **Fix PRs are expected** — plan for 1 fix PR per branch to resolve typecheck issues from conflict resolutions. This is normal, not a failure.
16. **Use `--no-verify` in worktrees** — husky hooks fail in `/tmp/` worktrees. Always push/commit with `--no-verify`.
17. **Automation success varies by branch** — core/1.42 got 18/26 auto-PRs (69%), cloud/1.42 got 1/25 (4%). Cloud branches diverge more. Plan for manual fallback.
18. **Test-then-resolve pattern** — for branches with low automation success, run a dry-run loop to classify clean vs conflict PRs before processing. This is much faster than resolving conflicts serially.
19. **Public-API conflict resolutions need oracle review** — when a conflict touches `node.onXxx` callbacks, `LGraphNode`/`LGraphCanvas`/`LGraph`/`Subgraph` methods, or types in `litegraph-augmentation.d.ts`, consult oracle BEFORE pushing. Custom-node packages depend on these contracts. A literal cherry-pick of a refactor-style fix can silently break extensions still using the old contract — sometimes recreating the very bug the PR was fixing. Document any backport-only compatibility shim explicitly in the commit body.
20. **Cherry-picked tests can require unbackported test scaffolding** — when a PR modifies a test file that was _added_ on main by an earlier unbackported PR, the cherry-pick reports modify/delete on that file. Drop it from the backport (`git rm`) and document which PR introduced it. Don't smuggle in test infrastructure without its runtime prerequisites.
21. **Per-PR validation catches issues earlier than wave verification** — for high-stakes branches, run `pnpm typecheck && pnpm exec eslint <changed files> && pnpm exec oxfmt --check` per PR before pushing. Wave verification still matters (it catches cross-PR interactions), but per-PR makes attribution trivial when something fails.
## CI Failure Triage
@@ -341,40 +203,3 @@ Common failure categories:
| Type error | Interface changed on main but not branch | May need manual adaptation |
**Never assume a failure is safe to skip.** Present all failures to the user with analysis.
## PR Body Template (Manual Cherry-Picks)
Manual cherry-pick PRs need detail beyond the automation's terse default. Use this template — reviewers will look here before re-deriving conflict-resolution logic from the diff.
```markdown
Manual backport of #ORIG to `TARGET` for inclusion in `vX.Y.Z`.
Cherry-picked from upstream merge commit `SHORT_SHA`.
## Why
[1-2 sentences from the original PR's "Summary" — what bug, what fix mechanism]
## Conflict resolution
- **`path/to/file`** — [what conflicted on this branch] → [resolution chosen + why]
- **`path/to/dropped-test.test.ts`** — added on main by unrelated PR #XXXX (not backported). Dropped from this backport; runtime fix intact.
- [...]
## Backport-only compatibility fix (if applicable)
[If you added a shim that wasn't in the upstream PR, document it here — what extension surface, what contract, what the shim preserves, why the upstream version would have regressed it]
## Validation
- `pnpm typecheck`
- `pnpm test:unit -- run <targeted suites>` ✅ (N/N passing)
- `pnpm exec eslint <changed files>` ✅ (0 errors)
- `pnpm exec oxfmt --check` ✅ (clean)
[If manual e2e was skipped, explain why — e.g., requires live backend, headless not feasible. State that source is byte-identical to upstream + how long it's been baking on main.]
Original PR: #ORIG / Original commit: `FULL_SHA`
```
The conflict-resolution section is non-negotiable — every conflict you resolved by hand needs a one-liner. This makes archaeology trivial six months later when someone asks "why does this look slightly different from main?"

View File

@@ -2,25 +2,26 @@
## During Execution
Maintain `execution-log.md` with per-branch tables (this is internal, markdown tables are fine here):
Maintain `execution-log.md` with per-branch tables:
```markdown
| PR# | Title | Status | Backport PR | Notes |
| ----- | ----- | ------ | ----------- | ------- |
| #XXXX | Title | merged | #YYYY | Details |
| PR# | Title | CI Status | Status | Backport PR | Notes |
| ----- | ----- | ------------------------------ | --------------------------------- | ----------- | ------- |
| #XXXX | Title | ✅ Pass / ❌ Fail / ⏳ Pending | ✅ Merged / ⏭️ Skip / ⏸️ Deferred | #YYYY | Details |
```
## Wave Verification Log
Track verification results per wave within execution-log.md:
Track verification results per wave:
```markdown
Wave N Verification -- TARGET_BRANCH
## Wave N Verification TARGET_BRANCH
- PRs merged: #A, #B, #C
- Typecheck: pass / fail
- Fix PR: #YYYY (if needed)
- Typecheck: ✅ Pass / ❌ Fail
- Unit tests: ✅ Pass / ❌ Fail
- Issues found: (if any)
- Human review needed: (list any non-trivial conflict resolutions)
```
## Session Report Template
@@ -62,42 +63,40 @@ Wave N Verification -- TARGET_BRANCH
- Feature branches that need tracking for future sessions?
```
## Final Deliverables
## Final Deliverable: Visual Summary
After all branches are complete and verified, generate these files in `~/temp/backport-session/`:
At session end, generate a **mermaid diagram** showing all backported PRs organized by target branch and category (MUST/SHOULD), plus a summary table. Present this to the user as the final output.
### 1. execution-log.md (internal)
```mermaid
graph TD
subgraph branch1["☁️ cloud/X.XX — N PRs"]
C1["#XXXX title"]
C2["#XXXX title"]
end
Per-branch tables with PR#, title, status, backport PR#, notes. Markdown tables are fine — this is for internal tracking, not Slack.
subgraph branch2must["🔴 core/X.XX MUST — N PRs"]
M1["#XXXX title"]
end
### 2. backport-author-accountability.md (Slack-compatible)
subgraph branch2should["🟡 core/X.XX SHOULD — N PRs"]
S1["#XXXX-#XXXX N auto-merged"]
S2["#XXXX-#XXXX N manual picks"]
end
See SKILL.md "Final Deliverables" section. Plain text, no emojis/tables/headers/bold. Authors sorted alphabetically with PRs nested under each.
classDef cloudStyle fill:#1a3a5c,stroke:#4da6ff,color:#e0f0ff
classDef coreStyle fill:#1a4a2e,stroke:#4dff88,color:#e0ffe8
classDef mustStyle fill:#5c1a1a,stroke:#ff4d4d,color:#ffe0e0
classDef shouldStyle fill:#4a3a1a,stroke:#ffcc4d,color:#fff5e0
```
### 3. slack-status-update.md (Slack-compatible)
See SKILL.md "Final Deliverables" section. Plain text summary that pastes cleanly into Slack. Includes branch counts, notable fixes, conflict patterns, author count.
## Slack Formatting Rules
Both shareable files (author accountability + status update) must follow these rules:
- No emojis (no checkmarks, no arrows, no icons)
- No markdown tables (use plain lists with dashes)
- No headers (no # or ##)
- No bold (\*_) or italic (_)
- No inline code backticks
- Use -- instead of em dash
- Use plain dashes (-) for lists with 4-space indent for nesting
- Line breaks between sections for readability
These files should paste directly into a Slack message and look clean.
Use the `mermaid` tool to render this diagram and present it alongside the summary table as the session's final deliverable.
## Files to Track
All in `~/temp/backport-session/`:
- `candidate_list.md` — all candidates per branch
- `decisions.md` — MUST/SHOULD/SKIP with rationale
- `wave-plan.md` — execution order
- `execution-log.md` — real-time status
- `backport-session-report.md` — final summary
- `execution-plan.md` -- approved PRs with merge SHAs (input)
- `execution-log.md` -- real-time status with per-branch tables (internal)
- `backport-author-accountability.md` -- PRs grouped by author (Slack-compatible)
- `slack-status-update.md` -- session summary (Slack-compatible)
All in `~/temp/backport-session/`.

View File

@@ -0,0 +1,278 @@
---
name: reproduce-issue
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
---
# Issue Reproduction Skill
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
## Architecture
Two videos are produced:
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
```
Phase 1: Research → Read issue, understand prerequisites
Phase 2: Environment → Install custom nodes, load workflows, configure settings
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
Phase 5: Report → Generate a structured reproduction report
```
## Prerequisites
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
- `gh` CLI (authenticated, for reading issues)
- ComfyUI backend with Python environment (for installing custom nodes)
## Phase 1: Research the Issue
1. Fetch the issue details:
```bash
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
```
2. Extract from the issue body:
- **Reproduction steps** (the exact sequence)
- **Prerequisites**: specific workflows, custom nodes, settings, models
- **Environment**: OS, browser, ComfyUI version
- **Media**: screenshots or videos showing the bug
3. Search the codebase for related code:
- Find the feature/component mentioned in the issue
- Understand how it works currently
- Identify what state the UI needs to be in
## Phase 2: Environment Setup
Set up everything the issue requires BEFORE attempting reproduction.
### Custom Nodes
If the issue mentions custom nodes:
```bash
# Find the custom node repo
# Clone into ComfyUI's custom_nodes directory
cd <comfyui_path>/custom_nodes
git clone <custom_node_repo_url>
# Install dependencies if needed
cd <custom_node_name>
pip install -r requirements.txt 2>/dev/null || true
# Restart ComfyUI server to load the new nodes
```
### Workflows
If the issue references a specific workflow:
```bash
# Download workflow JSON if a URL is provided
curl -L "<workflow_url>" -o /tmp/test-workflow.json
# Load it via the API
curl -X POST http://127.0.0.1:8188/api/workflow \
-H "Content-Type: application/json" \
-d @/tmp/test-workflow.json
```
Or load via playwright-cli:
```bash
playwright-cli goto "http://127.0.0.1:8188"
# Drag-and-drop or use File > Open to load the workflow
```
### Settings
If the issue requires specific settings:
```bash
# Use playwright-cli to open settings and change them
playwright-cli press "Control+,"
playwright-cli snapshot
# Find and modify the relevant setting
```
## Phase 3: Interactive Exploration — Research Video
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
```bash
# Open browser and start video recording
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
# Take a snapshot to see current state
playwright-cli snapshot
# Interact based on what you see
playwright-cli click <ref>
playwright-cli fill <ref> "text"
playwright-cli press "Control+s"
# Check results
playwright-cli snapshot
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
```
### Key Principles
- **Observe before acting**: Always `snapshot` before interacting
- **Retry and adapt**: If a step fails, try a different approach
- **Document what works**: Keep notes on which steps trigger the bug
- **Don't give up**: Try multiple approaches if the first doesn't work
- **Establish prerequisites**: Many bugs require specific UI state:
- Save a workflow first (File > Save)
- Make changes to dirty the workflow
- Open multiple tabs
- Add specific node types
- Change settings
- Resize the window
### Common ComfyUI Interactions via playwright-cli
| Action | Command |
| ------------------- | -------------------------------------------------------------- |
| Open hamburger menu | `playwright-cli click` on the C logo button |
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
| Add node | Double-click canvas → type node name → select from results |
| Connect nodes | Drag from output slot to input slot |
| Save workflow | `playwright-cli press "Control+s"` |
| Save As | Menu > File > Save As |
| Select node | Click on the node |
| Delete node | Select → `playwright-cli press "Delete"` |
| Right-click menu | `playwright-cli click <ref> --button right` |
| Keyboard shortcut | `playwright-cli press "Control+z"` |
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
Once the bug is confirmed, **stop the research video** and **close the research browser**:
```bash
playwright-cli video-stop
playwright-cli close
```
Now start a **fresh browser session** for the clean reproduce video (Video 2).
**IMPORTANT constraints:**
- **Max 5 minutes** — the reproduce video must be short and focused
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
1. **Open browser and start recording**:
```bash
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
```
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
3. **Take key screenshots** at critical moments:
```bash
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
# ... trigger the bug ...
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
```
4. **Stop recording and close** immediately after the bug is demonstrated:
```bash
playwright-cli video-stop
playwright-cli close
```
## Phase 5: Generate Report
Create a reproduction report at `tmp/qa/reproduce-report.md`:
```markdown
# Issue Reproduction Report
- **Issue**: <issue_url>
- **Title**: <issue_title>
- **Date**: <today>
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
## Environment
- ComfyUI Server: <url>
- OS: <os>
- Custom Nodes Installed: <list or "none">
- Settings Changed: <list or "none">
## Prerequisites
List everything that had to be set up before the bug could be triggered:
1. ...
2. ...
## Reproduction Steps
Minimal steps to reproduce (the clean sequence):
1. ...
2. ...
3. ...
## Expected Behavior
<from the issue>
## Actual Behavior
<what actually happened>
## Evidence
- Research video: `research-video/video.webm` (full exploration session)
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
- Screenshots: `before-bug.png`, `bug-visible.png`
## Root Cause Analysis (if identified)
<code pointers, hypothesis about what's going wrong>
## Notes
<any additional observations, workarounds discovered, related issues>
```
## Handling Failures
If the bug **cannot be reproduced**:
1. Document what you tried and why it didn't work
2. Check if the issue was already fixed (search git log for related commits)
3. Check if it's environment-specific (OS, browser, specific version)
4. Set report status to "Not Reproduced" with detailed notes
5. The report is still valuable — it saves others from repeating the same investigation
## CI Integration
In CI, this skill runs as a Claude Code agent with:
- `ANTHROPIC_API_KEY` for Claude
- `GEMINI_API_KEY` for initial issue analysis (optional)
- ComfyUI server pre-started in the container
- `playwright-cli` pre-installed
The CI workflow:
1. Gemini generates a reproduce guide (markdown) from the issue
2. Claude agent receives the guide and runs this skill
3. Claude explores interactively, installs dependencies, retries
4. Claude records a clean demo once reproduced
5. Video and report are uploaded as artifacts

View File

@@ -0,0 +1,277 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Automated quality assurance for the ComfyUI frontend. The pipeline reproduces reported bugs using Playwright E2E tests, records video evidence, and deploys reports to Cloudflare Pages.
## Architecture Overview
The QA pipeline uses a **three-phase approach**:
1. **RESEARCH** — Claude writes Playwright E2E tests to reproduce bugs (assertion-backed, no hallucination)
2. **REPRODUCE** — Deterministic replay of the research test with video recording
3. **REPORT** — Deploy results to Cloudflare Pages with badge, video, and verdict
### Key Design Decision
Earlier iterations used AI vision (Gemini) to drive a browser and judge results from video. This was abandoned after discovering **AI reviewers hallucinate** — Gemini reported "REPRODUCED" when videos showed idle screens. The current approach uses **Playwright assertions** as the source of truth: if the test passes, the bug is proven.
## Prerequisites
- Node.js 22+
- `pnpm` package manager
- `gh` CLI (authenticated)
- Playwright browsers: `npx playwright install chromium`
- Environment variables:
- `GEMINI_API_KEY` — for PR analysis and video review
- `ANTHROPIC_API_KEY` — for Claude Agent SDK (research phase)
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — for report deployment
## Pipeline Scripts
| Script | Role | Model |
|---|---|---|
| `scripts/qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide | gemini-3.1-pro-preview |
| `scripts/qa-agent.ts` | Research phase: Claude writes E2E tests | claude-sonnet-4-6 (Agent SDK) |
| `scripts/qa-record.ts` | Before/after video recording with Gemini-driven actions | gemini-3.1-pro-preview |
| `scripts/qa-reproduce.ts` | Deterministic replay with narration | gemini-3-flash-preview |
| `scripts/qa-video-review.ts` | Video comparison review | gemini-3-flash-preview |
| `scripts/qa-generate-test.ts` | Regression test generation from QA report | gemini-3-flash-preview |
| `scripts/qa-deploy-pages.sh` | Deploy to Cloudflare Pages + badge | — |
| `scripts/qa-batch.sh` | Batch-trigger QA for multiple issues | — |
| `scripts/qa-report-template.html` | Report site (light/dark, seekbar, copy badge) | — |
## Triggering QA
### Via GitHub Labels
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after comparison)
- **`qa-full`** — Full QA (3-OS matrix, after-only)
- **`qa-issue`** — Reproduce a bug from an issue
### Via Batch Script
```bash
# Trigger QA for specific issue numbers
./scripts/qa-batch.sh 10394 10238 9996
# From a triage file (top 5 Tier 1 issues)
./scripts/qa-batch.sh --from tmp/issues.md --top 5
# Preview without pushing
./scripts/qa-batch.sh --dry-run 10394
# Clean up old trigger branches
./scripts/qa-batch.sh --cleanup
```
### Via Workflow Dispatch
Go to Actions → "PR: QA" → Run workflow → choose mode (focused/full).
## CI Workflow (`.github/workflows/pr-qa.yaml`)
```
resolve-matrix → analyze-pr ──┐
├→ qa-before (main branch, worktree build)
├→ qa-after (PR branch)
└→ report (video review, deploy, comment)
```
Before/after jobs run **in parallel** on separate runners for clean isolation.
### Issue Reproduce Mode
For issues (not PRs), the pipeline:
1. Fetches the issue body and comments
2. Runs `qa-analyze-pr.ts --type issue` to generate a QA guide
3. Runs the research phase (Claude writes E2E test to reproduce)
4. Records video of the test execution
5. Posts results as a comment on the issue
## Running Locally
### Step 1: Environment Setup
```bash
# Ensure ComfyUI server is running
# Default: http://127.0.0.1:8188
# Install Playwright browsers
npx playwright install chromium
```
### Step 2: Analyze the Issue/PR
```bash
# For a PR
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides
# For an issue
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides \
--type issue
```
### Step 3: Record Before/After
```bash
# Before (main branch)
pnpm exec tsx scripts/qa-record.ts \
--mode before \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-before \
--qa-guide qa-guides/qa-guide-1.json
# After (PR branch)
pnpm exec tsx scripts/qa-record.ts \
--mode after \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-after \
--qa-guide qa-guides/qa-guide-1.json
```
### Step 4: Review Videos
```bash
pnpm exec tsx scripts/qa-video-review.ts \
--artifacts-dir /tmp/qa-artifacts \
--video-file qa-session.mp4 \
--before-video qa-before-session.mp4 \
--output-dir /tmp/video-reviews \
--pr-context /tmp/pr-context.txt
```
## Research Phase Details (`qa-agent.ts`)
Claude receives:
- The issue description and comments
- A QA guide from `qa-analyze-pr.ts`
- An accessibility tree snapshot of the current UI
Claude's tools:
- **`inspect(selector?)`** — Read a11y tree to discover element selectors
- **`writeTest(code)`** — Write a Playwright `.spec.ts` file
- **`runTest()`** — Execute the test and get pass/fail + errors
- **`done(verdict, summary, evidence, testCode)`** — Finish with verdict
The test uses the project's Playwright fixtures (`comfyPageFixture`), giving access to `comfyPage.page`, `comfyPage.menu`, `comfyPage.settings`, etc.
### Verdict Logic
- **REPRODUCED** — Test passes (asserting the bug exists) → bug is proven
- **NOT_REPRODUCIBLE** — Claude exhausted attempts, test cannot pass
- **INCONCLUSIVE** — Agent timed out or encountered infrastructure issues
Auto-completion: if a test passed but `done()` was never called, the pipeline auto-completes with REPRODUCED.
## Manual QA (Fallback)
When the automated pipeline isn't suitable (e.g., visual-only bugs, complex multi-step interactions), use **playwright-cli** for manual browser interaction:
```bash
# Install
npm install -g @playwright/cli@latest
# Open browser and navigate
playwright-cli open http://127.0.0.1:8188
# Get element references
playwright-cli snapshot
# Interact
playwright-cli click e1
playwright-cli fill e2 "test text"
playwright-cli press Escape
playwright-cli screenshot --filename=f.png
```
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation to refresh refs.
## Manual QA Test Plan
When performing manual QA (either via playwright-cli or the automated pipeline), systematically test each area below.
### Application Load & Routes
| Test | Steps |
|---|---|
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### Canvas & Graph View
| Test | Steps |
|---|---|
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Context menus | Right-click node vs empty canvas — different menus |
### Sidebar Tabs
| Test | Steps |
|---|---|
| Workflows tab | Press W — workflows sidebar opens |
| Node Library tab | Press N — node library opens |
| Model Library tab | Press M — model library opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
### Settings Dialog
| Test | Steps |
|---|---|
| Open settings | Press Ctrl+, or click settings button |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Close settings | Press Escape or click close button |
### Execution & Queue
| Test | Steps |
|---|---|
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
## Report Site
Deployed to Cloudflare Pages at `https://comfy-qa.pages.dev/<branch>/`.
Features:
- Light/dark theme
- Seekable video player with preload
- Copy badge button (markdown)
- Date-stamped badges (e.g., `QA0327`)
- Vertical box badge for issues and PRs
## Known Issues & Troubleshooting
See `docs/qa/TROUBLESHOOTING.md` for common failures:
- `set -euo pipefail` + grep with no match → append `|| true`
- `__name is not defined` in `page.evaluate` → use `addScriptTag`
- Cursor not visible in videos → monkey-patch `page.mouse` methods
- Agent not calling `done()` → auto-complete from passing test
## Backlog
See `docs/qa/backlog.md` for planned improvements:
- **Type B comparison**: Different commits for regression detection
- **Type C comparison**: Cross-browser testing
- **Pre-seed assets**: Upload test images before recording
- **Lazy a11y tree**: Reduce token usage with `inspect(selector)` vs full dump

View File

@@ -1,246 +0,0 @@
---
name: hardening-flaky-e2e-tests
description: 'Diagnoses and fixes flaky Playwright e2e tests by replacing race-prone patterns with retry-safe alternatives. Use when triaging CI flakes, hardening spec files, fixing timing races, or asked to stabilize browser tests. Triggers on: flaky, flake, harden, stabilize, race condition in e2e, intermittent failure.'
---
# Hardening Flaky E2E Tests
Fix flaky Playwright specs by identifying race-prone patterns and replacing them with retry-safe alternatives. This skill covers diagnosis, pattern matching, and mechanical transforms — not writing new tests (see `writing-playwright-tests` for that).
## Workflow
### 1. Gather CI Evidence
```bash
gh run list --workflow=ci-test.yaml --limit=5
gh run download <run-id> -n playwright-report
```
- Open `report.json` and search for `"status": "flaky"` entries.
- Collect file paths, test titles, and error messages.
- Do NOT trust green checks alone — flaky tests that passed on retry still need fixing.
- Use `error-context.md`, traces, and page snapshots before editing code.
- Pull the newest run after each push instead of assuming the flaky set is unchanged.
### 2. Classify the Flake
Read the failing assertion and match it against the pattern table. Most flakes fall into one of these categories:
| # | Pattern | Signature in Code | Fix |
| --- | ------------------------------------- | --------------------------------------------------------- | ---------------------------------------------------------------- |
| 1 | **Snapshot-then-assert** | `expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
| 2 | **Immediate count** | `const n = await loc.count(); expect(n).toBe(3)` | `await expect(loc).toHaveCount(3)` |
| 3 | **nextFrame after menu click** | `clickMenuItem(x); nextFrame()` | `clickMenuItem(x); contextMenu.waitForHidden()` |
| 4 | **Tight poll timeout** | `expect.poll(..., { timeout: 250 })` | ≥2000 ms; prefer default 5000 ms |
| 5 | **Immediate evaluate after mutation** | `setSetting(k, v); expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
| 6 | **Screenshot without readiness** | `loadWorkflow(); nextFrame(); toHaveScreenshot()` | `waitForNodes()` or poll state first |
| 7 | **Non-deterministic node order** | `getNodeRefsByType('X')[0]` with >1 match | `getNodeRefById(id)` or guard `toHaveLength(1)` |
| 8 | **Fake readiness helper** | Helper clicks but doesn't assert state | Remove; poll the actual value |
| 9 | **Immediate graph state after drop** | `expect(await getLinkCount()).toBe(1)` | `await expect.poll(() => getLinkCount()).toBe(1)` |
| 10 | **Immediate boundingBox/layout read** | `const box = await loc.boundingBox(); expect(box!.width)` | `await expect.poll(() => loc.boundingBox().then(b => b?.width))` |
### 3. Apply the Transform
#### Rule: Choose the Smallest Correct Assertion
- **Locator state** → use built-in retrying assertions: `toBeVisible()`, `toHaveText()`, `toHaveCount()`, `toHaveClass()`
- **Single async value** → `expect.poll(() => asyncFn()).toBe(expected)`
- **Multiple assertions that must settle together** → `expect(async () => { ... }).toPass()`
- **Never** use `waitForTimeout()` to hide a race.
```typescript
// ✅ Single value — use expect.poll
await expect
.poll(() => comfyPage.page.evaluate(() => window.app!.graph.links.length))
.toBe(3)
// ✅ Locator count — use toHaveCount
await expect(comfyPage.page.locator('.dom-widget')).toHaveCount(2)
// ✅ Multiple conditions — use toPass
await expect(async () => {
expect(await node1.getValue()).toBe('foo')
expect(await node2.getValue()).toBe('bar')
}).toPass({ timeout: 5000 })
```
#### Rule: Wait for the Real Readiness Boundary
Visible is not always ready. Prefer user-facing assertions when possible; poll internal state only when there is no UI surface to assert on.
Common readiness boundaries:
| After this action... | Wait for... |
| -------------------------------------- | ------------------------------------------------------------ |
| Canvas interaction (drag, click node) | `await comfyPage.nextFrame()` |
| Menu item click | `await contextMenu.waitForHidden()` |
| Workflow load | `await comfyPage.workflow.loadWorkflow(...)` (built-in wait) |
| Settings write | Poll the setting value with `expect.poll()` |
| Node pin/bypass/collapse toggle | `await expect.poll(() => nodeRef.isPinned()).toBe(true)` |
| Graph mutation (add/remove node, link) | Poll link/node count |
| Clipboard write | Poll pasted value |
| Screenshot | Ensure nodes are rendered: `waitForNodes()` or poll state |
#### Rule: Expose Locators for Retrying Assertions
When a helper returns a count via `await loc.count()`, callers can't use `toHaveCount()`. Expose the underlying `Locator` as a getter so callers choose between:
```typescript
// Helper exposes locator
get domWidgets(): Locator {
return this.page.locator('.dom-widget')
}
// Caller uses retrying assertion
await expect(comfyPage.domWidgets).toHaveCount(2)
```
Replace count methods with locator getters so callers can use retrying assertions directly.
#### Rule: Fix Check-then-Act Races in Helpers
```typescript
// ❌ Race: count can change between check and waitFor
const count = await locator.count()
if (count > 0) {
await locator.waitFor({ state: 'hidden' })
}
// ✅ Direct: waitFor handles both cases
await locator.waitFor({ state: 'hidden' })
```
#### Rule: Remove force:true from Clicks
`force: true` bypasses actionability checks, hiding real animation/visibility races. Remove it and fix the underlying timing issue.
```typescript
// ❌ Hides the race
await closeButton.click({ force: true })
// ✅ Surfaces the real issue — fix with proper wait
await closeButton.click()
await dialog.waitForHidden()
```
#### Rule: Handle Non-deterministic Element Order
When `getNodeRefsByType` returns multiple nodes, the order is not guaranteed. Don't use index `[0]` blindly.
```typescript
// ❌ Assumes order
const node = (await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode'))[0]
// ✅ Find by ID or proximity
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
let target = nodes[0]
for (const n of nodes) {
const pos = await n.getPosition()
if (Math.abs(pos.y - expectedY) < minDist) target = n
}
```
Or guard the assumption:
```typescript
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
expect(nodes).toHaveLength(1)
const node = nodes[0]
```
#### Rule: Use toPass for Timing-sensitive Dismiss Guards
Some UI elements (e.g. LiteGraph's graphdialog) have built-in dismiss delays. Retry the entire dismiss action:
```typescript
// ✅ Retry click+assert together
await expect(async () => {
await comfyPage.canvas.click({ position: { x: 10, y: 10 } })
await expect(dialog).toBeHidden({ timeout: 500 })
}).toPass({ timeout: 5000 })
```
### 4. Keep Changes Narrow
- Shared helpers should drive setup to a stable boundary.
- Do not encode one-spec timing assumptions into generic helpers.
- If a race only matters to one spec, prefer a local wait in that spec.
- If a helper fails before the real test begins, remove or relax the brittle precondition and let downstream UI interaction prove readiness.
### 5. Verify Narrowly
```bash
# Targeted rerun with repetition
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts --repeat-each 10
# Single test by line number (avoids grep quoting issues on Windows)
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts:42
```
- Use `--repeat-each 10` for targeted flake verification (use 20 for single test cases).
- Verify with the smallest command that exercises the flaky path.
### 6. Watch CI E2E Runs
After pushing, use `gh` to monitor the E2E workflow:
```bash
# Find the run for the current branch
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1
# Watch it live (blocks until complete, streams logs)
gh run watch <run-id>
# One-liner: find and watch the latest E2E run for the current branch
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId" | xargs gh run watch
```
On Windows (PowerShell):
```powershell
# One-liner equivalent
gh run watch (gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId")
```
After the run completes:
```bash
# Download the Playwright report artifact
gh run download <run-id> -n playwright-report
# View the run summary in browser
gh run view <run-id> --web
```
Also watch the unit test workflow in parallel if you changed helpers:
```bash
gh run list --workflow="CI: Tests Unit" --branch=$(git branch --show-current) --limit=1
```
### 7. Pre-merge Checklist
Before merging a flaky-test fix, confirm:
- [ ] The latest CI artifact was inspected directly
- [ ] The root cause is stated as a race or readiness mismatch
- [ ] The fix waits on the real readiness boundary
- [ ] The assertion primitive matches the job (poll vs toHaveCount vs toPass)
- [ ] The fix stays local unless a shared helper truly owns the race
- [ ] Local verification uses a targeted rerun
- [ ] No behavioral changes to the test — only timing/retry strategy updated
## Local Noise — Do Not Fix
These are local distractions, not CI root causes:
- Missing local input fixture files required by the test path
- Missing local models directory
- Teardown `EPERM` while restoring the local browser-test user data directory
- Local screenshot baseline differences on Windows
Rules:
- First confirm whether it blocks the exact flaky path under investigation.
- Do not commit temporary local assets used only for verification.
- Do not commit local screenshot baselines.

View File

@@ -171,7 +171,7 @@ test('canvas text rendering with many nodes', async ({ comfyPage }) => {
| ----------------- | ----------------------------------------------------- |
| Perf test file | `browser_tests/tests/performance.spec.ts` |
| PerformanceHelper | `browser_tests/fixtures/helpers/PerformanceHelper.ts` |
| Perf reporter | `browser_tests/fixtures/utils/perfReporter.ts` |
| Perf reporter | `browser_tests/helpers/perfReporter.ts` |
| CI workflow | `.github/workflows/ci-perf-report.yaml` |
| Report generator | `scripts/perf-report.ts` |
| Stats utilities | `scripts/perf-stats.ts` |

View File

@@ -1,156 +0,0 @@
---
name: reviewing-unit-tests
description: Use when reviewing Vitest unit-test diffs in ComfyUI_frontend, especially new mocks, store tests, component tests, or bugfix regression tests.
---
# Reviewing Unit Tests for ComfyUI_frontend
## Overview
Review for behavior and current repo rules, not motion. Compare to authoritative rules, not prior diffs or legacy snippets.
## Review Workflow
1. Identify the test type: component, store, composable, util, or bugfix regression.
2. Name the behavior the test proves. If you cannot say it in one sentence, request changes.
3. Open the authoritative doc section before judging structure.
4. Scan the red flags below.
5. State the verdict first. Name the failure mode. Cite the doc or rule.
## Source of Truth / Precedence
When docs and examples conflict, use this order:
1. Explicit repo rules, lint rules, and note blocks.
2. [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md)
3. Rule sections in [`docs/testing/unit-testing.md`](../../../docs/testing/unit-testing.md), [`docs/testing/store-testing.md`](../../../docs/testing/store-testing.md), and [`docs/testing/component-testing.md`](../../../docs/testing/component-testing.md)
4. Example snippets
5. Prior diffs
Apply these repo-specific clarifications:
- [`docs/testing/component-testing.md`](../../../docs/testing/component-testing.md) starts with the authoritative rule: new component tests use `@testing-library/vue` with `@testing-library/user-event`. The `@vue/test-utils` snippets below it are legacy examples.
- [`docs/testing/store-testing.md`](../../../docs/testing/store-testing.md) still contains `as any` examples. Treat them as legacy snippets, not approval for new or edited test code.
- If docs conflict, prefer the stricter newer rule and call out the doc ambiguity. Do not approve through it.
- Motion != fix.
## 30-Second Red Flags
| If you see... | Failure mode | Default action |
| ----------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------- |
| New `@vue/test-utils` import in a new component test | legacy test API | Request changes |
| `vi.mock('vue-i18n', ...)` | mocked i18n | Request changes |
| `as any`, `@ts-expect-error`, `as Mock`, `as ReturnType<typeof vi.fn>`, `as unknown as X` | unnecessary cast or type escape | Request changes unless the author proves no safer type exists |
| `getXMock()`, renamed wrapper, or helper that only returns a mocked value | alias-by-renaming | Request changes |
| `beforeEach` recreates the return object for a module-mocked composable or service | shared mock setup drift | Request changes |
| Assertions only check defaults, mock plumbing, or CSS hooks | non-behavioral test | Request changes |
| Bugfix test has no proof it fails on pre-fix code | unproven regression | Request changes |
## Rationalization Table
| Excuse | Reality |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------ |
| "I restructured the mocks" | If the indirection stayed, nothing improved. Flag `alias-by-renaming`. |
| "The docs do it" | Rule, note, and lint beat legacy snippet. Compare to the current rule, not the nearest example. |
| "TypeScript required the cast" | `vi.mocked()` usually narrows mock methods. Assertion-only references need no cast. |
| "Putting it in `beforeEach` is DRY" | Recreating module mock state in hooks hides singleton behavior and drifts from the documented pattern. |
| "It is only a nit" | Explicit repo-rule violations are never nits. |
| "No behavior changed, just cleanup" | Motion != fix. Ask what behavior got stronger. |
| "Mental revert is enough" | For bugfix tests, establish red on pre-fix code or ask the author to show it. |
## Mocking Rules
- Fail helpers that do not remove repeated setup, encode domain meaning, or simplify assertions. Barely earning the abstraction is not enough.
- For composables with reactive or singleton state, define stable mock state inside the `vi.mock()` factory. Access it per test via the composable itself. See [`docs/testing/unit-testing.md`](../../../docs/testing/unit-testing.md) "Mocking Composables with Reactive State".
- This does not ban local test data builders or per-test `vi.spyOn(...)`.
- Mock seams, not the project-owned module you are trying to exercise. For store tests, prefer real Pinia plus `createTestingPinia({ stubActions: false })` per [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md) and [`docs/testing/store-testing.md`](../../../docs/testing/store-testing.md).
### Alias-by-Renaming
```ts
// Before
const mockAdd = vi.hoisted(() => vi.fn())
// After: same indirection, new name
function getToastAddMock() {
return useToast().add
}
```
If the wrapper only renames or relays a mocked value, fail it. Inline the lookup at the call site or fetch the singleton mock via the documented pattern.
### `vi.mocked()` Scope
| Use case | `vi.mocked()` required? |
| --------------------------------------------------------------- | ----------------------- |
| `.mockReturnValue`, `.mockResolvedValue`, `.mockImplementation` | Yes |
| `.mock.calls`, `.mock.results` | Yes |
| `expect(fn).toHaveBeenCalled()` | No |
| `expect(fn).toHaveBeenCalledWith(...)` | No |
- Flag casts whenever `vi.mocked()` would narrow correctly.
- Do not add `vi.mocked()` around assertion-only references just for style.
### Reset Hygiene
- Flag per-mock `mockClear()` or `mockReset()` when `vi.clearAllMocks()` or `vi.resetAllMocks()` already runs in the relevant hook chain.
- Review for redundancy or broken state management. Do not bikeshed `clearAllMocks` vs `resetAllMocks` unless behavior depends on it.
### Third-Party Seams
- Distinguish trivial hooks from behavior-rich APIs.
- Mocking single-method third-party hooks like `primevue/usetoast` is usually acceptable.
- That exception does not justify mocking behavior-rich third-party modules.
### `vue-i18n`
- Never mock `vue-i18n` in component tests.
- Use real `createI18n` per [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md) and the shared [`testI18n`](../../../src/components/searchbox/v2/__test__/testUtils.ts) setup.
## Test-Body Rules
| Smell | Review bar |
| ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Change-detector test | Reject. Default values alone prove nothing. |
| Mock-only assertion | Accept collaborator-call assertions only when the call is the meaningful external effect and the test also exercises the triggering behavior. |
| Non-behavioral assertion | Reject tests that only check classes, utility hooks, or styling internals. |
| New component test using `@vue/test-utils` | Request changes. Use `@testing-library/vue` plus `@testing-library/user-event`. |
| `any`, `as any`, or `@ts-expect-error` in new or edited test code | Request changes unless the author proves no safer type exists. Legacy doc snippets do not authorize it. |
## Bugfix Regression Proof
For `fix:` PRs or bugfix diffs:
1. Identify the production change that fixes the bug.
2. Verify the new test fails on pre-fix code, or ask the author to show it.
3. If the test passes on broken code, request changes.
A regression test that never proves red does not pin the bug.
## Review Output Rules
- State verdict before procedural questions.
- Do not lead with approval language like `LGTM, just one nit` or `approve and move on?`.
- Name the failure mode directly: `alias-by-renaming`, `unnecessary cast`, `mocked i18n`, `mock-only assertion`, `unproven regression`.
- Link the authoritative doc section in the review comment.
- If an explicit repo rule, lint rule, or authoritative doc note is violated, do not downgrade it to "minor deviation" or "nit".
## Quick Reference
| When you see... | Read this |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| New `vi.mock(...)` for a composable | [`docs/testing/unit-testing.md`](../../../docs/testing/unit-testing.md) -> "Mocking Composables with Reactive State" |
| New store test or store mock | [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md) setup + [`docs/testing/store-testing.md`](../../../docs/testing/store-testing.md) |
| New component test | Top note in [`docs/testing/component-testing.md`](../../../docs/testing/component-testing.md) |
| `vue-i18n` in a component test | [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md) + [`src/components/searchbox/v2/__test__/testUtils.ts`](../../../src/components/searchbox/v2/__test__/testUtils.ts) |
| Cast around a mock | [`docs/guidance/typescript.md`](../../../docs/guidance/typescript.md) -> "Type Assertion Hierarchy" |
## Key Files to Read
| Purpose | Path |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------- |
| Composable mocking patterns | [`docs/testing/unit-testing.md`](../../../docs/testing/unit-testing.md) |
| Store testing patterns | [`docs/testing/store-testing.md`](../../../docs/testing/store-testing.md) |
| Repo-wide Vitest setup defaults | [`docs/testing/vitest-patterns.md`](../../../docs/testing/vitest-patterns.md) |
| Component testing rule for new tests | [`docs/testing/component-testing.md`](../../../docs/testing/component-testing.md) |
| Real i18n setup | [`src/components/searchbox/v2/__test__/testUtils.ts`](../../../src/components/searchbox/v2/__test__/testUtils.ts) |

View File

@@ -0,0 +1,361 @@
---
name: ticket-intake
description: 'Parse ticket URL (Notion or GitHub), extract all data, initialize pipeline run. Use when starting work on a new ticket or when asked to pick up a ticket.'
---
# Ticket Intake
Parses a ticket URL from supported sources (Notion or GitHub), extracts all relevant information, and creates a ticket in the pipeline API.
> **🚨 CRITICAL REQUIREMENT**: This skill MUST register the ticket in the Pipeline API and update the source (Notion/GitHub). If these steps are skipped, the entire pipeline breaks. See [Mandatory API Calls](#mandatory-api-calls-execute-all-three) below.
## Supported Sources
| Source | URL Pattern | Provider File |
| ------ | --------------------------------------------------- | --------------------- |
| Notion | `https://notion.so/...` `https://www.notion.so/...` | `providers/notion.md` |
| GitHub | `https://github.com/{owner}/{repo}/issues/{n}` | `providers/github.md` |
## Quick Start
When given a ticket URL:
1. **Detect source type** from URL pattern
2. **Load provider-specific logic** from `providers/` directory
3. Fetch ticket content via appropriate API
4. Extract and normalize properties to common schema
5. **Register ticket in pipeline API** ← MANDATORY
6. **Update source** (Notion status / GitHub comment) ← MANDATORY
7. **Run verification script** to confirm API registration
8. Output summary and handoff to `research-orchestrator`
## Configuration
Uses the **production API** by default. No configuration needed for read operations.
**Defaults (no setup required):**
- API URL: `https://api-gateway-856475788601.us-central1.run.app`
- Read-only endpoints at `/public/*` require no authentication
**For write operations** (transitions, creating tickets), set:
```bash
export PIPELINE_API_KEY="..." # Get from GCP Secret Manager or ask admin
```
**Optional (for local working artifacts):**
```bash
PIPELINE_DIR="${PIPELINE_DIR:-$HOME/repos/ticket-to-pr-pipeline}"
```
## Mandatory API Calls (Execute ALL Three)
**⚠️ These three API calls are the ENTIRE POINT of this skill. Without them, the ticket is invisible to the pipeline, downstream skills will fail, and Notion status won't update.**
**You MUST make these HTTP requests.** Use `curl` from bash — do not just read this as documentation.
### Call 1: Create Ticket
```bash
API_URL="${PIPELINE_API_URL:-https://api-gateway-856475788601.us-central1.run.app}"
API_KEY="${PIPELINE_API_KEY}"
curl -s -X POST "${API_URL}/v1/tickets" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"notion_page_id": "NOTION_PAGE_UUID_HERE",
"title": "TICKET_TITLE_HERE",
"source": "notion",
"metadata": {
"description": "DESCRIPTION_HERE",
"priority": "High",
"labels": [],
"acceptanceCriteria": []
}
}'
```
Save the returned `id` — you need it for the next two calls.
### Call 2: Transition to RESEARCH
```bash
TICKET_ID="id-from-step-1"
curl -s -X POST "${API_URL}/v1/tickets/${TICKET_ID}/transition" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"to_state": "RESEARCH",
"reason": "Intake complete, starting research"
}'
```
### Call 3: Queue Source Update
```bash
curl -s -X POST "${API_URL}/v1/sync/queue" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"ticket_id": "TICKET_ID_HERE",
"action": "update_status",
"payload": { "status": "In Progress" },
"priority": "normal"
}'
```
> **Note:** The action MUST be `"update_status"` (not `"UPDATE_NOTION_STATUS"`). Valid actions: `update_status`, `update_pr_url`, `mark_done`.
### TypeScript Equivalent (if using pipeline client)
```typescript
import { PipelineClient } from '@pipeline/client'
const client = new PipelineClient({
apiUrl:
process.env.PIPELINE_API_URL ||
'https://api-gateway-856475788601.us-central1.run.app',
agentId: process.env.AGENT_ID!
})
const ticket = await client.createTicket({
notion_page_id: pageId,
title: ticketTitle,
source: 'notion',
metadata: { description, priority, labels, acceptanceCriteria }
})
await client.transitionState(
ticket.id,
'RESEARCH',
'Intake complete, starting research'
)
await client.queueSync(ticket.id, 'update_status', { status: 'In Progress' })
```
## Workflow
### Step 1: Detect Source Type
Parse the URL to determine source:
```javascript
if (url.includes('notion.so')) {
source = 'notion'
// Load providers/notion.md
} else if (url.match(/github\.com\/[^\/]+\/[^\/]+\/issues\/\d+/)) {
source = 'github'
// Load providers/github.md
} else {
// Error: Unsupported URL format
}
```
### Step 2: Load Provider and Fetch Data
Read the appropriate provider file for source-specific instructions:
- **Notion**: `providers/notion.md` - Uses Notion MCP, handles Slack links
- **GitHub**: `providers/github.md` - Uses `gh` CLI, handles Dosu comments
Follow the provider's instructions for:
- Fetching content
- Extracting properties
- **Updating the source** (Notion status → "In Progress", Assignee → pipeline owner)
### Step 3: Normalize to Common Schema
All providers must extract normalized ticket data following `schema.md`:
```json
{
"id": "abc12345",
"url": "https://...",
"source": "notion | github",
"title": "Ticket title",
"description": "Full description",
"status": "Not Started",
"assignee": "username",
"priority": "High",
"area": "UI",
"labels": ["bug", "frontend"],
"acceptanceCriteria": ["Criterion 1", "Criterion 2"],
"fetchedAt": "2024-01-15T10:30:00Z"
}
```
### Step 4: Register Ticket in Pipeline API (MANDATORY — DO NOT SKIP)
**Execute all three API calls from [Mandatory API Calls](#mandatory-api-calls-execute-all-three) above.**
This is not optional. This is not documentation. You MUST make these HTTP requests right now.
1. `createTicket()` → save the returned ticket ID
2. `transitionState(id, 'RESEARCH')` → confirm state changed
3. `queueSync(id, 'update_status', { status: 'In Progress' })` → confirm queued
**If any call fails**, retry once. If it still fails, report the error prominently — do NOT silently continue.
### Step 5: Run Verification Script
After making the API calls, run the verification script to confirm everything worked:
```bash
bash scripts/verify-intake.sh TICKET_ID_OR_NOTION_PAGE_ID
```
**If the script is not available locally**, verify manually via the public API:
```bash
curl -s "${API_URL}/public/tickets/${TICKET_ID}" | jq '{id, state, title, notion_page_id}'
```
Expected output:
```json
{
"id": "...",
"state": "RESEARCH",
"title": "...",
"notion_page_id": "..."
}
```
**If `state` is not `RESEARCH`, go back to Step 4 and complete the missing calls.**
### Step 6: Output Summary and Handoff
Print a clear summary:
```markdown
## Ticket Intake Complete
**Source:** Notion | GitHub
**Title:** [Ticket title]
**ID:** abc12345
**Status:** In Progress (queued)
**Priority:** High
**Area:** UI
### Description
[Brief description or first 200 chars]
### Acceptance Criteria
- [ ] Criterion 1
- [ ] Criterion 2
### Links
- **Ticket:** [Original URL]
- **Slack:** [Slack thread content fetched via slackdump] (Notion only)
### Pipeline
- **API Ticket ID:** abc12345
- **State:** RESEARCH
- **Verified:** ✅ (via verify-intake.sh or public API)
```
**After printing the summary, immediately handoff** to continue the pipeline. Use the `handoff` tool with all necessary context (ticket ID, source, title, description, slack context if any):
> **Handoff goal:** "Continue pipeline for ticket {ID} ({title}). Ticket is in RESEARCH state. Load skill: `research-orchestrator` to begin research phase. Ticket data: source={source}, notion_page_id={pageId}, priority={priority}. {slack context summary if available}"
**Do NOT wait for human approval to proceed.** The intake phase is complete — handoff immediately.
## Error Handling
### Unsupported URL
```
❌ Unsupported ticket URL format.
Supported formats:
- Notion: https://notion.so/... or https://www.notion.so/...
- GitHub: https://github.com/{owner}/{repo}/issues/{number}
Received: [provided URL]
```
### Provider-Specific Errors
See individual provider files for source-specific error handling:
- `providers/notion.md` - Authentication, page not found
- `providers/github.md` - Auth, rate limits, issue not found
### Missing Properties
Continue with available data and note what's missing:
```
⚠️ Some properties unavailable:
- Priority: not found (using default: Medium)
- Area: not found
Proceeding with available data...
```
### API Call Failures
```
❌ Pipeline API call failed: {method} {endpoint}
Status: {status}
Error: {message}
Retrying once...
❌ Retry also failed. INTAKE IS INCOMPLETE.
The ticket was NOT registered in the pipeline.
Downstream skills will not work until this is fixed.
```
## Notes
- This skill focuses ONLY on intake — it does not do research
- Slack thread content is fetched automatically via the `slackdump` skill — no manual copy-paste needed
- ALL API calls (createTicket, transitionState, queueSync) are MANDATORY — never skip them
- The `queueSync` action must be `"update_status"`, NOT `"UPDATE_NOTION_STATUS"`
- Pipeline state is tracked via the API, not local files
- Working artifacts (research-report.md, plan.md) can be saved locally to `$PIPELINE_DIR/runs/{ticket-id}/`
- The `source` field in the ticket determines which research strategies to use
## API Client Reference
### Available Methods
| Method | Description |
| ----------------------------------------------------------- | ------------------------------------------------------------------- |
| `createTicket({ notion_page_id, title, source, metadata })` | Create a new ticket in the API |
| `getTicket(id)` | Retrieve a ticket by ID |
| `findByNotionId(notionPageId)` | Look up a ticket by its Notion page ID |
| `listTickets({ state, agent_id, limit, offset })` | List tickets with optional filters |
| `transitionState(id, state, reason)` | Move ticket to a new state (e.g., `'RESEARCH'`) |
| `setPRCreated(id, prUrl)` | Mark ticket as having a PR created |
| `queueSync(id, action, payload)` | Queue a sync action (`update_status`, `update_pr_url`, `mark_done`) |
| `registerBranch(id, branch, repo)` | Register working branch for automatic PR detection |
### Error Handling
```typescript
import { PipelineClient, PipelineAPIError } from '@pipeline/client';
try {
await client.createTicket({ ... });
} catch (error) {
if (error instanceof PipelineAPIError) {
console.error(`API Error ${error.status}: ${error.message}`);
}
throw error;
}
```

View File

@@ -0,0 +1,194 @@
# GitHub Provider - Ticket Intake
Provider-specific logic for ingesting tickets from GitHub Issues.
## URL Pattern
```
https://github.com/{owner}/{repo}/issues/{number}
https://www.github.com/{owner}/{repo}/issues/{number}
```
Extract: `owner`, `repo`, `issue_number` from URL.
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
- Access to the repository
## Fetch Issue Content
Use `gh` CLI to fetch issue details:
```bash
# Get issue details in JSON
gh issue view {number} --repo {owner}/{repo} --json title,body,state,labels,assignees,milestone,author,createdAt,comments,linkedPRs
# Get comments separately if needed
gh issue view {number} --repo {owner}/{repo} --comments
```
## Extract Ticket Data
Map GitHub issue fields to normalized ticket data (stored via API):
| GitHub Field | ticket.json Field | Notes |
| ------------ | ----------------- | -------------------------- |
| title | title | Direct mapping |
| body | description | Issue body/description |
| state | status | Map: open → "Not Started" |
| labels | labels | Array of label names |
| assignees | assignee | First assignee login |
| author | author | Issue author login |
| milestone | milestone | Milestone title if present |
| comments | comments | Array of comment objects |
| linkedPRs | linkedPRs | PRs linked to this issue |
### Priority Mapping
Infer priority from labels:
- `priority:critical`, `P0` → "Critical"
- `priority:high`, `P1` → "High"
- `priority:medium`, `P2` → "Medium"
- `priority:low`, `P3` → "Low"
- No priority label → "Medium" (default)
### Area Mapping
Infer area from labels:
- `area:ui`, `frontend`, `component:*` → "UI"
- `area:api`, `backend` → "API"
- `area:docs`, `documentation` → "Docs"
- `bug`, `fix` → "Bug"
- `enhancement`, `feature` → "Feature"
## Update Source
**For GitHub issues, update is optional but recommended.**
Add a comment to indicate work has started:
```bash
gh issue comment {number} --repo {owner}/{repo} --body "🤖 Pipeline started processing this issue."
```
Optionally assign to self:
```bash
gh issue edit {number} --repo {owner}/{repo} --add-assignee @me
```
Log any updates via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
githubWrites: [
...(ticket.metadata?.githubWrites || []),
{
action: 'comment',
issueNumber: 123,
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
## GitHub-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "github",
"githubOwner": "Comfy-Org",
"githubRepo": "ComfyUI_frontend",
"githubIssueNumber": 123,
"githubIssueUrl": "https://github.com/Comfy-Org/ComfyUI_frontend/issues/123",
"labels": ["bug", "area:ui", "priority:high"],
"linkedPRs": [456, 789],
"dosuComment": "..." // Extracted Dosu bot analysis if present
}
```
## Dosu Bot Detection
Many repositories use Dosu bot for automated issue analysis. Check comments for Dosu:
```bash
gh issue view {number} --repo {owner}/{repo} --comments | grep -A 100 "dosu"
```
Look for comments from:
- `dosu[bot]`
- `dosu-bot`
Extract Dosu analysis which typically includes:
- Root cause analysis
- Suggested files to modify
- Related issues/PRs
- Potential solutions
Store in ticket data via API:
```json
{
"dosuComment": {
"found": true,
"analysis": "...",
"suggestedFiles": ["src/file1.ts", "src/file2.ts"],
"relatedIssues": [100, 101]
}
}
```
## Extract Linked Issues/PRs
Parse issue body and comments for references:
- `#123` → Issue or PR reference
- `fixes #123`, `closes #123` → Linked issue
- `https://github.com/.../issues/123` → Full URL reference
Store in ticket data via API for research phase:
```json
{
"referencedIssues": [100, 101, 102],
"referencedPRs": [200, 201]
}
```
## Error Handling
### Authentication Error
```
⚠️ GitHub CLI not authenticated.
Run: gh auth login
```
### Issue Not Found
```
❌ GitHub issue not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this repository
- Run: gh auth status
```
### Rate Limiting
```
⚠️ GitHub API rate limited.
Wait a few minutes and try again.
Check status: gh api rate_limit
```

View File

@@ -0,0 +1,202 @@
# Notion Provider - Ticket Intake
Provider-specific logic for ingesting tickets from Notion.
## URL Pattern
```
https://www.notion.so/workspace/Page-Title-abc123def456...
https://notion.so/Page-Title-abc123def456...
https://www.notion.so/abc123def456...
```
Page ID is the 32-character hex string (with or without hyphens).
## Prerequisites
- Notion MCP connected and authenticated
- If not setup: `claude mcp add --transport http notion https://mcp.notion.com/mcp`
- Authenticate via `/mcp` command if prompted
## Fetch Ticket Content
Use `Notion:notion-fetch` with the page URL or ID:
```
Fetch the full page content including all properties
```
## Extract Ticket Data
Extract these properties (names may vary):
| Property | Expected Name | Type |
| ------------- | ------------------------- | ------------ |
| Title | Name / Title | Title |
| Status | Status | Select |
| Assignee | Assignee / Assigned To | Person |
| Description | - | Page content |
| Slack Link | Slack Link / Slack Thread | URL |
| GitHub PR | GitHub PR / PR Link | URL |
| Priority | Priority | Select |
| Area | Area / Category | Select |
| Related Tasks | Related Tasks | Relation |
**If properties are missing**: Note what's unavailable and continue with available data.
## Update Source (REQUIRED)
**⚠️ DO NOT SKIP THIS STEP. This is a required action, not optional.**
**⚠️ Notion Write Safety rules apply (see `$PIPELINE_DIR/docs/notion-write-safety.md` for full reference):**
- **Whitelist**: Only `Status`, `GitHub PR`, and `Assignee` fields may be written
- **Valid transitions**: Not Started → In Progress, In Progress → In Review, In Review → Done
- **Logging**: Every write attempt MUST be logged with timestamp, field, value, previous value, skill name, and success status
Use `Notion:notion-update-page` to update the ticket:
1. **Status**: Set to "In Progress" (only valid from "Not Started")
2. **Assignee**: Assign to pipeline owner (Notion ID: `175d872b-594c-81d4-ba5a-0002911c5966`)
```json
{
"page_id": "{page_id_from_ticket}",
"command": "update_properties",
"properties": {
"Status": "In Progress",
"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"
}
}
```
**After the update succeeds**, log the write via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
notionWrites: [
...(ticket.metadata?.notionWrites || []),
{
field: 'Status',
value: 'In Progress',
previousValue: 'Not Started',
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
If update fails, log with `success: false` and continue.
## Notion-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "notion",
"notionPageId": "abc123def456...",
"slackLink": "https://slack.com/...",
"relatedTasks": ["page-id-1", "page-id-2"]
}
```
## Slack Thread Handling
If a Slack link exists, use the `slackdump` skill to fetch the thread content programmatically.
### Slack URL Conversion
Notion stores Slack links in `slackMessage://` format:
```
slackMessage://comfy-organization.slack.com/CHANNEL_ID/THREAD_TS/MESSAGE_TS
```
Convert to browser-clickable format:
```
https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS_NO_DOT
```
**Example:**
- Input: `slackMessage://comfy-organization.slack.com/C075ANWQ8KS/1766022478.450909/1764772881.854829`
- Output: `https://comfy-organization.slack.com/archives/C075ANWQ8KS/p1764772881854829`
(Remove the dot from the last timestamp and prefix with `p`)
### Fetching Thread Content
Load the `slackdump` skill and use the **export-thread** workflow:
```bash
# Export thread by URL
slackdump dump "https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS"
# Or by colon notation (channel_id:thread_ts)
slackdump dump CHANNEL_ID:THREAD_TS
```
Save the thread content to `$RUN_DIR/slack-context.md` and include it in the ticket metadata.
> **No manual action required.** The slackdump CLI handles authentication via stored credentials at `~/.cache/slackdump/comfy-organization.bin`.
## Database Reference: Comfy Tasks
The "Comfy Tasks" database has these properties (verify via `notion-search`):
- **Status values**: Not Started, In Progress, In Review, Done
- **Team assignment**: "Frontend Team" for unassigned tickets
- **Filtering note**: Team filtering in Notion may have quirks - handle gracefully
### Pipeline Owner Details
When assigning tickets, use these identifiers:
| Platform | Identifier |
| --------------- | -------------------------------------- |
| Notion User ID | `175d872b-594c-81d4-ba5a-0002911c5966` |
| Notion Name | Christian Byrne |
| Notion Email | cbyrne@comfy.org |
| Slack User ID | U087MJCDHHC |
| GitHub Username | christian-byrne |
**To update Assignee**, use the Notion User ID (not name):
```
properties: {"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"}
```
### Finding Active Tickets
To list your active tickets:
```
Use Notion:notion-search for "Comfy Tasks"
Filter by Assignee = current user OR Team = "Frontend Team"
```
## Error Handling
### Authentication Error
```
⚠️ Notion authentication required.
Run: claude mcp add --transport http notion https://mcp.notion.com/mcp
Then authenticate via /mcp command.
```
### Page Not Found
```
❌ Notion page not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this page
- Try re-authenticating via /mcp
```

View File

@@ -0,0 +1,81 @@
# Ticket Schema
Common schema for normalized ticket data across all sources. This data is stored and retrieved via the Pipeline API, not local files.
## Ticket Data Schema
```json
{
// Required fields (all sources)
"id": "string", // Unique identifier (short form)
"url": "string", // Original URL
"source": "notion | github", // Source type
"title": "string", // Ticket title
"description": "string", // Full description/body
"fetchedAt": "ISO8601", // When ticket was fetched
// Common optional fields
"status": "string", // Current status
"assignee": "string", // Assigned user
"priority": "string", // Priority level
"area": "string", // Category/area
"labels": ["string"], // Tags/labels
"acceptanceCriteria": ["string"] // List of AC items
// Source-specific fields (see providers)
// Notion: notionPageId, slackLink, relatedTasks, notionWrites
// GitHub: githubOwner, githubRepo, githubIssueNumber, linkedPRs, dosuComment, referencedIssues
}
```
## Ticket State Schema (via API)
State is managed via the Pipeline API using `client.transitionState()`:
```json
{
"ticketId": "string",
"state": "intake | research | planning | implementation | pr_created | done | failed",
"stateChangedAt": "ISO8601",
// Timestamps tracked by API
"createdAt": "ISO8601",
"updatedAt": "ISO8601"
}
```
## Priority Normalization
All sources should normalize to these values:
| Normalized | Description |
| ---------- | ------------------------- |
| Critical | Production down, security |
| High | Blocking work, urgent |
| Medium | Normal priority (default) |
| Low | Nice to have, backlog |
## Status Normalization
Pipeline tracks these statuses internally:
| Status | Description |
| -------------- | ---------------------------- |
| research | Gathering context |
| planning | Creating implementation plan |
| implementation | Writing code |
| review | Code review in progress |
| qa | Quality assurance |
| done | PR merged or completed |
## ID Generation
IDs are generated by the API when creating tickets. For reference:
- **Notion**: First 8 characters of page ID
- **GitHub**: `gh-{owner}-{repo}-{issue_number}` (sanitized)
Examples:
- Notion: `abc12345`
- GitHub: `gh-comfy-org-frontend-123`

View File

@@ -114,7 +114,7 @@ await expect(async () => {
## CI Debugging
1. Download artifacts from failed CI run
2. Extract and view trace: `pnpm dlx playwright show-trace trace.zip`
2. Extract and view trace: `npx playwright show-trace trace.zip`
3. CI deploys HTML report to Cloudflare Pages (link in PR comment)
4. Reproduce CI: `CI=true pnpm test:browser`
5. Local runs: `pnpm test:browser:local`

View File

@@ -19,26 +19,15 @@ reviews:
- name: End-to-end regression coverage for fixes
mode: error
instructions: |
Use only PR metadata already available in the review context:
- the PR title
- commit subjects in this PR
- The files changed in this PR relative to the PR base (equivalent to `base...head`)
- the PR description.
Do not rely on shell commands.
Do not inspect reverse diffs, files changed only on the base branch, or files outside this PR.
If the changed-file list or commit subjects are unavailable, mark the check inconclusive instead of guessing.
Use only PR metadata already available in the review context: the PR title, commit subjects in this PR, the files changed in this PR relative to the PR base (equivalent to `base...head`), and the PR description.
Do not rely on shell commands. Do not inspect reverse diffs, files changed only on the base branch, or files outside this PR. If the changed-file list or commit subjects are unavailable, mark the check inconclusive instead of guessing.
Fail if all of the following are true:
1. The PR title and/or any commit subject in the PR uses bug-fix language such as `fix`, `fixed`, `fixes`, `fixing`, `bugfix`, or `hotfix`.
2. The PR changes files under `src/` or `packages/` related to the main frontend application but the PR does not change at least one file under `browser_tests/`.
3. The PR description lacks a concrete explanation of why an end-to-end regression test was not added.
Do not fail if the changes are exclusively in `apps/website`, just documentation changes, or changes related to CI processes.
The goal is to make sure that fixes include End-to-End regression tests. Do not insist on tests when the PR is not fixing a bug.
Pass otherwise.
When failing, mention which bug-fix signal you found and ask the author to either add or update a Playwright regression test under `browser_tests/` or add a concrete explanation in the PR description of why an end-to-end regression test is not practical.
Pass if at least one of the following is true:
1. Neither the PR title nor any commit subject in the PR uses bug-fix language such as `fix`, `fixed`, `fixes`, `fixing`, `bugfix`, or `hotfix`.
2. The PR changes at least one file under `browser_tests/`.
3. The PR description includes a concrete, non-placeholder explanation of why an end-to-end regression test was not added.
Fail otherwise. When failing, mention which bug-fix signal you found and ask the author to either add or update a Playwright regression test under `browser_tests/` or add a concrete explanation in the PR description of why an end-to-end regression test is not practical.
- name: ADR compliance for entity/litegraph changes
mode: warning
instructions: |

View File

@@ -1,12 +0,0 @@
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[*.md]
trim_trailing_whitespace = false

View File

@@ -46,9 +46,3 @@ ALGOLIA_API_KEY=684d998c36b67a9a9fce8fc2d8860579
# SENTRY_ORG=comfy-org
# SENTRY_PROJECT=cloud-frontend-staging
# SENTRY_PROJECT_PROD= # prod project slug for sourcemap uploads
# Ashby (apps/website careers page build).
# Server-only; read inside the Astro build context. Do NOT prefix with PUBLIC_.
# When unset, the committed snapshot at apps/website/src/data/ashby-roles.snapshot.json is used.
# WEBSITE_ASHBY_API_KEY=
# WEBSITE_ASHBY_JOB_BOARD_NAME=comfy-org

View File

@@ -1,23 +0,0 @@
name: Ashby Pull
description: 'Refresh the apps/website Ashby roles snapshot from the Ashby job board API'
inputs:
api_key:
description: 'Ashby API key (WEBSITE_ASHBY_API_KEY).'
required: true
job_board_name:
description: 'Ashby job board name (WEBSITE_ASHBY_JOB_BOARD_NAME).'
required: true
runs:
using: 'composite'
steps:
# Note: this action assumes the frontend repo is checked out at the workspace root.
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Refresh Ashby snapshot
shell: bash
env:
WEBSITE_ASHBY_API_KEY: ${{ inputs.api_key }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ inputs.job_board_name }}
run: pnpm --filter @comfyorg/website ashby:refresh-snapshot

View File

@@ -1,87 +0,0 @@
# Outputs default to 'true' for non-pull_request events (push, merge_group):
# granular path filtering is a PR-only optimization. This avoids the silent
# skip footgun where a job gated on e.g. `app-website-changes == 'true'`
# would never run on push.
#
# Shared dependency files (root package.json, pnpm-lock.yaml,
# pnpm-workspace.yaml) are folded into every app-* and packages-changes
# output so a lockfile bump correctly invalidates each granular gate. They
# are NOT folded into docs-changes.
#
# Two paths-filter steps are needed because predicate-quantifier=every is
# required for the negated globs in `should-run` but breaks multi-pattern
# OR filters like `docs:` and `deps:`.
#
# Requires the caller to have checked out the repository.
name: 'Detect Path Changes'
description: >
Computes typed *-changes outputs and a back-compat should-run for
path-gated CI jobs.
outputs:
should-run:
description: 'Any file outside `apps/`, `docs/`, `.storybook/`, or `**/*.md` changed.'
value: ${{ github.event_name != 'pull_request' || steps.relevant.outputs.relevant == 'true' }}
app-website-changes:
description: 'Shared deps or `apps/website/**` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' || steps.filter.outputs.app_website == 'true' }}
app-desktop-changes:
description: 'Shared deps or `apps/desktop-ui/**` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' || steps.filter.outputs.app_desktop == 'true' }}
app-frontend-changes:
description: 'Shared deps or `src/**` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' || steps.filter.outputs.app_frontend == 'true' }}
packages-changes:
description: 'Shared deps or `packages/**` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' || steps.filter.outputs.packages == 'true' }}
storybook-changes:
description: 'Shared deps or `.storybook/**` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' || steps.filter.outputs.storybook == 'true' }}
docs-changes:
description: '`docs/**` or any `**/*.md` changed (deps NOT folded in).'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.docs == 'true' }}
dependency-changes:
description: 'Root `package.json`, `pnpm-lock.yaml`, or `pnpm-workspace.yaml` changed.'
value: ${{ github.event_name != 'pull_request' || steps.filter.outputs.deps == 'true' }}
runs:
using: composite
steps:
- name: Filter typed changes
if: ${{ github.event_name == 'pull_request' }}
id: filter
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
app_website:
- 'apps/website/**'
app_desktop:
- 'apps/desktop-ui/**'
app_frontend:
- 'src/**'
packages:
- 'packages/**'
storybook:
- '.storybook/**'
docs:
- 'docs/**'
- '**/*.md'
deps:
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- name: Filter relevant changes
if: ${{ github.event_name == 'pull_request' }}
id: relevant
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
predicate-quantifier: 'every'
filters: |
relevant:
- '**'
- '!apps/**'
- '!docs/**'
- '!.storybook/**'
- '!**/*.md'

View File

@@ -1,65 +0,0 @@
name: Find Workflow Run
description: Finds a workflow run for a given commit SHA and outputs its status and run ID.
inputs:
workflow-id:
description: The workflow filename (e.g., 'ci-size-data.yaml')
required: true
head-sha:
description: The commit SHA to find runs for
required: true
not-found-status:
description: Status to output when no run exists
required: false
default: pending
token:
description: GitHub token for API access
required: true
outputs:
status:
description: One of 'ready', 'pending', 'failed', or the not-found-status value
value: ${{ steps.find.outputs.status }}
run-id:
description: The workflow run ID (only set when status is 'ready')
value: ${{ steps.find.outputs.run-id }}
runs:
using: composite
steps:
- name: Find workflow run
id: find
uses: actions/github-script@v8
env:
WORKFLOW_ID: ${{ inputs.workflow-id }}
HEAD_SHA: ${{ inputs.head-sha }}
NOT_FOUND_STATUS: ${{ inputs.not-found-status }}
with:
github-token: ${{ inputs.token }}
script: |
const { data: runs } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: process.env.WORKFLOW_ID,
head_sha: process.env.HEAD_SHA,
per_page: 1,
});
const run = runs.workflow_runs[0];
if (!run) {
core.setOutput('status', process.env.NOT_FOUND_STATUS);
return;
}
if (run.status !== 'completed') {
core.setOutput('status', 'pending');
return;
}
if (run.conclusion !== 'success') {
core.setOutput('status', 'failed');
return;
}
core.setOutput('status', 'ready');
core.setOutput('run-id', String(run.id));

View File

@@ -1,31 +0,0 @@
name: 'Lint and format verify'
description: >
Runs the lint/format/knip verification suite plus a conditional
browser-tests typecheck. Shared by ci-lint-format.yaml (PR) and
ci-lint-format-queue.yaml (merge queue) so both paths run the exact
same checks. The caller is responsible for checkout and frontend setup
before invoking this action.
runs:
using: composite
steps:
- name: Detect browser_tests changes
id: changed-paths
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
browser_tests:
- 'browser_tests/**'
- name: Verify lint and format
shell: bash
run: |
pnpm lint
pnpm stylelint
pnpm format:check
pnpm knip
- name: Typecheck browser tests
if: steps.changed-paths.outputs.browser_tests == 'true'
shell: bash
run: pnpm typecheck:browser

View File

@@ -1,88 +0,0 @@
name: Resolve PR from workflow_run
description: >
Resolves the PR number from a workflow_run event using pull_requests[0]
with a listPullRequestsAssociatedWithCommit fallback.
Skips closed/merged PRs and stale runs (head SHA mismatch).
inputs:
token:
description: GitHub token for API calls
required: false
default: ${{ github.token }}
outputs:
skip:
description: "'true' when no open PR was found or the run is stale"
value: ${{ steps.resolve.outputs.skip }}
number:
description: The PR number (empty when skip is true)
value: ${{ steps.resolve.outputs.number }}
base:
description: The PR base branch (empty when skip is true)
value: ${{ steps.resolve.outputs.base }}
head-sha:
description: The PR head SHA (empty when skip is true)
value: ${{ steps.resolve.outputs.head-sha }}
runs:
using: composite
steps:
- name: Resolve PR
id: resolve
uses: actions/github-script@v8
with:
github-token: ${{ inputs.token }}
script: |
let pr = context.payload.workflow_run.pull_requests?.[0];
if (!pr) {
const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({
owner: context.repo.owner,
repo: context.repo.repo,
commit_sha: context.payload.workflow_run.head_sha,
});
pr = prs.find(p => p.state === 'open');
}
// Fork PRs: pull_requests is empty and commit SHA may not be in
// the base repo graph. Fall back to pulls.list with head filter.
if (!pr && context.payload.workflow_run.head_repository?.owner?.login) {
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
head: `${context.payload.workflow_run.head_repository.owner.login}:${context.payload.workflow_run.head_branch}`,
per_page: 1,
});
pr = prs.find(p => p.head.sha === context.payload.workflow_run.head_sha);
}
if (!pr) {
core.info('No open PR found for this workflow run — skipping.');
core.setOutput('skip', 'true');
return;
}
const { data: livePr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
});
if (livePr.state !== 'open') {
core.info(`PR #${pr.number} is ${livePr.state} — skipping.`);
core.setOutput('skip', 'true');
return;
}
if (livePr.head.sha !== context.payload.workflow_run.head_sha) {
core.info(
`Stale run: workflow SHA ${context.payload.workflow_run.head_sha} != PR head ${livePr.head.sha}`
);
core.setOutput('skip', 'true');
return;
}
core.setOutput('base', livePr.base.ref);
core.setOutput('head-sha', livePr.head.sha);
core.setOutput('skip', 'false');
core.setOutput('number', String(pr.number));

View File

@@ -44,12 +44,17 @@ runs:
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Start ComfyUI server
if: ${{ inputs.launch_server == 'true' }}
shell: bash
working-directory: ComfyUI
env:
EXTRA_SERVER_PARAMS: ${{ inputs.extra_server_params }}
run: |
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
wait-for-it --service 127.0.0.1:8188 -t 600
python main.py --cpu --multi-user --front-end-root ../dist $EXTRA_SERVER_PARAMS &
for i in $(seq 1 300); do
curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1 && echo "Server ready" && exit 0
sleep 2
done
echo "::error::ComfyUI server did not start within 600s" && exit 1

View File

@@ -1,79 +0,0 @@
name: Upsert Comment Section
description: >
Manage a consolidated PR comment with independently-updatable sections.
All website CI workflows share the marker <!-- WEBSITE_CI_REPORT -->.
Valid section names: "e2e", "preview", "screenshot-update".
inputs:
pr-number:
description: PR number to comment on
required: true
section-name:
description: 'Section identifier: "e2e", "preview", or "screenshot-update"'
required: true
section-content:
description: Markdown content for this section
required: true
comment-marker:
description: Top-level HTML comment marker (must be <!-- WEBSITE_CI_REPORT --> for all callers)
required: true
token:
description: GitHub token with pull-requests write permission
required: true
runs:
using: composite
steps:
- uses: actions/github-script@v8
env:
INPUT_PR_NUMBER: ${{ inputs.pr-number }}
INPUT_SECTION_NAME: ${{ inputs.section-name }}
INPUT_SECTION_CONTENT: ${{ inputs.section-content }}
INPUT_COMMENT_MARKER: ${{ inputs.comment-marker }}
with:
github-token: ${{ inputs.token }}
script: |
const prNumber = Number(process.env.INPUT_PR_NUMBER)
const sectionName = process.env.INPUT_SECTION_NAME
const sectionContent = process.env.INPUT_SECTION_CONTENT
const commentMarker = process.env.INPUT_COMMENT_MARKER
const sectionStart = `<!-- section:${sectionName}:start -->`
const sectionEnd = `<!-- section:${sectionName}:end -->`
const sectionBlock = `${sectionStart}\n${sectionContent}\n${sectionEnd}`
// Escape special regex characters in delimiter strings
const escapeRegex = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
const comments = await github.paginate(
github.rest.issues.listComments,
{ ...context.repo, issue_number: prNumber }
)
const existing = comments.find(
(c) =>
c.user?.login === 'github-actions[bot]' &&
c.body?.includes(commentMarker)
)
if (!existing) {
return github.rest.issues.createComment({
...context.repo,
issue_number: prNumber,
body: `${commentMarker}\n${sectionBlock}`
})
}
const body = existing.body ?? ''
const sectionRegex = new RegExp(
`${escapeRegex(sectionStart)}[\\s\\S]*?${escapeRegex(sectionEnd)}`
)
const updated = sectionRegex.test(body)
? body.replace(sectionRegex, sectionBlock)
: body.trimEnd() + '\n\n' + sectionBlock
return github.rest.issues.updateComment({
...context.repo,
comment_id: existing.id,
body: updated
})

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

View File

@@ -0,0 +1,107 @@
# Description: When upstream comfy-api is updated, click dispatch to update the TypeScript type definitions in this repo
name: 'Api: Update Registry API Types'
on:
# Manual trigger
workflow_dispatch:
# Triggered from comfy-api repo
repository_dispatch:
types: [comfy-api-updated]
jobs:
update-registry-types:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Checkout comfy-api repository
uses: actions/checkout@v6
with:
repository: Comfy-Org/comfy-api
path: comfy-api
token: ${{ secrets.COMFY_API_PAT }}
clean: true
- name: Get API commit information
id: api-info
run: |
cd comfy-api
API_COMMIT=$(git rev-parse --short HEAD)
echo "commit=${API_COMMIT}" >> $GITHUB_OUTPUT
cd ..
- name: Generate API types
run: |
echo "Generating TypeScript types from comfy-api@${{ steps.api-info.outputs.commit }}..."
mkdir -p ./packages/registry-types/src
pnpm dlx openapi-typescript ./comfy-api/openapi.yml --output ./packages/registry-types/src/comfyRegistryTypes.ts
- name: Validate generated types
run: |
if [ ! -f ./packages/registry-types/src/comfyRegistryTypes.ts ]; then
echo "Error: Types file was not generated."
exit 1
fi
# Check if file is not empty
if [ ! -s ./packages/registry-types/src/comfyRegistryTypes.ts ]; then
echo "Error: Generated types file is empty."
exit 1
fi
- name: Lint generated types
run: |
echo "Linting generated Comfy Registry API types..."
pnpm lint:fix:no-cache -- ./packages/registry-types/src/comfyRegistryTypes.ts
- name: Check for changes
id: check-changes
run: |
if [[ -z $(git status --porcelain ./packages/registry-types/src/comfyRegistryTypes.ts) ]]; then
echo "No changes to Comfy Registry API types detected."
echo "changed=false" >> $GITHUB_OUTPUT
exit 0
else
echo "Changes detected in Comfy Registry API types."
echo "changed=true" >> $GITHUB_OUTPUT
fi
- name: Create Pull Request
if: steps.check-changes.outputs.changed == 'true'
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0 # v8.1.0
with:
token: ${{ secrets.PR_GH_TOKEN }}
commit-message: '[chore] Update Comfy Registry API types from comfy-api@${{ steps.api-info.outputs.commit }}'
title: '[chore] Update Comfy Registry API types from comfy-api@${{ steps.api-info.outputs.commit }}'
body: |
## Automated API Type Update
This PR updates the Comfy Registry API types from the latest comfy-api OpenAPI specification.
- API commit: ${{ steps.api-info.outputs.commit }}
- Generated on: ${{ github.event.repository.updated_at }}
These types are automatically generated using openapi-typescript.
branch: update-registry-types-${{ steps.api-info.outputs.commit }}
base: main
labels: CNR
delete-branch: true
add-paths: |
packages/registry-types/src/comfyRegistryTypes.ts

View File

@@ -12,30 +12,17 @@ permissions:
contents: read
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
scan:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'

View File

@@ -1,29 +0,0 @@
# Description: Lint and format verification for GitHub merge queue runs.
# Paired with ci-lint-format.yaml — workflow name and job name must match
# so branch protection resolves a single required check in both the
# pull_request and merge_group contexts. This file runs verify-only steps
# with a read-only token; auto-fix and PR comments live in the PR workflow.
name: 'CI: Lint Format'
on:
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
lint-and-format:
runs-on: ubuntu-latest
steps:
- name: Checkout merge group ref
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Verify lint and format
uses: ./.github/actions/lint-format-verify

View File

@@ -1,7 +1,4 @@
# Description: Linting and code formatting validation for pull requests.
# Paired with ci-lint-format-queue.yaml - workflow name and job name must
# match so branch protection resolves a single required check in both the
# pull_request and merge_group contexts.
# Description: Linting and code formatting validation for pull requests
name: 'CI: Lint Format'
on:
@@ -29,6 +26,14 @@ jobs:
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Detect browser_tests changes
id: changed-paths
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
browser_tests:
- 'browser_tests/**'
- name: Run ESLint with auto-fix
run: pnpm lint:fix
@@ -72,8 +77,16 @@ jobs:
echo "See CONTRIBUTING.md for more details."
exit 1
- name: Verify lint and format
uses: ./.github/actions/lint-format-verify
- name: Final validation
run: |
pnpm lint
pnpm stylelint
pnpm format:check
pnpm knip
- name: Typecheck browser tests
if: steps.changed-paths.outputs.browser_tests == 'true'
run: pnpm typecheck:browser
- name: Comment on PR about auto-fix
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name == github.repository

View File

@@ -14,29 +14,16 @@ permissions:
contents: read
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
validate-fonts:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
@@ -81,17 +68,15 @@ jobs:
echo '✅ No proprietary fonts found in dist'
validate-licenses:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'

View File

@@ -3,8 +3,10 @@ name: 'CI: Performance Report'
on:
push:
branches: [main, core/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
concurrency:
group: perf-${{ github.ref }}
@@ -14,24 +16,12 @@ permissions:
contents: read
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
perf-tests:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' && github.repository == 'Comfy-Org/ComfyUI_frontend' }}
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
timeout-minutes: 30
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.17
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.12
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
@@ -54,14 +44,10 @@ jobs:
- name: Start ComfyUI server
uses: ./.github/actions/start-comfyui-server
# PRs run each test once to keep wall time bounded; main runs 3× so the
# baseline saved to perf-data has enough samples to median over noise.
- name: Run performance tests
id: perf
continue-on-error: true
env:
PERF_REPEAT: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && '3' || '2' }}
run: pnpm exec playwright test --project=performance --workers=1 --repeat-each=$PERF_REPEAT
run: pnpm exec playwright test --project=performance --workers=1 --repeat-each=3
- name: Upload perf metrics
if: always()
@@ -72,6 +58,21 @@ jobs:
retention-days: 30
if-no-files-found: warn
- name: Save PR metadata
if: github.event_name == 'pull_request'
run: |
mkdir -p temp/perf-meta
echo "${{ github.event.number }}" > temp/perf-meta/number.txt
echo "${{ github.event.pull_request.base.ref }}" > temp/perf-meta/base.txt
echo "${{ github.event.pull_request.head.sha }}" > temp/perf-meta/head-sha.txt
- name: Upload PR metadata
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v6
with:
name: perf-meta
path: temp/perf-meta/
- name: Save perf baseline to perf-data branch
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.perf.outcome == 'success'
continue-on-error: true

View File

@@ -16,21 +16,8 @@ permissions:
contents: read
jobs:
changes:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
collect:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
steps:
@@ -45,6 +32,13 @@ jobs:
- name: Collect size data
run: node scripts/size-collect.js
- name: Save PR metadata
if: ${{ github.event_name == 'pull_request' }}
run: |
echo ${{ github.event.number }} > ./temp/size/number.txt
echo ${{ github.base_ref }} > ./temp/size/base.txt
echo ${{ github.event.pull_request.head.sha }} > ./temp/size/head-sha.txt
- name: Upload size data
uses: actions/upload-artifact@v6
with:

View File

@@ -1,158 +0,0 @@
name: 'CI: E2E Coverage'
on:
workflow_run:
workflows: ['CI: Tests E2E']
types:
- completed
concurrency:
group: e2e-coverage-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: true
permissions:
contents: read
jobs:
merge:
if: >
github.repository == 'Comfy-Org/ComfyUI_frontend' &&
github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
has-coverage: ${{ steps.coverage-shards.outputs.has-coverage }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Download all shard coverage data
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
run_id: ${{ github.event.workflow_run.id }}
name: e2e-coverage-shard-.*
name_is_regexp: true
path: temp/coverage-shards
if_no_artifact_found: warn
- name: Detect shard coverage data
id: coverage-shards
run: |
if [ -d temp/coverage-shards ] && find temp/coverage-shards -name 'coverage.lcov' -type f | grep -q .; then
echo "has-coverage=true" >> "$GITHUB_OUTPUT"
else
echo "has-coverage=false" >> "$GITHUB_OUTPUT"
echo "No E2E coverage shard artifacts found; treating this run as skipped." >> "$GITHUB_STEP_SUMMARY"
fi
- name: Install lcov
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: sudo apt-get install -y -qq lcov
- name: Merge shard coverage into single LCOV
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
mkdir -p coverage/playwright
LCOV_FILES=$(find temp/coverage-shards -name 'coverage.lcov' -type f)
ADD_ARGS=""
for f in $LCOV_FILES; do ADD_ARGS="$ADD_ARGS -a $f"; done
lcov $ADD_ARGS -o coverage/playwright/coverage.lcov
wc -l coverage/playwright/coverage.lcov
- name: Validate merged coverage
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
MERGED_SF=$(grep -c '^SF:' coverage/playwright/coverage.lcov || echo 0)
MERGED_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
MERGED_LF=$(awk -F: '/^LF:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
echo "### Merged coverage" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_SF** source files" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_LH / $MERGED_LF** lines hit" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Shard | Files | Lines Hit |" >> "$GITHUB_STEP_SUMMARY"
echo "|-------|-------|-----------|" >> "$GITHUB_STEP_SUMMARY"
for f in $(find temp/coverage-shards -name 'coverage.lcov' -type f | sort); do
SHARD=$(basename "$(dirname "$f")")
SHARD_SF=$(grep -c '^SF:' "$f" || echo 0)
SHARD_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' "$f")
echo "| $SHARD | $SHARD_SF | $SHARD_LH |" >> "$GITHUB_STEP_SUMMARY"
if [ "$MERGED_LH" -lt "$SHARD_LH" ]; then
echo "::error::Merged LH ($MERGED_LH) < shard LH ($SHARD_LH) in $SHARD — possible data loss"
fi
done
- name: Upload merged coverage data
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: actions/upload-artifact@v6
with:
name: e2e-coverage
path: coverage/playwright/
retention-days: 30
if-no-files-found: warn
- name: Upload E2E coverage to Codecov
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
with:
files: coverage/playwright/coverage.lcov
flags: e2e
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
- name: Generate HTML coverage report
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
if [ ! -s coverage/playwright/coverage.lcov ]; then
echo "No coverage data; generating placeholder report."
mkdir -p coverage/html
WORKFLOW_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}"
echo "<html><body><h1>No E2E coverage data available for this run.</h1><p><a href=\"${WORKFLOW_URL}\">View workflow run</a></p></body></html>" > coverage/html/index.html
exit 0
fi
genhtml coverage/playwright/coverage.lcov \
-o coverage/html \
--title "ComfyUI E2E Coverage" \
--no-function-coverage \
--precision 1 \
--ignore-errors source
- name: Upload HTML report artifact
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: actions/upload-artifact@v6
with:
name: e2e-coverage-html
path: coverage/html/
retention-days: 30
deploy:
needs: merge
if: >
github.event.workflow_run.head_branch == 'main' &&
needs.merge.outputs.has-coverage == 'true' &&
github.event.workflow_run.event == 'push'
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Download HTML report
uses: actions/download-artifact@v7
with:
name: e2e-coverage-html
path: coverage/html
- name: Upload to GitHub Pages
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3.0.1
with:
path: coverage/html
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5

View File

@@ -6,10 +6,6 @@ on:
workflows: ['CI: Tests E2E']
types: [requested, completed]
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_repository.full_name }}-${{ github.event.workflow_run.head_branch }}
cancel-in-progress: true
jobs:
deploy-and-comment-forked-pr:
runs-on: ubuntu-latest
@@ -34,33 +30,49 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Resolve PR from workflow_run context
- name: Get PR Number
id: pr
uses: ./.github/actions/resolve-pr-from-workflow-run
uses: actions/github-script@v8
with:
script: |
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
});
const pr = prs.find(p => p.head.sha === context.payload.workflow_run.head_sha);
if (!pr) {
console.log('No PR found for SHA:', context.payload.workflow_run.head_sha);
return null;
}
console.log(`Found PR #${pr.number} from fork: ${context.payload.workflow_run.head_repository.full_name}`);
return pr.number;
- name: Handle Test Start
if: steps.pr.outputs.skip != 'true' && github.event.action == 'requested'
if: steps.pr.outputs.result != 'null' && github.event.action == 'requested'
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
chmod +x scripts/cicd/pr-playwright-deploy-and-comment.sh
./scripts/cicd/pr-playwright-deploy-and-comment.sh \
"${{ steps.pr.outputs.number }}" \
"${{ steps.pr.outputs.result }}" \
"${{ github.event.workflow_run.head_branch }}" \
"starting"
- name: Download and Deploy Reports
if: steps.pr.outputs.skip != 'true' && github.event.action == 'completed'
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
if: steps.pr.outputs.result != 'null' && github.event.action == 'completed'
uses: actions/download-artifact@v7
with:
run_id: ${{ github.event.workflow_run.id }}
name: playwright-report-.*
name_is_regexp: true
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ github.event.workflow_run.id }}
pattern: playwright-report-*
path: reports
if_no_artifact_found: warn
- name: Handle Test Completion
if: steps.pr.outputs.skip != 'true' && github.event.action == 'completed' && hashFiles('reports/**') != ''
if: steps.pr.outputs.result != 'null' && github.event.action == 'completed'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
@@ -73,6 +85,6 @@ jobs:
chmod +x scripts/cicd/pr-playwright-deploy-and-comment.sh
./scripts/cicd/pr-playwright-deploy-and-comment.sh \
"${{ steps.pr.outputs.number }}" \
"${{ steps.pr.outputs.result }}" \
"${{ github.event.workflow_run.head_branch }}" \
"completed"

View File

@@ -4,9 +4,10 @@ name: 'CI: Tests E2E'
on:
push:
branches: [main, master, core/*, desktop/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
merge_group:
paths-ignore: ['**/*.md']
workflow_dispatch:
concurrency:
@@ -14,20 +15,7 @@ concurrency:
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
setup:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -65,7 +53,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 60
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.17
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.16
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
@@ -98,7 +86,6 @@ jobs:
run: pnpm exec playwright test --project=chromium --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --reporter=blob
env:
PLAYWRIGHT_BLOB_OUTPUT_DIR: ./blob-report
COLLECT_COVERAGE: 'true'
- name: Upload blob report
uses: actions/upload-artifact@v6
@@ -108,22 +95,13 @@ jobs:
path: blob-report/
retention-days: 1
- name: Upload shard coverage data
if: always()
uses: actions/upload-artifact@v6
with:
name: e2e-coverage-shard-${{ matrix.shardIndex }}
path: coverage/playwright/
retention-days: 1
if-no-files-found: warn
playwright-tests:
# Ideally, each shard runs test in 6 minutes, but allow up to 15 minutes
timeout-minutes: 15
needs: setup
runs-on: ubuntu-latest
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.17
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.16
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
@@ -175,9 +153,9 @@ jobs:
# Merge sharded test reports (no container needed - only runs CLI)
merge-reports:
needs: [changes, playwright-tests-chromium-sharded]
needs: [playwright-tests-chromium-sharded]
runs-on: ubuntu-latest
if: ${{ !cancelled() && needs.changes.outputs.should-run == 'true' }}
if: ${{ !cancelled() }}
steps:
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
@@ -206,38 +184,14 @@ jobs:
path: ./playwright-report/
retention-days: 30
# Gate job — single required check that passes whether the matrix ran or was
# skipped. Branch rulesets require this instead of the individual matrix-
# expanded check names so PRs with no e2e-relevant changes aren't stuck.
e2e-status:
if: ${{ always() }}
needs: [changes, playwright-tests-chromium-sharded, playwright-tests]
runs-on: ubuntu-latest
steps:
- name: Check E2E results
env:
SHOULD_RUN: ${{ needs.changes.outputs.should-run }}
SHARDED: ${{ needs.playwright-tests-chromium-sharded.result }}
BROWSERS: ${{ needs.playwright-tests.result }}
run: |
[[ "$SHOULD_RUN" != "true" ]] && echo "E2E skipped" && exit 0
[[ "$SHARDED" != "success" || "$BROWSERS" != "success" ]] && echo "E2E failed" && exit 1
echo "E2E passed"
#### BEGIN Deployment and commenting (non-forked PRs only)
# when using pull_request event, we have permission to comment directly
# if its a forked repo, we need to use workflow_run event in a separate workflow (pr-playwright-deploy.yaml)
# Post starting comment for non-forked PRs
comment-on-pr-start:
needs: changes
runs-on: ubuntu-latest
if: >-
${{
needs.changes.outputs.should-run == 'true' &&
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.fork == false
}}
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
permissions:
pull-requests: write
steps:
@@ -256,15 +210,9 @@ jobs:
# Deploy and comment for non-forked PRs only
deploy-and-comment:
needs: [changes, playwright-tests, merge-reports]
needs: [playwright-tests, merge-reports]
runs-on: ubuntu-latest
if: >-
${{
always() &&
needs.changes.outputs.should-run == 'true' &&
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.fork == false
}}
if: always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
permissions:
pull-requests: write
contents: read

View File

@@ -6,10 +6,6 @@ on:
workflows: ['CI: Tests Storybook']
types: [requested, completed]
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_repository.full_name }}-${{ github.event.workflow_run.head_branch }}
cancel-in-progress: true
jobs:
deploy-and-comment-forked-pr:
runs-on: ubuntu-latest
@@ -34,23 +30,40 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Resolve PR from workflow_run context
- name: Get PR Number
id: pr
uses: ./.github/actions/resolve-pr-from-workflow-run
uses: actions/github-script@v8
with:
script: |
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
});
const pr = prs.find(p => p.head.sha === context.payload.workflow_run.head_sha);
if (!pr) {
console.log('No PR found for SHA:', context.payload.workflow_run.head_sha);
return null;
}
console.log(`Found PR #${pr.number} from fork: ${context.payload.workflow_run.head_repository.full_name}`);
return pr.number;
- name: Handle Storybook Start
if: steps.pr.outputs.skip != 'true' && github.event.action == 'requested'
if: steps.pr.outputs.result != 'null' && github.event.action == 'requested'
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
chmod +x scripts/cicd/pr-storybook-deploy-and-comment.sh
./scripts/cicd/pr-storybook-deploy-and-comment.sh \
"${{ steps.pr.outputs.number }}" \
"${{ steps.pr.outputs.result }}" \
"${{ github.event.workflow_run.head_branch }}" \
"starting"
- name: Download and Deploy Storybook
if: steps.pr.outputs.skip != 'true' && github.event.action == 'completed' && github.event.workflow_run.conclusion == 'success'
if: steps.pr.outputs.result != 'null' && github.event.action == 'completed' && github.event.workflow_run.conclusion == 'success'
uses: actions/download-artifact@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
@@ -59,7 +72,7 @@ jobs:
path: storybook-static
- name: Handle Storybook Completion
if: steps.pr.outputs.skip != 'true' && github.event.action == 'completed'
if: steps.pr.outputs.result != 'null' && github.event.action == 'completed'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
@@ -69,6 +82,6 @@ jobs:
run: |
chmod +x scripts/cicd/pr-storybook-deploy-and-comment.sh
./scripts/cicd/pr-storybook-deploy-and-comment.sh \
"${{ steps.pr.outputs.number }}" \
"${{ steps.pr.outputs.result }}" \
"${{ github.event.workflow_run.head_branch }}" \
"completed"

View File

@@ -8,29 +8,10 @@ on:
branches: [main]
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
storybook-changes: ${{ steps.changes.outputs.storybook-changes }}
app-frontend-changes: ${{ steps.changes.outputs.app-frontend-changes }}
packages-changes: ${{ steps.changes.outputs.packages-changes }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
# Post starting comment for non-forked PRs
comment-on-pr-start:
needs: changes
runs-on: ubuntu-latest
if: |
github.event_name == 'pull_request'
&& github.event.pull_request.head.repo.fork == false
&& (needs.changes.outputs.storybook-changes == 'true'
|| needs.changes.outputs.app-frontend-changes == 'true'
|| needs.changes.outputs.packages-changes == 'true')
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
permissions:
pull-requests: write
steps:
@@ -49,13 +30,8 @@ jobs:
# Build Storybook for all PRs (free Cloudflare deployment)
storybook-build:
needs: changes
runs-on: ubuntu-latest
if: |
github.event_name == 'pull_request'
&& (needs.changes.outputs.storybook-changes == 'true'
|| needs.changes.outputs.app-frontend-changes == 'true'
|| needs.changes.outputs.packages-changes == 'true')
if: github.event_name == 'pull_request'
outputs:
conclusion: ${{ steps.job-status.outputs.conclusion }}
workflow-url: ${{ steps.workflow-url.outputs.url }}
@@ -91,15 +67,8 @@ jobs:
# Chromatic deployment only for version-bump-* branches or manual triggers
chromatic-deployment:
needs: changes
runs-on: ubuntu-latest
if: |
github.event_name == 'workflow_dispatch'
|| (github.event_name == 'pull_request'
&& startsWith(github.head_ref, 'version-bump-')
&& (needs.changes.outputs.storybook-changes == 'true'
|| needs.changes.outputs.app-frontend-changes == 'true'
|| needs.changes.outputs.packages-changes == 'true'))
if: github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && startsWith(github.head_ref, 'version-bump-'))
outputs:
conclusion: ${{ steps.job-status.outputs.conclusion }}
workflow-url: ${{ steps.workflow-url.outputs.url }}
@@ -138,15 +107,9 @@ jobs:
# Deploy and comment for non-forked PRs only
deploy-and-comment:
needs: [changes, storybook-build]
needs: [storybook-build]
runs-on: ubuntu-latest
if: |
always()
&& github.event_name == 'pull_request'
&& github.event.pull_request.head.repo.fork == false
&& (needs.changes.outputs.storybook-changes == 'true'
|| needs.changes.outputs.app-frontend-changes == 'true'
|| needs.changes.outputs.packages-changes == 'true')
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false && always()
permissions:
pull-requests: write
contents: read

View File

@@ -4,29 +4,17 @@ name: 'CI: Tests Unit'
on:
push:
branches: [main, master, dev*, core/*, desktop/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
merge_group:
paths-ignore: ['**/*.md']
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
test:
needs: changes
if: ${{ needs.changes.outputs.should-run == 'true' }}
runs-on: ubuntu-latest
steps:
@@ -38,20 +26,9 @@ jobs:
- name: Run Vitest tests with coverage
run: pnpm test:coverage
- name: Upload unit coverage artifact
if: always() && github.event_name == 'push'
uses: actions/upload-artifact@v6
with:
name: unit-coverage
path: coverage/lcov.info
retention-days: 30
if-no-files-found: warn
- name: Upload coverage to Codecov
if: always()
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
with:
files: coverage/lcov.info
flags: unit
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false

View File

@@ -1,167 +0,0 @@
---
name: 'CI: Vercel Website Preview'
on:
pull_request:
types: [opened, synchronize, reopened]
branches-ignore:
- 'core/**'
- 'cloud/**'
paths:
- 'apps/website/**'
- 'packages/design-system/**'
- 'packages/tailwind-utils/**'
push:
branches: [main]
paths:
- 'apps/website/**'
- 'packages/design-system/**'
- 'packages/tailwind-utils/**'
env:
VERCEL_ORG_ID: ${{ secrets.VERCEL_WEBSITE_ORG_ID }}
VERCEL_PROJECT_ID: ${{ secrets.VERCEL_WEBSITE_PROJECT_ID }}
VERCEL_TOKEN: ${{ secrets.VERCEL_WEBSITE_TOKEN }}
VERCEL_SCOPE: comfyui
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
deploy-preview:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
permissions:
contents: read
env:
ALIAS_HOST: comfy-website-preview-pr-${{ github.event.pull_request.number }}.vercel.app
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
- name: Install Vercel CLI
run: npm install --global vercel@latest
- name: Pull Vercel environment information
run: vercel pull --yes --environment=preview
- name: Build project artifacts
env:
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ secrets.WEBSITE_ASHBY_JOB_BOARD_NAME }}
run: vercel build
- name: Fetch head commit metadata
id: head-commit
uses: actions/github-script@v8
with:
script: |
const { data } = await github.rest.repos.getCommit({
owner: context.repo.owner,
repo: context.repo.repo,
ref: context.payload.pull_request.head.sha,
})
const author = data.author?.login || data.commit.author?.name || ''
const message = (data.commit.message || '').split('\n', 1)[0]
core.setOutput('author', author)
core.setOutput('message', message)
- name: Deploy project artifacts to Vercel
id: deploy
env:
GIT_COMMIT_REF: ${{ github.event.pull_request.head.ref }}
GIT_COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
GIT_AUTHOR_LOGIN: ${{ steps.head-commit.outputs.author }}
GIT_COMMIT_MESSAGE: ${{ steps.head-commit.outputs.message }}
GIT_PR_ID: ${{ github.event.pull_request.number }}
GIT_REPO: ${{ github.repository }}
run: |
URL=$(vercel deploy --prebuilt \
--meta githubCommitRef="$GIT_COMMIT_REF" \
--meta githubCommitSha="$GIT_COMMIT_SHA" \
--meta githubCommitAuthorLogin="$GIT_AUTHOR_LOGIN" \
--meta githubCommitMessage="$GIT_COMMIT_MESSAGE" \
--meta githubPrId="$GIT_PR_ID" \
--meta githubRepo="$GIT_REPO")
echo "url=$URL" >> "$GITHUB_OUTPUT"
- name: Alias deployment to stable PR hostname
id: alias-set
continue-on-error: true
env:
DEPLOY_URL: ${{ steps.deploy.outputs.url }}
run: |
vercel alias set "$DEPLOY_URL" "$ALIAS_HOST" --scope="$VERCEL_SCOPE"
- name: Publish preview outputs
env:
DEPLOY_URL: ${{ steps.deploy.outputs.url }}
ALIAS_OK: ${{ steps.alias-set.outcome == 'success' }}
run: |
if [[ "$ALIAS_OK" == "true" ]]; then
STABLE_URL="https://$ALIAS_HOST"
else
STABLE_URL="$DEPLOY_URL"
fi
mkdir -p temp/vercel-preview
echo "$DEPLOY_URL" > temp/vercel-preview/url.txt
echo "$STABLE_URL" > temp/vercel-preview/stable-url.txt
{
echo "**Preview:** $STABLE_URL"
if [[ "$ALIAS_OK" == "true" ]]; then
echo "**This commit:** $DEPLOY_URL"
else
echo "_Stable alias update failed — URL reflects this commit only._"
fi
} >> "$GITHUB_STEP_SUMMARY"
- name: Upload preview metadata
uses: actions/upload-artifact@v6
with:
name: vercel-preview
path: temp/vercel-preview
deploy-production:
if: github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
- name: Install Vercel CLI
run: npm install --global vercel@latest
- name: Pull Vercel environment information
run: vercel pull --yes --environment=production
- name: Build project artifacts
env:
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ secrets.WEBSITE_ASHBY_JOB_BOARD_NAME }}
run: vercel build --prod
- name: Deploy project artifacts to Vercel
id: deploy
run: |
URL=$(vercel deploy --prebuilt --prod)
echo "url=$URL" >> "$GITHUB_OUTPUT"
- name: Add deployment URL to summary
run: echo "**Production:** ${{ steps.deploy.outputs.url }}" >> "$GITHUB_STEP_SUMMARY"

View File

@@ -1,42 +0,0 @@
# Description: Build and validate the marketing website (apps/website)
name: 'CI: Website Build'
on:
push:
branches: [main, master, website/*]
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
app-website-changes: ${{ steps.changes.outputs.app-website-changes }}
packages-changes: ${{ steps.changes.outputs.packages-changes }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
build:
needs: changes
if: ${{ needs.changes.outputs.app-website-changes == 'true' || needs.changes.outputs.packages-changes == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Build website
env:
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ secrets.WEBSITE_ASHBY_JOB_BOARD_NAME }}
run: pnpm --filter @comfyorg/website build

View File

@@ -1,270 +0,0 @@
name: 'CI: Website E2E'
on:
push:
branches: [main]
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
concurrency:
group: ${{ github.workflow }}-${{ github.repository }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
app-website-changes: ${{ steps.changes.outputs.app-website-changes }}
packages-changes: ${{ steps.changes.outputs.packages-changes }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
website-e2e:
needs: changes
if: ${{ needs.changes.outputs.app-website-changes == 'true' || needs.changes.outputs.packages-changes == 'true' }}
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.58.1-noble
timeout-minutes: 15
permissions:
contents: read
outputs:
test-outcome: ${{ steps.tests.outcome }}
report-url: ${{ steps.deploy.outputs.url }}
screenshot-failures: ${{ steps.failures.outputs.screenshot }}
other-failures: ${{ steps.failures.outputs.other }}
# Evaluated at job level (not from a step) — static expression.
is-pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false }}
steps:
- uses: actions/checkout@v6
- name: Install pnpm
run: corepack enable && corepack prepare
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build website
env:
WEBSITE_GITHUB_STARS_OVERRIDE: 110000
run: pnpm --filter @comfyorg/website build
- name: Run Playwright tests
id: tests
run: pnpm --filter @comfyorg/website test:e2e
- name: Upload test report
uses: actions/upload-artifact@v6
if: ${{ !cancelled() }}
with:
name: website-playwright-report
path: apps/website/playwright-report/
retention-days: 30
- name: Deploy report to Cloudflare
id: deploy
if: always() && !cancelled()
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
HEAD_REF: ${{ github.head_ref || github.ref_name }}
run: |
BRANCH=$(echo "$HEAD_REF" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9-]/-/g;s/--*/-/g;s/^-\|-$//g')
DEPLOY_OK=false
for i in 1 2 3; do
echo "Deployment attempt $i of 3..."
OUTPUT=$(npx wrangler@^4.0.0 pages deploy apps/website/playwright-report \
--project-name=comfyui-website-e2e \
--branch="$BRANCH" 2>&1) && { DEPLOY_OK=true; break; } || echo "$OUTPUT"
[ $i -lt 3 ] && sleep 10
done
echo "$OUTPUT"
if [ "$DEPLOY_OK" != "true" ]; then
echo "::error::All 3 deployment attempts failed"
exit 1
fi
URL=$(echo "$OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1)
echo "url=${URL}" >> $GITHUB_OUTPUT
- name: Categorize failures
id: failures
if: always() && !cancelled() && steps.tests.outcome != 'success'
uses: actions/github-script@v8
with:
script: |
const fs = require('fs')
const report = JSON.parse(fs.readFileSync('apps/website/results.json', 'utf8'))
function isFailed(t) { return t.status === 'unexpected' || t.status === 'flaky' }
function isVisual(spec) {
return spec.file?.includes('visual') ||
spec.tests?.some(t => t.results?.some(r => r.error?.message?.includes('toHaveScreenshot')))
}
function specsOf(suite) {
return [
...(suite.specs || []),
...(suite.suites || []).flatMap(specsOf)
]
}
// True: Visual
// False: Other
const failed = specsOf(report)
.flatMap(spec => (spec.tests || [])
.filter(isFailed)
.map(() => isVisual(spec)))
const screenshotFailures = failed.filter(Boolean).length
core.setOutput('screenshot', screenshotFailures)
core.setOutput('other', failed.length - screenshotFailures)
- name: Write job summary
if: always() && !cancelled()
uses: actions/github-script@v8
env:
TEST_OUTCOME: ${{ steps.tests.outcome }}
REPORT_URL: ${{ steps.deploy.outputs.url }}
SCREENSHOT_FAILURES: ${{ steps.failures.outputs.screenshot }}
OTHER_FAILURES: ${{ steps.failures.outputs.other }}
with:
script: |
const passed = process.env.TEST_OUTCOME === 'success'
const reportUrl = process.env.REPORT_URL
const screenshotFailures = parseInt(process.env.SCREENSHOT_FAILURES) || 0
const otherFailures = parseInt(process.env.OTHER_FAILURES) || 0
const lines = ['## 🌐 Website E2E', '']
if (passed) {
lines.push('> [!TIP]', '> All tests passed.')
} else {
lines.push('> [!CAUTION]', '> Some tests failed.')
}
const rows = [
['Status', passed ? '✅ Passed' : '❌ Failed'],
['Report', reportUrl ? `[View Report](${reportUrl})` : '_unavailable_']
]
if (!passed) {
rows.push(
['Screenshot diffs', String(screenshotFailures)],
['Other failures', String(otherFailures)]
)
}
lines.push(
'',
'| | |',
'|---|---|',
...rows.map(([k, v]) => `| **${k}** | ${v} |`)
)
await core.summary.addRaw(lines.join('\n')).write()
post-starting-comment:
# Safe to comment from pull_request trigger: fork PRs are excluded by the guard below.
# This avoids a ci-*/pr-* workflow_run split for a comment that must appear immediately.
needs: changes
if: |
github.event_name == 'pull_request'
&& github.event.pull_request.head.repo.fork == false
&& (needs.changes.outputs.app-website-changes == 'true' || needs.changes.outputs.packages-changes == 'true')
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
concurrency:
group: website-pr-comment-${{ github.event.pull_request.number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/upsert-comment-section
with:
pr-number: ${{ github.event.pull_request.number }}
section-name: e2e
comment-marker: '<!-- WEBSITE_CI_REPORT -->'
token: ${{ secrets.GITHUB_TOKEN }}
section-content: |-
## 🌐 Website E2E
<!-- WEBSITE_E2E_STATUS -->
> [!NOTE]
> Tests are running… [View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
post-result-comment:
needs: website-e2e
if: always() && !cancelled() && needs.website-e2e.outputs.is-pr == 'true'
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
concurrency:
group: website-pr-comment-${{ github.event.pull_request.number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v6
- name: Build e2e section content
id: content
uses: actions/github-script@v8
env:
TEST_OUTCOME: ${{ needs.website-e2e.outputs.test-outcome }}
REPORT_URL: ${{ needs.website-e2e.outputs.report-url }}
SCREENSHOT_FAILURES: ${{ needs.website-e2e.outputs.screenshot-failures }}
OTHER_FAILURES: ${{ needs.website-e2e.outputs.other-failures }}
with:
script: |
const passed = process.env.TEST_OUTCOME === 'success'
const reportUrl = process.env.REPORT_URL
const screenshotFailures = parseInt(process.env.SCREENSHOT_FAILURES) || 0
const otherFailures = parseInt(process.env.OTHER_FAILURES) || 0
const lines = ['## 🌐 Website E2E', '<!-- WEBSITE_E2E_STATUS -->', '']
if (passed) {
lines.push('> [!TIP]', '> All tests passed.')
} else {
lines.push('> [!CAUTION]', '> Some tests failed.')
}
const rows = [
['Status', passed ? '✅ Passed' : '❌ Failed'],
['Report', reportUrl ? `[View Report](${reportUrl})` : '_unavailable_']
]
if (!passed) {
rows.push(
['Screenshot diffs', String(screenshotFailures)],
['Other failures', String(otherFailures)]
)
}
lines.push(
'',
'| | |',
'|---|---|',
...rows.map(([k, v]) => `| **${k}** | ${v} |`)
)
if (screenshotFailures > 0) {
const s = screenshotFailures === 1 ? '' : 's'
lines.push('', `- [ ] Update website screenshots (${screenshotFailures} screenshot diff${s})`)
}
if (otherFailures > 0) {
lines.push(
'',
'> [!WARNING]',
`> ${otherFailures} non-screenshot failure${otherFailures === 1 ? '' : 's'} — these require manual review.`
)
}
core.setOutput('section-content', lines.join('\n'))
- uses: ./.github/actions/upsert-comment-section
with:
pr-number: ${{ github.event.pull_request.number }}
section-name: e2e
comment-marker: '<!-- WEBSITE_CI_REPORT -->'
token: ${{ secrets.GITHUB_TOKEN }}
section-content: ${{ steps.content.outputs.section-content }}

View File

@@ -1,149 +0,0 @@
name: 'Coverage: Slack Notification'
on:
workflow_run:
workflows: ['CI: Tests Unit']
branches: [main]
types:
- completed
permissions:
contents: read
actions: read
pull-requests: read
jobs:
notify:
if: >
github.repository == 'Comfy-Org/ComfyUI_frontend' &&
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Download current unit coverage
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
run_id: ${{ github.event.workflow_run.id }}
name: unit-coverage
path: coverage
- name: Download previous unit coverage baseline
continue-on-error: true
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
branch: main
workflow: coverage-slack-notify.yaml
name: unit-coverage-baseline
path: temp/coverage-baseline
if_no_artifact_found: warn
- name: Download latest E2E coverage
continue-on-error: true
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
branch: main
workflow: ci-tests-e2e-coverage.yaml
name: e2e-coverage
path: temp/e2e-coverage
if_no_artifact_found: warn
- name: Download previous E2E coverage baseline
continue-on-error: true
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
branch: main
workflow: coverage-slack-notify.yaml
name: e2e-coverage-baseline
path: temp/e2e-coverage-baseline
if_no_artifact_found: warn
- name: Resolve merged PR metadata
id: pr-meta
uses: actions/github-script@v8
with:
script: |
const sha = context.payload.workflow_run.head_sha;
const { data: commit } = await github.rest.repos.getCommit({
owner: context.repo.owner,
repo: context.repo.repo,
ref: sha,
});
const message = commit.commit.message ?? '';
const firstLine = message.split('\n')[0];
const match = firstLine.match(/\(#(\d+)\)\s*$/);
if (!match) {
core.setOutput('skip', 'true');
core.info('No PR number found in commit message — skipping.');
return;
}
const prNumber = match[1];
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: Number(prNumber),
});
core.setOutput('skip', 'false');
core.setOutput('number', prNumber);
core.setOutput('url', pr.html_url);
core.setOutput('author', pr.user.login);
- name: Generate Slack notification
if: steps.pr-meta.outputs.skip != 'true'
id: slack-payload
env:
PR_URL: ${{ steps.pr-meta.outputs.url }}
PR_NUMBER: ${{ steps.pr-meta.outputs.number }}
PR_AUTHOR: ${{ steps.pr-meta.outputs.author }}
run: |
PAYLOAD=$(pnpm exec tsx scripts/coverage-slack-notify.ts \
--pr-url="$PR_URL" \
--pr-number="$PR_NUMBER" \
--author="$PR_AUTHOR")
if [ -n "$PAYLOAD" ]; then
echo "has_payload=true" >> "$GITHUB_OUTPUT"
DELIM="SLACK_PAYLOAD_$(date +%s)"
echo "payload<<$DELIM" >> "$GITHUB_OUTPUT"
printf '%s\n' "$PAYLOAD" >> "$GITHUB_OUTPUT"
echo "$DELIM" >> "$GITHUB_OUTPUT"
else
echo "has_payload=false" >> "$GITHUB_OUTPUT"
fi
- name: Post to Slack
if: steps.slack-payload.outputs.has_payload == 'true'
continue-on-error: true
env:
SLACK_PAYLOAD: ${{ steps.slack-payload.outputs.payload }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
run: |
# Channel: #p-deprecated-frontend-automated-testing
BODY=$(echo "$SLACK_PAYLOAD" | jq --arg ch "C0AP09LKRDZ" '. + {channel: $ch}')
curl -sf -X POST \
-H "Authorization: Bearer $SLACK_BOT_TOKEN" \
-H "Content-Type: application/json" \
-d "$BODY" \
-o /dev/null \
https://slack.com/api/chat.postMessage
- name: Save unit coverage baseline
if: always() && hashFiles('coverage/lcov.info') != ''
uses: actions/upload-artifact@v6
with:
name: unit-coverage-baseline
path: coverage/lcov.info
retention-days: 90
if-no-files-found: warn
- name: Save E2E coverage baseline
if: always() && hashFiles('temp/e2e-coverage/coverage.lcov') != ''
uses: actions/upload-artifact@v6
with:
name: e2e-coverage-baseline
path: temp/e2e-coverage/coverage.lcov
retention-days: 90
if-no-files-found: warn

View File

@@ -1,123 +0,0 @@
name: Model Page Discovery
on:
schedule:
- cron: '0 9 * * 1'
workflow_dispatch:
jobs:
discover:
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
steps:
- name: Fetch model labels from hub API
id: hub
shell: bash
run: |
set -euo pipefail
curl -fsSL 'https://comfy.org/api/hub/labels?type=model' -o hub-labels.json
echo "Fetched $(jq '.labels | length' hub-labels.json) model labels from hub"
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v6
with:
sparse-checkout: apps/website/src/config/generated-models.json
- name: Compare against existing models
id: compare
shell: bash
run: |
set -euo pipefail
HUB_SLUGS=$(jq -r '[.labels[].name]' hub-labels.json)
EXISTING_SLUGS=$(node -e "
const fs = require('fs');
const models = JSON.parse(
fs.readFileSync(
'apps/website/src/config/generated-models.json',
'utf8'
)
);
console.log(JSON.stringify(models.map(m => m.slug)));
" 2>/dev/null || echo '[]')
ADDED_SLUGS=$(node -e "
const hub = $HUB_SLUGS;
const existing = new Set($EXISTING_SLUGS);
console.log(JSON.stringify(hub.filter(s => !existing.has(s))));
")
COUNT=$(node -e "console.log($ADDED_SLUGS.length)")
echo "new_count=$COUNT" >> \$GITHUB_OUTPUT
echo "new_slugs=$ADDED_SLUGS" >> \$GITHUB_OUTPUT
if [ "\$COUNT" -eq 0 ]; then
echo "No new models found."
else
echo "Found \$COUNT new model(s)"
fi
- name: Check for existing open discovery issue
id: existing_issue
if: steps.compare.outputs.new_count != '0'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
COUNT=$(gh issue list \
--repo "$GITHUB_REPOSITORY" \
--state open \
--search 'in:title "New models detected"' \
--json number \
--jq 'length')
echo "open_count=$COUNT" >> $GITHUB_OUTPUT
- name: Open GitHub issue for new models
if: |
steps.compare.outputs.new_count != '0' &&
steps.existing_issue.outputs.open_count == '0'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NEW_SLUGS: ${{ steps.compare.outputs.new_slugs }}
NEW_COUNT: ${{ steps.compare.outputs.new_count }}
shell: bash
run: |
SLUG_LIST=$(node -e "
const slugs = $NEW_SLUGS;
console.log(slugs.map(s => '- \`' + s + '\`').join('\n'));
")
gh issue create \
--repo "$GITHUB_REPOSITORY" \
--title "New models detected — add to model pages" \
--body "## $NEW_COUNT new model(s) found in hub
The weekly model discovery scan found model labels on the hub not yet in
\`apps/website/src/config/generated-models.json\`.
### New slugs ($NEW_COUNT)
$SLUG_LIST
### Next steps
1. Review which of these warrant an SEO model page
2. For local models: run \`SKIP_THUMBNAILS=1 pnpm generate:models\` and commit the result
3. For partner/API models: add to \`API_PROVIDER_MAP\` in \`generate-models.ts\`, regenerate, commit
---
*Generated by the [model-page-discovery workflow](https://github.com/$GITHUB_REPOSITORY/actions/workflows/model-page-discovery.yaml)*"
- name: Skip — open issue already exists
if: |
steps.compare.outputs.new_count != '0' &&
steps.existing_issue.outputs.open_count != '0'
run: echo "An open discovery issue already exists — skipping creation."
- name: No new models found
if: steps.compare.outputs.new_count == '0'
run: echo "No new models found — nothing to do."

1092
.github/workflows/pr-qa.yaml vendored Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -2,7 +2,7 @@ name: 'PR: Unified Report'
on:
workflow_run:
workflows: ['CI: Size Data', 'CI: Performance Report', 'CI: E2E Coverage']
workflows: ['CI: Size Data', 'CI: Performance Report']
types:
- completed
@@ -30,25 +30,110 @@ jobs:
- name: Resolve PR from workflow_run context
id: pr-meta
uses: ./.github/actions/resolve-pr-from-workflow-run
uses: actions/github-script@v8
with:
script: |
let pr = context.payload.workflow_run.pull_requests?.[0];
if (!pr) {
const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({
owner: context.repo.owner,
repo: context.repo.repo,
commit_sha: context.payload.workflow_run.head_sha,
});
pr = prs.find(p => p.state === 'open');
}
- name: Find size workflow run
if (!pr) {
core.info('No open PR found for this workflow run — skipping.');
core.setOutput('skip', 'true');
return;
}
// Verify the workflow_run head SHA matches the current PR head
const { data: livePr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
});
if (livePr.head.sha !== context.payload.workflow_run.head_sha) {
core.info(`Stale run: workflow SHA ${context.payload.workflow_run.head_sha} != PR head ${livePr.head.sha}`);
core.setOutput('skip', 'true');
return;
}
core.setOutput('skip', 'false');
core.setOutput('number', String(pr.number));
core.setOutput('base', livePr.base.ref);
core.setOutput('head-sha', livePr.head.sha);
- name: Find size workflow run for this commit
if: steps.pr-meta.outputs.skip != 'true'
id: find-size
uses: ./.github/actions/find-workflow-run
uses: actions/github-script@v8
with:
workflow-id: ci-size-data.yaml
head-sha: ${{ steps.pr-meta.outputs.head-sha }}
token: ${{ secrets.GITHUB_TOKEN }}
script: |
const headSha = '${{ steps.pr-meta.outputs.head-sha }}';
const { data: runs } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'ci-size-data.yaml',
head_sha: headSha,
per_page: 1,
});
- name: Find perf workflow run
const run = runs.workflow_runs[0];
if (!run) {
core.setOutput('status', 'pending');
return;
}
if (run.status !== 'completed') {
core.setOutput('status', 'pending');
return;
}
if (run.conclusion !== 'success') {
core.setOutput('status', 'failed');
return;
}
core.setOutput('status', 'ready');
core.setOutput('run-id', String(run.id));
- name: Find perf workflow run for this commit
if: steps.pr-meta.outputs.skip != 'true'
id: find-perf
uses: ./.github/actions/find-workflow-run
uses: actions/github-script@v8
with:
workflow-id: ci-perf-report.yaml
head-sha: ${{ steps.pr-meta.outputs.head-sha }}
token: ${{ secrets.GITHUB_TOKEN }}
script: |
const headSha = '${{ steps.pr-meta.outputs.head-sha }}';
const { data: runs } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'ci-perf-report.yaml',
head_sha: headSha,
per_page: 1,
});
const run = runs.workflow_runs[0];
if (!run) {
core.setOutput('status', 'pending');
return;
}
if (run.status !== 'completed') {
core.setOutput('status', 'pending');
return;
}
if (run.conclusion !== 'success') {
core.setOutput('status', 'failed');
return;
}
core.setOutput('status', 'ready');
core.setOutput('run-id', String(run.id));
- name: Download size data (current)
if: steps.pr-meta.outputs.skip != 'true' && steps.find-size.outputs.status == 'ready'
@@ -69,25 +154,6 @@ jobs:
path: temp/size-prev
if_no_artifact_found: warn
- name: Find coverage workflow run
if: steps.pr-meta.outputs.skip != 'true'
id: find-coverage
uses: ./.github/actions/find-workflow-run
with:
workflow-id: ci-tests-e2e-coverage.yaml
head-sha: ${{ steps.pr-meta.outputs.head-sha }}
not-found-status: skip
token: ${{ secrets.GITHUB_TOKEN }}
- name: Download coverage data
if: steps.pr-meta.outputs.skip != 'true' && steps.find-coverage.outputs.status == 'ready'
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
name: e2e-coverage
run_id: ${{ steps.find-coverage.outputs.run-id }}
path: temp/coverage
if_no_artifact_found: warn
- name: Download perf metrics (current)
if: steps.pr-meta.outputs.skip != 'true' && steps.find-perf.outputs.status == 'ready'
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
@@ -123,10 +189,9 @@ jobs:
- name: Generate unified report
if: steps.pr-meta.outputs.skip != 'true'
run: >
pnpm exec tsx scripts/unified-report.ts
node scripts/unified-report.js
--size-status=${{ steps.find-size.outputs.status }}
--perf-status=${{ steps.find-perf.outputs.status }}
--coverage-status=${{ steps.find-coverage.outputs.status }}
> pr-report.md
- name: Remove legacy separate comments

View File

@@ -77,7 +77,7 @@ jobs:
needs: setup
runs-on: ubuntu-latest
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.17
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.16
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,245 +0,0 @@
name: 'PR: Update Website Screenshots'
on:
pull_request:
types: [labeled]
issue_comment:
types: [created, edited]
concurrency:
group: ${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number || github.ref }}
cancel-in-progress: false
jobs:
update-screenshots:
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.58.1-noble
timeout-minutes: 15
permissions:
contents: write
issues: write
pull-requests: read
# Trigger: (1) label, (2) /slash-command, or (3) checkbox in E2E status comment
# ⚠️ This condition is duplicated on `post-starting-comment` — keep them in sync.
if: >
( github.event_name == 'pull_request' &&
github.event.label.name == 'Update Website Screenshots' ) ||
( github.event.issue.pull_request &&
github.event_name == 'issue_comment' &&
(
github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'
) &&
startsWith(github.event.comment.body, '/update-website-screenshots') ) ||
( github.event.issue.pull_request &&
github.event_name == 'issue_comment' &&
github.event.comment.user.login == 'github-actions[bot]' &&
github.actor != 'github-actions[bot]' &&
contains(github.event.comment.body, '<!-- WEBSITE_E2E_STATUS -->') &&
contains(github.event.comment.body, '- [x] Update website screenshots') )
outputs:
pr-number: ${{ steps.pr-info.outputs.pr-number }}
update-outcome: ${{ steps.update-screenshots.outcome }}
has-changes: ${{ steps.commit.outputs.has-changes }}
changed-count: ${{ steps.commit.outputs.changed-count }}
steps:
- name: Verify sender permissions
if: >
github.event_name == 'issue_comment' &&
contains(github.event.comment.body, '<!-- WEBSITE_E2E_STATUS -->')
uses: actions/github-script@v8
with:
script: |
const { data } = await github.rest.repos.getCollaboratorPermissionLevel({
...context.repo,
username: context.actor
})
if (!['admin', 'write'].includes(data.permission)) {
core.setFailed(`User ${context.actor} does not have write access`)
}
- name: Get PR info
id: pr-info
uses: actions/github-script@v8
env:
PR_NUMBER: ${{ github.event.number || github.event.issue.number }}
with:
script: |
const prNumber = Number(process.env.PR_NUMBER)
const { data: pr } = await github.rest.pulls.get({
...context.repo,
pull_number: prNumber
})
core.setOutput('pr-number', prNumber)
core.setOutput('branch', pr.head.ref)
- uses: actions/checkout@v6
with:
ref: ${{ steps.pr-info.outputs.branch }}
token: ${{ secrets.PR_GH_TOKEN }}
- name: Install pnpm
run: corepack enable && corepack prepare
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build website
env:
WEBSITE_GITHUB_STARS_OVERRIDE: 110000
run: pnpm --filter @comfyorg/website build
- name: Update screenshots
id: update-screenshots
run: pnpm --filter @comfyorg/website test:visual:update
continue-on-error: true
- name: Commit updated screenshots
id: commit
if: steps.update-screenshots.outcome == 'success'
run: |
git config --global --add safe.directory "$(pwd)"
git config --global user.name 'github-actions'
git config --global user.email 'github-actions@github.com'
CHANGED=$(git status --porcelain=v1 --untracked-files=all -- apps/website/e2e/ | wc -l)
echo "changed-count=${CHANGED}" >> $GITHUB_OUTPUT
if [ "$CHANGED" -eq 0 ]; then
echo "No screenshot changes to commit"
echo "has-changes=false" >> $GITHUB_OUTPUT
exit 0
fi
echo "has-changes=true" >> $GITHUB_OUTPUT
git add apps/website/e2e/
git commit -m "[automated] Update website screenshot expectations"
git push origin ${{ steps.pr-info.outputs.branch }}
- name: Upload test report
uses: actions/upload-artifact@v6
if: ${{ !cancelled() }}
with:
name: website-screenshot-update-report
path: apps/website/playwright-report/
retention-days: 14
- name: Remove label
if: always() && github.event_name == 'pull_request'
uses: actions/github-script@v8
env:
PR_NUMBER: ${{ steps.pr-info.outputs.pr-number }}
with:
script: |
try {
await github.rest.issues.removeLabel({
...context.repo,
issue_number: Number(process.env.PR_NUMBER),
name: 'Update Website Screenshots'
})
} catch (e) {
if (e.status !== 404) {
throw e
}
core.info('Label "Update Website Screenshots" was already removed')
}
post-starting-comment:
# Runs in parallel with update-screenshots to show "in progress" immediately.
# ⚠️ This condition is duplicated from `update-screenshots` — keep them in sync.
if: >
( github.event_name == 'pull_request' &&
github.event.label.name == 'Update Website Screenshots' ) ||
( github.event.issue.pull_request &&
github.event_name == 'issue_comment' &&
(
github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'
) &&
startsWith(github.event.comment.body, '/update-website-screenshots') ) ||
( github.event.issue.pull_request &&
github.event_name == 'issue_comment' &&
github.event.comment.user.login == 'github-actions[bot]' &&
github.actor != 'github-actions[bot]' &&
contains(github.event.comment.body, '<!-- WEBSITE_E2E_STATUS -->') &&
contains(github.event.comment.body, '- [x] Update website screenshots') )
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
concurrency:
group: website-pr-comment-${{ github.event.number || github.event.issue.number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/upsert-comment-section
with:
pr-number: ${{ github.event.number || github.event.issue.number }}
section-name: screenshot-update
comment-marker: '<!-- WEBSITE_CI_REPORT -->'
token: ${{ secrets.GITHUB_TOKEN }}
section-content: |-
## 📸 Screenshot Update
> [!NOTE]
> Updating screenshots… [View workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
post-result-comment:
needs: update-screenshots
if: always() && !cancelled() && needs.update-screenshots.result != 'skipped'
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
concurrency:
group: website-pr-comment-${{ needs.update-screenshots.outputs.pr-number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v6
- name: Build screenshot-update section content
id: content
uses: actions/github-script@v8
env:
UPDATE_OUTCOME: ${{ needs.update-screenshots.outputs.update-outcome }}
HAS_CHANGES: ${{ needs.update-screenshots.outputs.has-changes }}
CHANGED_COUNT: ${{ needs.update-screenshots.outputs.changed-count }}
with:
script: |
const outcome = process.env.UPDATE_OUTCOME
const hasChanges = process.env.HAS_CHANGES === 'true'
const changedCount = parseInt(process.env.CHANGED_COUNT) || 0
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`
const lines = ['## 📸 Screenshot Update', '']
if (outcome !== 'success') {
lines.push(
'> [!CAUTION]',
`> Screenshot update failed. [View workflow run](${runUrl})`
)
} else if (!hasChanges) {
lines.push(
'> [!TIP]',
'> All screenshots are already up to date.'
)
} else {
const s = changedCount === 1 ? '' : 's'
lines.push(
'> [!TIP]',
`> Updated ${changedCount} screenshot${s} and pushed to the branch.`
)
}
core.setOutput('section-content', lines.join('\n'))
- uses: ./.github/actions/upsert-comment-section
with:
pr-number: ${{ needs.update-screenshots.outputs.pr-number }}
section-name: screenshot-update
comment-marker: '<!-- WEBSITE_CI_REPORT -->'
token: ${{ secrets.GITHUB_TOKEN }}
section-content: ${{ steps.content.outputs.section-content }}

View File

@@ -1,70 +0,0 @@
---
name: 'PR: Vercel Website Preview'
on:
workflow_run:
workflows: ['CI: Vercel Website Preview']
types:
- completed
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_repository.full_name }}-${{ github.event.workflow_run.head_branch }}
cancel-in-progress: true
jobs:
comment:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
actions: read
# Uses head_branch as proxy for PR number (unavailable at job-level in workflow_run).
# Preview and E2E comment writes are NOT mutually serialized — the race window is
# small and self-healing on next push.
concurrency:
group: website-pr-comment-${{ github.event.workflow_run.head_branch }}
cancel-in-progress: false
if: >
github.repository == 'Comfy-Org/ComfyUI_frontend' &&
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.conclusion == 'success'
steps:
- uses: actions/checkout@v6
- name: Download preview metadata
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
name: vercel-preview
run_id: ${{ github.event.workflow_run.id }}
path: temp/vercel-preview
- name: Resolve PR number from workflow_run context
id: pr-meta
uses: ./.github/actions/resolve-pr-from-workflow-run
- name: Read preview URLs
if: steps.pr-meta.outputs.skip != 'true'
id: urls
run: |
echo "stable-url=$(cat temp/vercel-preview/stable-url.txt)" >> "$GITHUB_OUTPUT"
echo "unique-url=$(cat temp/vercel-preview/url.txt)" >> "$GITHUB_OUTPUT"
echo "short-sha=${HEAD_SHA:0:7}" >> "$GITHUB_OUTPUT"
env:
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
- name: Post preview comment
if: steps.pr-meta.outputs.skip != 'true'
uses: ./.github/actions/upsert-comment-section
with:
pr-number: ${{ steps.pr-meta.outputs.number }}
section-name: preview
comment-marker: '<!-- WEBSITE_CI_REPORT -->'
token: ${{ secrets.GITHUB_TOKEN }}
section-content: |-
## 🔗 Website Preview
**Website Preview:** ${{ steps.urls.outputs.stable-url }}
<sub>This commit: ${{ steps.urls.outputs.unique-url }}</sub>
<sub>Last updated: ${{ github.event.workflow_run.updated_at }} for `${{ steps.urls.outputs.short-sha }}`</sub>

View File

@@ -1,23 +1,14 @@
# Release workflow for ComfyUI frontend: version bump → PyPI publish → ComfyUI PR.
# Runs on a bi-weekly schedule for minor releases, or manually for patch/hotfix releases.
name: 'Release: ComfyUI'
# Automated bi-weekly workflow to bump ComfyUI frontend RC releases
name: 'Release: Bi-weekly ComfyUI'
on:
# Bi-weekly schedule: Monday at 20:00 UTC
# Schedule for Monday at 12:00 PM PST (20:00 UTC)
schedule:
- cron: '0 20 * * 1'
# Manual trigger for both on-demand minor and patch/hotfix releases
# Allow manual triggering (bypasses bi-weekly check)
workflow_dispatch:
inputs:
release_type:
description: 'minor = next minor version (bi-weekly cadence), patch = hotfix for current production version'
required: true
default: 'minor'
type: choice
options:
- minor
- patch
comfyui_fork:
description: 'ComfyUI fork to use for PR (e.g., Comfy-Org/ComfyUI)'
required: false
@@ -50,11 +41,10 @@ jobs:
- name: Summary
run: |
echo "## Release Check" >> $GITHUB_STEP_SUMMARY
echo "## Bi-weekly Check" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- Is release week: ${{ steps.check.outputs.is_release_week }}" >> $GITHUB_STEP_SUMMARY
echo "- Manual trigger: ${{ github.event_name == 'workflow_dispatch' }}" >> $GITHUB_STEP_SUMMARY
echo "- Release type: ${{ inputs.release_type || 'minor (scheduled)' }}" >> $GITHUB_STEP_SUMMARY
resolve-version:
needs: check-release-week
@@ -86,8 +76,6 @@ jobs:
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
with:
package_json_file: frontend/package.json
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -101,8 +89,6 @@ jobs:
- name: Resolve release information
id: resolve
working-directory: frontend
env:
RELEASE_TYPE: ${{ inputs.release_type || 'minor' }}
run: |
set -euo pipefail

View File

@@ -142,20 +142,6 @@ jobs:
fi
echo "✅ Branch '$BRANCH' exists"
- name: Ensure packageManager field exists
run: |
if ! grep -q '"packageManager"' package.json; then
# Old branches (e.g. core/1.42) predate the packageManager field.
# Inject it so pnpm/action-setup can resolve the version.
node -e "
const fs = require('fs');
const pkg = JSON.parse(fs.readFileSync('package.json','utf8'));
pkg.packageManager = 'pnpm@10.33.0';
fs.writeFileSync('package.json', JSON.stringify(pkg, null, 2) + '\n');
"
echo "Injected packageManager into package.json for legacy branch"
fi
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0

View File

@@ -1,59 +0,0 @@
# Description: Manual workflow to refresh the apps/website Ashby roles snapshot
# and open a PR. Merging the PR triggers the existing Vercel website production
# deploy via ci-vercel-website-preview.yaml.
name: 'Release: Website'
on:
workflow_dispatch:
concurrency:
group: release-website
cancel-in-progress: true
jobs:
refresh-snapshot:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
ref: main
persist-credentials: false
- name: Refresh Ashby snapshot
uses: ./.github/actions/ashby-pull
with:
api_key: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
job_board_name: ${{ secrets.WEBSITE_ASHBY_JOB_BOARD_NAME }}
- name: Create Pull Request
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0 # v8.1.0
with:
token: ${{ secrets.PR_GH_TOKEN }}
commit-message: 'chore(website): refresh Ashby roles snapshot'
title: 'chore(website): refresh Ashby roles snapshot'
body: |
Automated refresh of `apps/website/src/data/ashby-roles.snapshot.json`
from the Ashby job board API.
**Flow:**
1. `Release: Website` workflow ran (manual trigger).
2. This PR opens with the regenerated snapshot.
3. `CI: Vercel Website Preview` deploys a preview for review.
4. Merging to `main` triggers the production Vercel deploy.
The snapshot fallback in `apps/website/src/utils/ashby.ts` remains
intact: builds without `WEBSITE_ASHBY_API_KEY` continue to use the
committed snapshot.
Triggered by workflow run `${{ github.run_id }}`.
branch: chore/refresh-ashby-snapshot-${{ github.run_id }}
base: main
labels: |
Release:Website
delete-branch: true

5
.gitignore vendored
View File

@@ -99,4 +99,7 @@ vitest.config.*.timestamp*
# Weekly docs check output
/output.txt
.amp
.amp
.playwright-cli/
.playwright/
.claude/scheduled_tasks.lock

View File

@@ -1,13 +1,5 @@
#!/usr/bin/env bash
# Skip in CI: the canonical knip check runs in ci-lint-format on every
# PR, and bot workflows (e.g. i18n-update-core) populate ComfyUI/ via
# setup-comfyui-server, which contaminates knip's project glob with the
# devtools copy under custom_nodes and produces false-positive failures.
if [ -n "${CI-}" ]; then
exit 0
fi
# Run Knip with cache via package script
pnpm knip 1>&2

View File

@@ -9,6 +9,7 @@
"packages/registry-types/src/comfyRegistryTypes.ts",
"public/materialdesignicons.min.css",
"src/types/generatedManagerTypes.ts",
"**/__fixtures__/**/*.json"
"**/__fixtures__/**/*.json",
"scripts/qa-report-template.html"
]
}

View File

@@ -64,7 +64,6 @@
]
}
],
"no-unsafe-optional-chaining": "error",
"no-self-assign": "allow",
"no-unused-expressions": "off",
"no-unused-private-class-members": "off",
@@ -84,16 +83,6 @@
"typescript/no-unsafe-declaration-merging": "off",
"typescript/no-unused-vars": "off",
"unicorn/no-empty-file": "off",
"vitest/require-mock-type-parameters": "off",
"vitest/consistent-each-for": [
"error",
{
"test": "for",
"it": "for",
"describe": "for",
"suite": "for"
}
],
"unicorn/no-new-array": "off",
"unicorn/no-single-promise-in-promise-methods": "off",
"unicorn/no-useless-fallback-in-spread": "off",
@@ -115,7 +104,8 @@
"allowInterfaces": "always"
}
],
"vue/no-import-compiler-macros": "error"
"vue/no-import-compiler-macros": "error",
"vue/no-dupe-keys": "error"
},
"overrides": [
{
@@ -126,60 +116,13 @@
},
{
"files": ["browser_tests/**/*.ts"],
"jsPlugins": ["eslint-plugin-playwright"],
"rules": {
"typescript/no-explicit-any": "error",
"no-async-promise-executor": "error",
"no-control-regex": "error",
"no-useless-rename": "error",
"no-unused-private-class-members": "error",
"unicorn/no-empty-file": "error",
"playwright/consistent-spacing-between-blocks": "error",
"playwright/expect-expect": [
"error",
{
"assertFunctionNames": [
"recordMeasurement",
"logMeasurement",
"builderSaveAs"
],
"assertFunctionPatterns": [
"^expect",
"^assert",
"^verify",
"^searchAndExpect",
"waitForOpen",
"waitForClosed",
"waitForRequest"
]
}
],
"playwright/max-nested-describe": "error",
"playwright/no-duplicate-hooks": "error",
"playwright/no-element-handle": "error",
"playwright/no-eval": "error",
"playwright/no-focused-test": "error",
"playwright/no-force-option": "error",
"playwright/no-networkidle": "error",
"playwright/no-page-pause": "error",
"playwright/no-skipped-test": "error",
"playwright/no-unsafe-references": "error",
"playwright/no-unused-locators": "error",
"playwright/no-useless-await": "error",
"playwright/no-useless-not": "error",
"playwright/no-wait-for-navigation": "error",
"playwright/no-wait-for-selector": "error",
"playwright/no-wait-for-timeout": "error",
"playwright/prefer-hooks-on-top": "error",
"playwright/prefer-locator": "error",
"playwright/prefer-to-have-count": "error",
"playwright/prefer-to-have-length": "error",
"playwright/prefer-web-first-assertions": "error",
"playwright/prefer-native-locators": "error",
"playwright/require-to-pass-timeout": "error",
"playwright/valid-expect": "error",
"playwright/valid-expect-in-promise": "error",
"playwright/valid-title": "error"
"unicorn/no-empty-file": "error"
}
}
]

View File

@@ -67,7 +67,6 @@
"ignoreFiles": [
"node_modules/**",
"dist/**",
"**/dist/**",
"playwright-report/**",
"public/**",
"src/lib/litegraph/**"

View File

@@ -1,12 +1,16 @@
{
"recommendations": [
"antfu.vite",
"austenc.tailwind-docs",
"bradlc.vscode-tailwindcss",
"davidanson.vscode-markdownlint",
"dbaeumer.vscode-eslint",
"donjayamanne.githistory",
"eamodio.gitlens",
"github.vscode-github-actions",
"github.vscode-pull-request-github",
"hbenl.vscode-test-explorer",
"kisstkondoros.vscode-codemetrics",
"lokalise.i18n-ally",
"ms-playwright.playwright",
"oxc.oxc-vscode",

View File

@@ -44,7 +44,6 @@ This project uses **pnpm**. Always prefer scripts defined in `package.json` (e.g
## Build, Test, and Development Commands
- `pnpm dev`: Start Vite dev server.
- `pnpm dev:cloud`: Dev server connected to cloud backend (testcloud.comfy.org)
- `pnpm dev:electron`: Dev server with Electron API mocks
- `pnpm build`: Type-check then production build to `dist/`
- `pnpm preview`: Preview the production build locally
@@ -180,12 +179,6 @@ This project uses **pnpm**. Always prefer scripts defined in `package.json` (e.g
24. Do not use function expressions if it's possible to use function declarations instead
25. Watch out for [Code Smells](https://wiki.c2.com/?CodeSmell) and refactor to avoid them
## Design Standards
Before implementing any user-facing feature, consult the [Comfy Design Standards](https://www.figma.com/design/QreIv5htUaSICNuO2VBHw0/Comfy-Design-Standards) Figma file. Use the Figma MCP to fetch it live — the file is the single source of truth and may be updated by designers at any time.
See `docs/guidance/design-standards.md` for Figma file keys, section node IDs, and component references.
## Testing Guidelines
See @docs/testing/\*.md for detailed patterns.
@@ -233,7 +226,6 @@ See @docs/testing/\*.md for detailed patterns.
- shadcn/vue: <https://www.shadcn-vue.com/>
- Reka UI: <https://reka-ui.com/>
- PrimeVue: <https://primevue.org>
- Comfy Design Standards: <https://www.figma.com/design/QreIv5htUaSICNuO2VBHw0/Comfy-Design-Standards>
- ComfyUI: <https://docs.comfy.org>
- Electron: <https://www.electronjs.org/docs/latest/>
- Wiki: <https://deepwiki.com/Comfy-Org/ComfyUI_frontend/1-overview>
@@ -312,16 +304,13 @@ When referencing Comfy-Org repos:
- Instead use a semantic value from the `style.css` theme
- e.g. `bg-node-component-surface`
- NEVER use `:class="[]"` to merge class names
- Always use `import { cn } from '@comfyorg/tailwind-utils'`
- Always use `import { cn } from '@/utils/tailwindUtil'`
- e.g. `<div :class="cn('text-node-component-header-icon', hasError && 'text-danger')" />`
- Use `cn()` inline in the template when feasible instead of creating a `computed` to hold the value
- NEVER use `!important` or the `!` important prefix for tailwind classes
- Find existing `!important` classes that are interfering with the styling and propose corrections of those instead.
- NEVER use arbitrary percentage values like `w-[80%]` when a Tailwind fraction utility exists
- Use `w-4/5` instead of `w-[80%]`, `w-1/2` instead of `w-[50%]`, etc.
- NEVER use font-size classes (`text-xs`, `text-sm`, etc.) to size `icon-[...]` (iconify) icons
- Iconify icons size via `width`/`height: 1.2em`, so font-size produces unpredictable results
- Use `size-*` classes for explicit sizing, or set font-size on the **parent** container and let `1.2em` scale naturally
## Agent-only rules

View File

@@ -69,9 +69,6 @@
/src/renderer/extensions/vueNodes/widgets/composables/usePainterWidget.ts @jtydhr88
/src/lib/litegraph/src/widgets/PainterWidget.ts @jtydhr88
# GLSL
/src/renderer/glsl/ @jtydhr88 @pythongosssss @christian-byrne
# 3D
/src/extensions/core/load3d.ts @jtydhr88
/src/extensions/core/load3dLazy.ts @jtydhr88

View File

@@ -62,37 +62,6 @@ python main.py --port 8188 --cpu
- Run `pnpm dev:electron` to start the dev server with electron API mocked
- Run `pnpm dev:cloud` to start the dev server against the cloud backend (instead of local ComfyUI server)
#### Testing with Cloud & Staging Environments
Some features — particularly **partner/API nodes** (e.g. BFL, OpenAI, Stability AI) — require a cloud backend for authentication and billing. Running these against a local ComfyUI instance will result in permission errors or logged-out states. There are two ways to connect to a cloud/staging backend:
**Option 1: Frontend — `pnpm dev:cloud`**
The simplest approach. This proxies all API requests to the test cloud environment:
```bash
pnpm dev:cloud
```
This sets `DEV_SERVER_COMFYUI_URL` to `https://testcloud.comfy.org/` automatically. You can also set this variable manually in your `.env` file to target a different environment:
```bash
# .env
DEV_SERVER_COMFYUI_URL=https://stagingcloud.comfy.org/
```
Any `*.comfy.org` URL automatically enables cloud mode, which includes the GCS media proxy needed for viewing generated images and videos. See [.env_example](.env_example) for all available cloud URLs.
**Option 2: Backend — `--comfy-api-base`**
Alternatively, launch the ComfyUI backend pointed at the staging API:
```bash
python main.py --comfy-api-base https://stagingapi.comfy.org --verbose
```
Then run `pnpm dev` as usual. This keeps the frontend in local mode but routes backend API calls through staging.
#### Access dev server on touch devices
Enable remote access to the dev server by setting `VITE_REMOTE_DEV` in `.env` to `true`.

View File

@@ -5,14 +5,12 @@
"scripts": {
"lint": "nx run @comfyorg/desktop-ui:lint",
"typecheck": "nx run @comfyorg/desktop-ui:typecheck",
"test:unit": "vitest run --config vitest.config.mts",
"storybook": "storybook dev -p 6007",
"build-storybook": "storybook build -o dist/storybook"
},
"dependencies": {
"@comfyorg/comfyui-electron-types": "catalog:",
"@comfyorg/shared-frontend-utils": "workspace:*",
"@comfyorg/tailwind-utils": "workspace:*",
"@primevue/core": "catalog:",
"@primevue/themes": "catalog:",
"@vueuse/core": "catalog:",

View File

@@ -20,15 +20,15 @@
}
.p-button-danger {
background-color: var(--color-coral-700);
background-color: var(--color-coral-red-600);
}
.p-button-danger:hover {
background-color: var(--color-coral-600);
background-color: var(--color-coral-red-500);
}
.p-button-danger:active {
background-color: var(--color-coral-500);
background-color: var(--color-coral-red-400);
}
.task-div .p-card {

View File

@@ -32,7 +32,7 @@ import { useI18n } from 'vue-i18n'
import { useTerminal } from '@/composables/bottomPanelTabs/useTerminal'
import { electronAPI, isElectron } from '@/utils/envUtil'
import { cn } from '@comfyorg/tailwind-utils'
import { cn } from '@/utils/tailwindUtil'
const { t } = useI18n()

View File

@@ -1,97 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it } from 'vitest'
import { createI18n } from 'vue-i18n'
import StartupDisplay from '@/components/common/StartupDisplay.vue'
const i18n = createI18n({
legacy: false,
locale: 'en',
messages: { en: { g: { logoAlt: 'ComfyUI' } } }
})
const ProgressBarStub = {
props: ['mode', 'value', 'showValue'],
template:
'<div data-testid="progress-bar" :data-mode="mode" :data-value="value" />'
}
function renderDisplay(
props: {
progressPercentage?: number
title?: string
statusText?: string
hideProgress?: boolean
fullScreen?: boolean
} = {}
) {
return render(StartupDisplay, {
props,
global: {
plugins: [[PrimeVue, { unstyled: true }], i18n],
stubs: { ProgressBar: ProgressBarStub }
}
})
}
describe('StartupDisplay', () => {
describe('progressMode', () => {
it('renders indeterminate mode when progressPercentage is undefined', () => {
renderDisplay()
expect(screen.getByTestId('progress-bar').dataset.mode).toBe(
'indeterminate'
)
})
it('renders determinate mode when progressPercentage is provided', () => {
renderDisplay({ progressPercentage: 50 })
expect(screen.getByTestId('progress-bar').dataset.mode).toBe(
'determinate'
)
})
it('passes progressPercentage as value to the progress bar', () => {
renderDisplay({ progressPercentage: 75 })
expect(screen.getByTestId('progress-bar').dataset.value).toBe('75')
})
})
describe('hideProgress', () => {
it('hides the progress bar when hideProgress is true', () => {
renderDisplay({ hideProgress: true })
expect(screen.queryByTestId('progress-bar')).toBeNull()
})
it('shows the progress bar by default', () => {
renderDisplay()
expect(screen.getByTestId('progress-bar')).toBeDefined()
})
})
describe('title', () => {
it('renders the title text when provided', () => {
renderDisplay({ title: 'Loading...' })
expect(screen.getByText('Loading...')).toBeDefined()
})
it('does not render h1 when title is not provided', () => {
renderDisplay()
expect(screen.queryByRole('heading', { level: 1 })).toBeNull()
})
})
describe('statusText', () => {
it('renders statusText with data-testid attribute', () => {
renderDisplay({ statusText: 'Starting server' })
expect(screen.getByTestId('startup-status-text').textContent).toContain(
'Starting server'
)
})
it('does not render statusText element when not provided', () => {
renderDisplay()
expect(screen.queryByTestId('startup-status-text')).toBeNull()
})
})
})

View File

@@ -1,208 +0,0 @@
import { render, screen, waitFor } from '@testing-library/vue'
import userEvent from '@testing-library/user-event'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi, beforeEach } from 'vitest'
vi.mock('@comfyorg/shared-frontend-utils/networkUtil', () => ({
checkUrlReachable: vi.fn()
}))
import { checkUrlReachable } from '@comfyorg/shared-frontend-utils/networkUtil'
import UrlInput from '@/components/common/UrlInput.vue'
import { ValidationState } from '@/utils/validationUtil'
const InputTextStub = {
props: ['modelValue', 'invalid'],
emits: ['update:modelValue', 'blur'],
template: `<input
data-testid="url-input"
:value="modelValue"
:data-invalid="invalid"
@input="$emit('update:modelValue', $event.target.value)"
@blur="$emit('blur')"
/>`
}
const InputIconStub = {
template: '<span data-testid="input-icon" />'
}
const IconFieldStub = {
template: '<div><slot /></div>'
}
function renderUrlInput(
modelValue = '',
validateUrlFn?: (url: string) => Promise<boolean>
) {
return render(UrlInput, {
props: { modelValue, ...(validateUrlFn ? { validateUrlFn } : {}) },
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
InputText: InputTextStub,
InputIcon: InputIconStub,
IconField: IconFieldStub
}
}
})
}
describe('UrlInput', () => {
beforeEach(() => {
vi.resetAllMocks()
})
describe('initial validation on mount', () => {
it('stays IDLE when modelValue is empty on mount', async () => {
renderUrlInput('')
await waitFor(() => {
expect(screen.getByTestId('url-input').dataset.invalid).toBe('false')
})
})
it('sets VALID state when modelValue is a reachable URL on mount', async () => {
vi.mocked(checkUrlReachable).mockResolvedValue(true)
renderUrlInput('https://example.com')
await waitFor(() => {
expect(screen.getByTestId('url-input').dataset.invalid).toBe('false')
})
})
it('sets INVALID state when URL is not reachable on mount', async () => {
vi.mocked(checkUrlReachable).mockResolvedValue(false)
renderUrlInput('https://unreachable.example')
await waitFor(() => {
expect(screen.getByTestId('url-input').dataset.invalid).toBe('true')
})
})
})
describe('input handling', () => {
it('resets validation state to IDLE on user input', async () => {
vi.mocked(checkUrlReachable).mockResolvedValue(false)
renderUrlInput('https://bad.example')
await waitFor(() => {
expect(screen.getByTestId('url-input').dataset.invalid).toBe('true')
})
const user = userEvent.setup()
await user.type(screen.getByTestId('url-input'), 'x')
expect(screen.getByTestId('url-input').dataset.invalid).toBe('false')
})
it('strips whitespace from typed input', async () => {
const onUpdate = vi.fn()
render(UrlInput, {
props: {
modelValue: '',
'onUpdate:modelValue': onUpdate
},
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
InputText: InputTextStub,
InputIcon: InputIconStub,
IconField: IconFieldStub
}
}
})
const user = userEvent.setup()
const input = screen.getByTestId('url-input')
await user.type(input, 'htt ps')
expect((input as HTMLInputElement).value).not.toContain(' ')
})
})
describe('blur handling', () => {
it('emits update:modelValue on blur', async () => {
const onUpdate = vi.fn()
render(UrlInput, {
props: {
modelValue: 'https://example.com',
'onUpdate:modelValue': onUpdate
},
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
InputText: InputTextStub,
InputIcon: InputIconStub,
IconField: IconFieldStub
}
}
})
const user = userEvent.setup()
await user.click(screen.getByTestId('url-input'))
await user.tab()
expect(onUpdate).toHaveBeenCalled()
})
it('normalizes URL on blur', async () => {
const onUpdate = vi.fn()
render(UrlInput, {
props: {
modelValue: 'https://example.com',
'onUpdate:modelValue': onUpdate
},
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
InputText: InputTextStub,
InputIcon: InputIconStub,
IconField: IconFieldStub
}
}
})
const user = userEvent.setup()
await user.click(screen.getByTestId('url-input'))
await user.tab()
const emittedUrl = onUpdate.mock.calls[0]?.[0]
expect(emittedUrl).toBe('https://example.com/')
})
})
describe('custom validateUrlFn', () => {
it('uses custom validateUrlFn when provided', async () => {
const customValidator = vi.fn().mockResolvedValue(true)
renderUrlInput('https://custom.example', customValidator)
await waitFor(() => {
expect(customValidator).toHaveBeenCalledWith('https://custom.example')
})
expect(checkUrlReachable).not.toHaveBeenCalled()
})
})
describe('state-change emission', () => {
it('emits state-change when validation state changes', async () => {
const onStateChange = vi.fn()
vi.mocked(checkUrlReachable).mockResolvedValue(true)
render(UrlInput, {
props: {
modelValue: 'https://example.com',
'onState-change': onStateChange
},
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
InputText: InputTextStub,
InputIcon: InputIconStub,
IconField: IconFieldStub
}
}
})
await waitFor(() => {
expect(onStateChange).toHaveBeenCalledWith(ValidationState.VALID)
})
})
})
})

View File

@@ -1,112 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi } from 'vitest'
import { createI18n } from 'vue-i18n'
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => ({
getPlatform: vi.fn().mockReturnValue('win32')
}))
}))
vi.mock('@/i18n', () => ({
t: (key: string) => key,
te: () => false,
st: (_key: string, fallback: string) => fallback
}))
import type { TorchDeviceType } from '@comfyorg/comfyui-electron-types'
import GpuPicker from '@/components/install/GpuPicker.vue'
const i18n = createI18n({
legacy: false,
locale: 'en',
missingWarn: false,
fallbackWarn: false,
messages: { en: {} }
})
const HardwareOptionStub = {
props: ['imagePath', 'placeholderText', 'subtitle', 'selected'],
emits: ['click'],
template:
'<button :data-testid="placeholderText" :data-selected="selected" @click="$emit(\'click\')" >{{ placeholderText }}</button>'
}
function renderPicker(device: TorchDeviceType | null = null) {
return render(GpuPicker, {
props: { device },
global: {
plugins: [[PrimeVue, { unstyled: true }], i18n],
stubs: {
HardwareOption: HardwareOptionStub,
Tag: {
props: ['value'],
template: '<span data-testid="recommended-tag">{{ value }}</span>'
}
}
}
})
}
describe('GpuPicker', () => {
describe('recommended badge', () => {
it('shows recommended badge for nvidia', () => {
renderPicker('nvidia')
expect(screen.getByTestId('recommended-tag')).toBeVisible()
})
it('shows recommended badge for amd', () => {
renderPicker('amd')
expect(screen.getByTestId('recommended-tag')).toBeVisible()
})
it('does not show recommended badge for cpu', () => {
renderPicker('cpu')
expect(screen.getByTestId('recommended-tag')).not.toBeVisible()
})
it('does not show recommended badge for unsupported', () => {
renderPicker('unsupported')
expect(screen.getByTestId('recommended-tag')).not.toBeVisible()
})
it('does not show recommended badge when no device is selected', () => {
renderPicker(null)
expect(screen.getByTestId('recommended-tag')).not.toBeVisible()
})
})
describe('selection state', () => {
it('marks nvidia as selected when device is nvidia', () => {
renderPicker('nvidia')
expect(screen.getByTestId('NVIDIA').dataset.selected).toBe('true')
})
it('marks cpu as selected when device is cpu', () => {
renderPicker('cpu')
expect(screen.getByTestId('CPU').dataset.selected).toBe('true')
})
it('marks unsupported as selected when device is unsupported', () => {
renderPicker('unsupported')
expect(screen.getByTestId('Manual Install').dataset.selected).toBe('true')
})
it('no option is selected when device is null', () => {
renderPicker(null)
expect(screen.getByTestId('CPU').dataset.selected).toBe('false')
expect(screen.getByTestId('NVIDIA').dataset.selected).toBe('false')
})
})
describe('gpu options on non-darwin platform', () => {
it('shows NVIDIA, AMD, CPU, and Manual Install options', () => {
renderPicker(null)
expect(screen.getByTestId('NVIDIA')).toBeDefined()
expect(screen.getByTestId('AMD')).toBeDefined()
expect(screen.getByTestId('CPU')).toBeDefined()
expect(screen.getByTestId('Manual Install')).toBeDefined()
})
})
})

View File

@@ -4,7 +4,7 @@
<button
:class="
cn(
'hardware-option flex h-[190px] w-[170px] flex-col items-center rounded-3xl border-4 bg-neutral-900/70 p-5 transition-all duration-200',
'hardware-option w-[170px] h-[190px] p-5 flex flex-col items-center rounded-3xl transition-all duration-200 bg-neutral-900/70 border-4',
selected ? 'border-solid border-brand-yellow' : 'border-transparent'
)
"
@@ -12,13 +12,13 @@
>
<!-- Icon/Logo Area - Rounded square container -->
<div
class="icon-container flex h-[110px] w-[110px] shrink-0 items-center justify-center overflow-hidden rounded-2xl bg-neutral-800"
class="icon-container w-[110px] h-[110px] shrink-0 rounded-2xl bg-neutral-800 flex items-center justify-center overflow-hidden"
>
<img
v-if="imagePath"
:src="imagePath"
:alt="placeholderText"
class="size-full object-cover"
class="w-full h-full object-cover"
style="object-position: 57% center"
draggable="false"
/>
@@ -28,7 +28,7 @@
</div>
<!-- Text Content -->
<div v-if="subtitle" class="mt-4 text-center">
<div v-if="subtitle" class="text-center mt-4">
<div class="text-sm text-neutral-500">{{ subtitle }}</div>
</div>
</button>
@@ -36,7 +36,7 @@
</template>
<script setup lang="ts">
import { cn } from '@comfyorg/tailwind-utils'
import { cn } from '@/utils/tailwindUtil'
interface Props {
imagePath?: string

View File

@@ -1,223 +0,0 @@
import { render, screen, waitFor } from '@testing-library/vue'
import userEvent from '@testing-library/user-event'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi, beforeEach } from 'vitest'
import { createI18n } from 'vue-i18n'
const mockValidateComfyUISource = vi.fn()
const mockShowDirectoryPicker = vi.fn()
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => ({
validateComfyUISource: mockValidateComfyUISource,
showDirectoryPicker: mockShowDirectoryPicker
}))
}))
import MigrationPicker from '@/components/install/MigrationPicker.vue'
const i18n = createI18n({
legacy: false,
locale: 'en',
messages: {
en: {
install: {
migrationSourcePathDescription: 'Source path description',
migrationOptional: 'Migration is optional',
selectItemsToMigrate: 'Select items to migrate',
pathValidationFailed: 'Validation failed',
failedToSelectDirectory: 'Failed to select directory',
locationPicker: {
migrationPathPlaceholder: 'Enter path'
}
}
}
}
})
const InputTextStub = {
props: ['modelValue', 'invalid'],
emits: ['update:modelValue'],
template: `<input
data-testid="source-input"
:value="modelValue"
@input="$emit('update:modelValue', $event.target.value)"
/>`
}
const CheckboxStub = {
props: ['modelValue', 'inputId', 'binary'],
emits: ['update:modelValue', 'click'],
template: `<input
type="checkbox"
:data-testid="'checkbox-' + inputId"
:checked="modelValue"
@change="$emit('update:modelValue', $event.target.checked)"
@click.stop="$emit('click')"
/>`
}
function renderPicker(sourcePath = '', migrationItemIds: string[] = []) {
return render(MigrationPicker, {
props: { sourcePath, migrationItemIds },
global: {
plugins: [[PrimeVue, { unstyled: true }], i18n],
stubs: {
InputText: InputTextStub,
Checkbox: CheckboxStub,
Button: { template: '<button data-testid="browse-btn" />' },
Message: {
props: ['severity'],
template: '<div data-testid="error-msg"><slot /></div>'
}
}
}
})
}
describe('MigrationPicker', () => {
beforeEach(() => {
vi.resetAllMocks()
})
describe('isValidSource', () => {
it('hides migration options when source path is empty', () => {
renderPicker('')
expect(screen.queryByText('Select items to migrate')).toBeNull()
})
it('shows migration options when source path is valid', async () => {
mockValidateComfyUISource.mockResolvedValue({ isValid: true })
const { rerender } = renderPicker('')
await rerender({ sourcePath: '/valid/path' })
await waitFor(() => {
expect(screen.getByText('Select items to migrate')).toBeDefined()
})
})
it('shows optional message when no valid source', () => {
renderPicker('')
expect(screen.getByText('Migration is optional')).toBeDefined()
})
})
describe('validateSource', () => {
it('clears error when source path becomes empty', async () => {
mockValidateComfyUISource.mockResolvedValue({
isValid: false,
error: 'Not found'
})
const user = userEvent.setup()
renderPicker()
await user.type(screen.getByTestId('source-input'), '/bad/path')
await waitFor(() => {
expect(screen.getByTestId('error-msg')).toBeDefined()
})
await user.clear(screen.getByTestId('source-input'))
await waitFor(() => {
expect(screen.queryByTestId('error-msg')).toBeNull()
})
})
it('shows error message when validation fails', async () => {
mockValidateComfyUISource.mockResolvedValue({
isValid: false,
error: 'Path not found'
})
const user = userEvent.setup()
renderPicker()
await user.type(screen.getByTestId('source-input'), '/bad/path')
await waitFor(() => {
expect(screen.getByTestId('error-msg')).toBeDefined()
})
})
it('shows no error when validation passes', async () => {
mockValidateComfyUISource.mockResolvedValue({ isValid: true })
const user = userEvent.setup()
renderPicker()
await user.type(screen.getByTestId('source-input'), '/valid/path')
await waitFor(() => {
expect(screen.queryByTestId('error-msg')).toBeNull()
})
})
})
describe('migrationItemIds watchEffect', () => {
it('emits all item IDs by default (all items start selected)', async () => {
const onUpdate = vi.fn()
render(MigrationPicker, {
props: {
sourcePath: '',
migrationItemIds: [],
'onUpdate:migrationItemIds': onUpdate
},
global: {
plugins: [[PrimeVue, { unstyled: true }], i18n],
stubs: {
InputText: InputTextStub,
Checkbox: CheckboxStub,
Button: { template: '<button />' },
Message: { template: '<div />' }
}
}
})
await waitFor(() => {
expect(onUpdate).toHaveBeenCalled()
const emittedIds = onUpdate.mock.calls[0][0]
expect(Array.isArray(emittedIds)).toBe(true)
expect(emittedIds.length).toBeGreaterThan(0)
})
})
})
describe('browse path', () => {
it('opens directory picker on browse click', async () => {
mockShowDirectoryPicker.mockResolvedValue(null)
renderPicker()
const user = userEvent.setup()
await user.click(screen.getByTestId('browse-btn'))
expect(mockShowDirectoryPicker).toHaveBeenCalledOnce()
})
it('updates source path when directory is selected', async () => {
mockShowDirectoryPicker.mockResolvedValue('/selected/path')
mockValidateComfyUISource.mockResolvedValue({ isValid: true })
const onUpdate = vi.fn()
render(MigrationPicker, {
props: {
sourcePath: '',
'onUpdate:sourcePath': onUpdate
},
global: {
plugins: [[PrimeVue, { unstyled: true }], i18n],
stubs: {
InputText: InputTextStub,
Checkbox: CheckboxStub,
Button: { template: '<button data-testid="browse-btn" />' },
Message: { template: '<div />' }
}
}
})
const user = userEvent.setup()
await user.click(screen.getByTestId('browse-btn'))
await waitFor(() => {
expect(onUpdate).toHaveBeenCalledWith('/selected/path')
})
})
})
})

View File

@@ -1,85 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi } from 'vitest'
import { defineComponent } from 'vue'
vi.mock('@/i18n', () => ({
t: (key: string) => key
}))
import StatusTag from '@/components/maintenance/StatusTag.vue'
const TagStub = defineComponent({
name: 'Tag',
props: {
icon: String,
severity: String,
value: String
},
template: `<span data-testid="tag" :data-icon="icon" :data-severity="severity" :data-value="value">{{ value }}</span>`
})
function renderStatusTag(props: { error: boolean; refreshing?: boolean }) {
return render(StatusTag, {
props,
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: { Tag: TagStub }
}
})
}
describe('StatusTag', () => {
describe('refreshing state', () => {
it('shows info severity when refreshing', () => {
renderStatusTag({ error: false, refreshing: true })
expect(screen.getByTestId('tag').dataset.severity).toBe('info')
})
it('shows refreshing translation key when refreshing', () => {
renderStatusTag({ error: false, refreshing: true })
expect(screen.getByTestId('tag').dataset.value).toBe(
'maintenance.refreshing'
)
})
it('shows question icon when refreshing', () => {
renderStatusTag({ error: false, refreshing: true })
expect(screen.getByTestId('tag').dataset.icon).toBeDefined()
})
})
describe('error state', () => {
it('shows danger severity when error is true', () => {
renderStatusTag({ error: true })
expect(screen.getByTestId('tag').dataset.severity).toBe('danger')
})
it('shows error translation key when error is true', () => {
renderStatusTag({ error: true })
expect(screen.getByTestId('tag').dataset.value).toBe('g.error')
})
})
describe('OK state', () => {
it('shows success severity when not refreshing and not error', () => {
renderStatusTag({ error: false })
expect(screen.getByTestId('tag').dataset.severity).toBe('success')
})
it('shows OK translation key when not refreshing and not error', () => {
renderStatusTag({ error: false })
expect(screen.getByTestId('tag').dataset.value).toBe('maintenance.OK')
})
})
describe('precedence', () => {
it('shows refreshing state when both refreshing and error are true', () => {
renderStatusTag({ error: true, refreshing: true })
expect(screen.getByTestId('tag').dataset.severity).toBe('info')
expect(screen.getByTestId('tag').dataset.value).toBe(
'maintenance.refreshing'
)
})
})
})

View File

@@ -1,89 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi } from 'vitest'
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => ({
Validation: { validateInstallation: vi.fn() }
}))
}))
vi.mock('@/constants/desktopMaintenanceTasks', () => ({
DESKTOP_MAINTENANCE_TASKS: []
}))
vi.mock('@/utils/refUtil', () => ({
useMinLoadingDurationRef: (source: { value: boolean }) => source
}))
const mockGetRunner = vi.fn()
vi.mock('@/stores/maintenanceTaskStore', () => ({
useMaintenanceTaskStore: vi.fn(() => ({
getRunner: mockGetRunner
}))
}))
import type { MaintenanceTask } from '@/types/desktop/maintenanceTypes'
import TaskCard from '@/components/maintenance/TaskCard.vue'
const baseTask: MaintenanceTask = {
id: 'testTask',
name: 'Test Task',
shortDescription: 'Short description',
errorDescription: 'Error occurred',
execute: vi.fn().mockResolvedValue(true)
}
const cardStubs = {
Card: {
template: '<div data-testid="card"><slot name="content"></slot></div>'
},
Button: { template: '<button />' }
}
function renderCard(
state: 'OK' | 'error' | 'warning' | 'skipped',
task: MaintenanceTask = baseTask
) {
mockGetRunner.mockReturnValue({
state,
executing: false,
refreshing: false,
resolved: false
})
return render(TaskCard, {
props: { task },
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: cardStubs
}
})
}
describe('TaskCard', () => {
describe('description computed', () => {
it('shows errorDescription when task state is error', () => {
renderCard('error')
expect(screen.getByText('Error occurred')).toBeDefined()
})
it('shows shortDescription when task state is OK', () => {
renderCard('OK')
expect(screen.getByText('Short description')).toBeDefined()
})
it('shows shortDescription when task state is warning', () => {
renderCard('warning')
expect(screen.getByText('Short description')).toBeDefined()
})
it('falls back to shortDescription when errorDescription is absent and state is error', () => {
const taskWithoutErrorDesc: MaintenanceTask = {
...baseTask,
errorDescription: undefined
}
renderCard('error', taskWithoutErrorDesc)
expect(screen.getByText('Short description')).toBeDefined()
})
})
})

View File

@@ -64,7 +64,7 @@ import { computed } from 'vue'
import { useMaintenanceTaskStore } from '@/stores/maintenanceTaskStore'
import type { MaintenanceTask } from '@/types/desktop/maintenanceTypes'
import { cn } from '@comfyorg/tailwind-utils'
import { cn } from '@/utils/tailwindUtil'
import { useMinLoadingDurationRef } from '@/utils/refUtil'
const taskStore = useMaintenanceTaskStore()

View File

@@ -1,97 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi } from 'vitest'
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => ({
Validation: { validateInstallation: vi.fn() }
}))
}))
vi.mock('@/constants/desktopMaintenanceTasks', () => ({
DESKTOP_MAINTENANCE_TASKS: []
}))
vi.mock('@/utils/refUtil', () => ({
useMinLoadingDurationRef: (source: { value: boolean }) => source
}))
vi.mock('@/i18n', () => ({
t: (key: string) => key
}))
const mockGetRunner = vi.fn()
vi.mock('@/stores/maintenanceTaskStore', () => ({
useMaintenanceTaskStore: vi.fn(() => ({
getRunner: mockGetRunner
}))
}))
import type { MaintenanceTask } from '@/types/desktop/maintenanceTypes'
import TaskListItem from '@/components/maintenance/TaskListItem.vue'
const baseTask: MaintenanceTask = {
id: 'testTask',
name: 'Test Task',
button: { text: 'Fix', icon: 'pi pi-check' },
execute: vi.fn().mockResolvedValue(true)
}
const ButtonStub = {
props: ['severity', 'label', 'icon', 'loading'],
template:
'<button :data-severity="severity" :data-label="label" :data-testid="label ? \'action-button\' : \'icon-button\'" />'
}
function renderItem(state: 'OK' | 'error' | 'warning' | 'skipped') {
mockGetRunner.mockReturnValue({
state,
executing: false,
refreshing: false,
resolved: false
})
return render(TaskListItem, {
props: { task: baseTask },
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
Button: ButtonStub,
Popover: { template: '<div />' },
TaskListStatusIcon: { template: '<span />' }
}
}
})
}
describe('TaskListItem', () => {
describe('severity computed', () => {
it('uses primary severity for error state', () => {
renderItem('error')
expect(screen.getByTestId('action-button').dataset.severity).toBe(
'primary'
)
})
it('uses primary severity for warning state', () => {
renderItem('warning')
expect(screen.getByTestId('action-button').dataset.severity).toBe(
'primary'
)
})
it('uses secondary severity for OK state', () => {
renderItem('OK')
expect(screen.getByTestId('action-button').dataset.severity).toBe(
'secondary'
)
})
it('uses secondary severity for skipped state', () => {
renderItem('skipped')
expect(screen.getByTestId('action-button').dataset.severity).toBe(
'secondary'
)
})
})
})

View File

@@ -76,7 +76,7 @@ const executeTask = async (task: MaintenanceTask) => {
message = t('maintenance.error.taskFailed')
} catch (error) {
message = error instanceof Error ? error.message : undefined
message = (error as Error)?.message
}
toast.add({

View File

@@ -1,44 +0,0 @@
import { render, screen } from '@testing-library/vue'
import PrimeVue from 'primevue/config'
import { describe, expect, it, vi } from 'vitest'
vi.mock('@/i18n', () => ({
t: (key: string) => key
}))
import TaskListStatusIcon from '@/components/maintenance/TaskListStatusIcon.vue'
type TaskState = 'warning' | 'error' | 'resolved' | 'OK' | 'skipped' | undefined
function renderIcon(state: TaskState, loading?: boolean) {
return render(TaskListStatusIcon, {
props: { state, loading },
global: {
plugins: [[PrimeVue, { unstyled: true }]],
stubs: {
ProgressSpinner: {
template: '<div data-testid="spinner" />'
}
}
}
})
}
describe('TaskListStatusIcon', () => {
describe('loading / no state', () => {
it('renders spinner when state is undefined', () => {
renderIcon(undefined)
expect(screen.getByTestId('spinner')).toBeDefined()
})
it('renders spinner when loading is true', () => {
renderIcon('OK', true)
expect(screen.getByTestId('spinner')).toBeDefined()
})
it('hides spinner when state is defined and not loading', () => {
renderIcon('OK', false)
expect(screen.queryByTestId('spinner')).toBeNull()
})
})
})

View File

@@ -1,124 +0,0 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { ref } from 'vue'
const { mockTerminal, MockTerminal, mockFitAddon, MockFitAddon } = vi.hoisted(
() => {
const mockTerminal = {
loadAddon: vi.fn(),
attachCustomKeyEventHandler: vi.fn(),
open: vi.fn(),
dispose: vi.fn(),
hasSelection: vi.fn<[], boolean>(),
resize: vi.fn(),
cols: 80,
rows: 24
}
const MockTerminal = vi.fn(function () {
return mockTerminal
})
const mockFitAddon = {
proposeDimensions: vi.fn().mockReturnValue({ cols: 80, rows: 24 })
}
const MockFitAddon = vi.fn(function () {
return mockFitAddon
})
return { mockTerminal, MockTerminal, mockFitAddon, MockFitAddon }
}
)
vi.mock('@xterm/xterm', () => ({ Terminal: MockTerminal }))
vi.mock('@xterm/addon-fit', () => ({ FitAddon: MockFitAddon }))
vi.mock('@xterm/xterm/css/xterm.css', () => ({}))
import { withSetup } from '@/test/withSetup'
import { useTerminal } from '@/composables/bottomPanelTabs/useTerminal'
function getKeyHandler(): (event: KeyboardEvent) => boolean {
return mockTerminal.attachCustomKeyEventHandler.mock.calls[0][0]
}
describe('useTerminal key event handler', () => {
beforeEach(() => {
vi.clearAllMocks()
mockTerminal.hasSelection.mockReturnValue(false)
const element = ref<HTMLElement | undefined>(undefined)
withSetup(() => useTerminal(element))
})
it('allows browser to handle copy when text is selected (Ctrl+C)', () => {
mockTerminal.hasSelection.mockReturnValue(true)
const event = {
type: 'keydown',
ctrlKey: true,
metaKey: false,
key: 'c'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(false)
})
it('allows browser to handle copy when text is selected (Meta+C)', () => {
mockTerminal.hasSelection.mockReturnValue(true)
const event = {
type: 'keydown',
ctrlKey: false,
metaKey: true,
key: 'c'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(false)
})
it('does not pass copy to browser when no text is selected', () => {
mockTerminal.hasSelection.mockReturnValue(false)
const event = {
type: 'keydown',
ctrlKey: true,
metaKey: false,
key: 'c'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(true)
})
it('allows browser to handle paste (Ctrl+V)', () => {
const event = {
type: 'keydown',
ctrlKey: true,
metaKey: false,
key: 'v'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(false)
})
it('allows browser to handle paste (Meta+V)', () => {
const event = {
type: 'keydown',
ctrlKey: false,
metaKey: true,
key: 'v'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(false)
})
it('does not intercept non-keydown events', () => {
mockTerminal.hasSelection.mockReturnValue(true)
const event = {
type: 'keyup',
ctrlKey: true,
metaKey: false,
key: 'c'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(true)
})
it('passes through unrelated key combinations', () => {
const event = {
type: 'keydown',
ctrlKey: false,
metaKey: false,
key: 'Enter'
} as KeyboardEvent
expect(getKeyHandler()(event)).toBe(true)
})
})

View File

@@ -1,48 +0,0 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockSerialize, MockSerializeAddon } = vi.hoisted(() => {
const mockSerialize = vi.fn<[], string>()
const MockSerializeAddon = vi.fn(function () {
return { serialize: mockSerialize }
})
return { mockSerialize, MockSerializeAddon }
})
vi.mock('@xterm/xterm', () => ({
Terminal: vi.fn(function () {
return { loadAddon: vi.fn(), dispose: vi.fn(), write: vi.fn() }
})
}))
vi.mock('@xterm/addon-serialize', () => ({
SerializeAddon: MockSerializeAddon
}))
import type { Terminal } from '@xterm/xterm'
import { withSetup } from '@/test/withSetup'
import { useTerminalBuffer } from '@/composables/bottomPanelTabs/useTerminalBuffer'
describe('useTerminalBuffer', () => {
beforeEach(() => {
vi.clearAllMocks()
mockSerialize.mockReturnValue('')
})
describe('copyTo', () => {
it('writes serialized buffer content to the destination terminal', () => {
mockSerialize.mockReturnValue('hello world')
const { copyTo } = withSetup(() => useTerminalBuffer())
const mockWrite = vi.fn()
copyTo({ write: mockWrite } as Pick<Terminal, 'write'>)
expect(mockWrite).toHaveBeenCalledWith('hello world')
})
it('writes empty string when buffer is empty', () => {
mockSerialize.mockReturnValue('')
const { copyTo } = withSetup(() => useTerminalBuffer())
const mockWrite = vi.fn()
copyTo({ write: mockWrite } as Pick<Terminal, 'write'>)
expect(mockWrite).toHaveBeenCalledWith('')
})
})
})

View File

@@ -1,50 +0,0 @@
import { describe, expect, it } from 'vitest'
import { DESKTOP_DIALOGS, getDialog } from '@/constants/desktopDialogs'
describe('getDialog', () => {
it('returns the matching dialog for a valid ID', () => {
const result = getDialog('reinstallVenv')
expect(result.id).toBe('reinstallVenv')
expect(result.title).toBe(DESKTOP_DIALOGS.reinstallVenv.title)
expect(result.message).toBe(DESKTOP_DIALOGS.reinstallVenv.message)
})
it('returns invalidDialog for an unknown string ID', () => {
const result = getDialog('unknownDialog')
expect(result.id).toBe('invalidDialog')
})
it('returns invalidDialog when given an array of strings', () => {
const result = getDialog(['reinstallVenv', 'other'])
expect(result.id).toBe('invalidDialog')
})
it('returns invalidDialog for empty string', () => {
const result = getDialog('')
expect(result.id).toBe('invalidDialog')
})
it('returns a deep clone — mutations do not affect the original', () => {
const result = getDialog('reinstallVenv')
const originalFirstLabel = DESKTOP_DIALOGS.reinstallVenv.buttons[0].label
result.buttons[0].label = 'Mutated'
expect(DESKTOP_DIALOGS.reinstallVenv.buttons[0].label).toBe(
originalFirstLabel
)
})
it('every button has a returnValue', () => {
for (const id of Object.keys(DESKTOP_DIALOGS)) {
const result = getDialog(id)
for (const button of result.buttons) {
expect(button.returnValue).toBeDefined()
}
}
})
it('invalidDialog has a close/cancel button', () => {
const result = getDialog('invalidDialog')
expect(result.buttons.some((b) => b.action === 'cancel')).toBe(true)
})
})

View File

@@ -1,75 +0,0 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockElectron } = vi.hoisted(() => ({
mockElectron: {
setBasePath: vi.fn(),
reinstall: vi.fn<[], Promise<void>>().mockResolvedValue(undefined),
uv: {
installRequirements: vi.fn<[], Promise<void>>(),
clearCache: vi.fn<[], Promise<void>>().mockResolvedValue(undefined),
resetVenv: vi.fn<[], Promise<void>>().mockResolvedValue(undefined)
}
}
}))
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => mockElectron)
}))
import { DESKTOP_MAINTENANCE_TASKS } from '@/constants/desktopMaintenanceTasks'
function findTask(id: string) {
const task = DESKTOP_MAINTENANCE_TASKS.find((t) => t.id === id)
if (!task) throw new Error(`Task not found: ${id}`)
return task
}
describe('desktopMaintenanceTasks', () => {
beforeEach(() => {
vi.resetAllMocks()
vi.spyOn(window, 'open').mockReturnValue(null)
mockElectron.reinstall.mockResolvedValue(undefined)
mockElectron.uv.clearCache.mockResolvedValue(undefined)
mockElectron.uv.resetVenv.mockResolvedValue(undefined)
})
describe('pythonPackages', () => {
it('returns true when installation succeeds', async () => {
mockElectron.uv.installRequirements.mockResolvedValue(undefined)
expect(await findTask('pythonPackages').execute()).toBe(true)
})
it('returns false when installation throws', async () => {
mockElectron.uv.installRequirements.mockRejectedValue(
new Error('install failed')
)
expect(await findTask('pythonPackages').execute()).toBe(false)
})
})
describe('URL-opening tasks', () => {
it('git execute opens the git download page', () => {
findTask('git').execute()
expect(window.open).toHaveBeenCalledWith(
'https://git-scm.com/downloads/',
'_blank'
)
})
it('uv execute opens the uv installation page', () => {
findTask('uv').execute()
expect(window.open).toHaveBeenCalledWith(
'https://docs.astral.sh/uv/getting-started/installation/',
'_blank'
)
})
it('vcRedist execute opens the VC++ redistributable download', () => {
findTask('vcRedist').execute()
expect(window.open).toHaveBeenCalledWith(
'https://aka.ms/vs/17/release/vc_redist.x64.exe',
'_blank'
)
})
})
})

View File

@@ -9,7 +9,6 @@ import en from '@frontend-locales/en/main.json' with { type: 'json' }
import enNodes from '@frontend-locales/en/nodeDefs.json' with { type: 'json' }
import enSettings from '@frontend-locales/en/settings.json' with { type: 'json' }
import { getDefaultLocale } from '@frontend-locales/localeConfig'
import { createI18n } from 'vue-i18n'
function buildLocale<
@@ -168,7 +167,7 @@ const messages: Record<string, LocaleMessages> = {
export const i18n = createI18n({
// Must set `false`, as Vue I18n Legacy API is for Vue 2
legacy: false,
locale: getDefaultLocale(),
locale: navigator.language.split('-')[0] || 'en',
fallbackLocale: 'en',
messages,
// Ignore warnings for locale options as each option is in its own language.

View File

@@ -1,288 +0,0 @@
import { createTestingPinia } from '@pinia/testing'
import type { InstallValidation } from '@comfyorg/comfyui-electron-types'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockElectron, testTasks } = vi.hoisted(() => {
const terminalTaskExecute = vi.fn().mockResolvedValue(true)
const basicTaskExecute = vi.fn().mockResolvedValue(true)
return {
mockElectron: {
Validation: {
validateInstallation: vi.fn()
}
},
testTasks: [
{
id: 'basicTask',
name: 'Basic Task',
execute: basicTaskExecute
},
{
id: 'terminalTask',
name: 'Terminal Task',
execute: terminalTaskExecute,
usesTerminal: true,
isInstallationFix: true
}
]
}
})
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => mockElectron)
}))
vi.mock('@/constants/desktopMaintenanceTasks', () => ({
DESKTOP_MAINTENANCE_TASKS: testTasks
}))
import { useMaintenanceTaskStore } from '@/stores/maintenanceTaskStore'
import type { MaintenanceTask } from '@/types/desktop/maintenanceTypes'
type PartialInstallValidation = Partial<InstallValidation> &
Record<string, unknown>
function makeUpdate(
overrides: PartialInstallValidation = {}
): InstallValidation {
return {
inProgress: false,
installState: 'installed',
...overrides
} as InstallValidation
}
function createStore() {
setActivePinia(createTestingPinia({ stubActions: false }))
return useMaintenanceTaskStore()
}
describe('useMaintenanceTaskStore', () => {
let store: ReturnType<typeof useMaintenanceTaskStore>
const [basicTask, terminalTask] = testTasks as MaintenanceTask[]
beforeEach(() => {
vi.resetAllMocks()
store = createStore()
})
describe('processUpdate', () => {
it('sets isRefreshing to true during in-progress update', () => {
store.processUpdate(makeUpdate({ inProgress: true }))
expect(store.isRefreshing).toBe(true)
})
it('sets isRefreshing to false when update is complete', () => {
store.processUpdate(makeUpdate({ inProgress: false, basicTask: 'OK' }))
expect(store.isRefreshing).toBe(false)
})
it('updates runner state for tasks present in the final update', () => {
store.processUpdate(makeUpdate({ basicTask: 'error' }))
expect(store.getRunner(basicTask).state).toBe('error')
})
it('sets task state to warning from update', () => {
store.processUpdate(makeUpdate({ basicTask: 'warning' }))
expect(store.getRunner(basicTask).state).toBe('warning')
})
it('marks runners as refreshing when task id is absent from in-progress update', () => {
store.processUpdate(makeUpdate({ inProgress: true }))
expect(store.getRunner(basicTask).refreshing).toBe(true)
})
it('marks task as skipped when absent from final update', () => {
store.processUpdate(makeUpdate({ inProgress: false }))
expect(store.getRunner(basicTask).state).toBe('skipped')
})
it('clears refreshing flag after final update', () => {
store.processUpdate(makeUpdate({ inProgress: true }))
store.processUpdate(makeUpdate({ inProgress: false }))
expect(store.getRunner(basicTask).refreshing).toBe(false)
})
it('stores lastUpdate and exposes unsafeBasePath', () => {
store.processUpdate(makeUpdate({ unsafeBasePath: true }))
expect(store.unsafeBasePath).toBe(true)
})
it('exposes unsafeBasePathReason from the update', () => {
store.processUpdate(
makeUpdate({ unsafeBasePath: true, unsafeBasePathReason: 'oneDrive' })
)
expect(store.unsafeBasePathReason).toBe('oneDrive')
})
})
describe('anyErrors', () => {
it('returns true when any task has error state', () => {
store.processUpdate(makeUpdate({ basicTask: 'error' }))
expect(store.anyErrors).toBe(true)
})
it('returns false when all tasks are OK', () => {
store.processUpdate(makeUpdate({ basicTask: 'OK', terminalTask: 'OK' }))
expect(store.anyErrors).toBe(false)
})
it('returns false when all tasks are warning', () => {
store.processUpdate(
makeUpdate({ basicTask: 'warning', terminalTask: 'warning' })
)
expect(store.anyErrors).toBe(false)
})
})
describe('runner state transitions', () => {
it('marks runner as resolved when transitioning from error to OK', () => {
store.processUpdate(makeUpdate({ basicTask: 'error' }))
store.processUpdate(makeUpdate({ basicTask: 'OK' }))
expect(store.getRunner(basicTask).resolved).toBe(true)
})
it('does not mark resolved for warning to OK transition', () => {
store.processUpdate(makeUpdate({ basicTask: 'warning' }))
store.processUpdate(makeUpdate({ basicTask: 'OK' }))
expect(store.getRunner(basicTask).resolved).toBeFalsy()
})
it('clears resolved flag when task returns to error', () => {
store.processUpdate(makeUpdate({ basicTask: 'error' }))
store.processUpdate(makeUpdate({ basicTask: 'OK' }))
store.processUpdate(makeUpdate({ basicTask: 'error' }))
expect(store.getRunner(basicTask).resolved).toBeFalsy()
})
})
describe('clearResolved', () => {
it('clears resolved flags on all runners', () => {
store.processUpdate(makeUpdate({ basicTask: 'error' }))
store.processUpdate(makeUpdate({ basicTask: 'OK' }))
expect(store.getRunner(basicTask).resolved).toBe(true)
store.clearResolved()
expect(store.getRunner(basicTask).resolved).toBeFalsy()
})
})
describe('execute', () => {
it('returns true when task execution succeeds', async () => {
vi.mocked(basicTask.execute).mockResolvedValue(true)
const result = await store.execute(basicTask)
expect(result).toBe(true)
})
it('returns false when task execution fails', async () => {
vi.mocked(basicTask.execute).mockResolvedValue(false)
const result = await store.execute(basicTask)
expect(result).toBe(false)
})
it('calls refreshDesktopTasks after successful installation-fix task', async () => {
vi.mocked(terminalTask.execute).mockResolvedValue(true)
await store.execute(terminalTask)
expect(
mockElectron.Validation.validateInstallation
).toHaveBeenCalledOnce()
})
it('does not call refreshDesktopTasks when task is not an installation fix', async () => {
vi.mocked(basicTask.execute).mockResolvedValue(true)
await store.execute(basicTask)
expect(
mockElectron.Validation.validateInstallation
).not.toHaveBeenCalled()
})
it('does not call refreshDesktopTasks when installation-fix task fails', async () => {
vi.mocked(terminalTask.execute).mockResolvedValue(false)
await store.execute(terminalTask)
expect(
mockElectron.Validation.validateInstallation
).not.toHaveBeenCalled()
})
it('sets runner executing to true during task execution', async () => {
let resolveTask!: (value: boolean) => void
vi.mocked(basicTask.execute).mockReturnValue(
new Promise<boolean>((resolve) => {
resolveTask = resolve
})
)
const executePromise = store.execute(basicTask)
expect(store.getRunner(basicTask).executing).toBe(true)
resolveTask(true)
await executePromise
expect(store.getRunner(basicTask).executing).toBe(false)
})
it('clears executing flag when task throws', async () => {
vi.mocked(basicTask.execute).mockRejectedValue(new Error('fail'))
await expect(store.execute(basicTask)).rejects.toThrow('fail')
expect(store.getRunner(basicTask).executing).toBe(false)
})
it('sets runner error message when task throws', async () => {
vi.mocked(basicTask.execute).mockRejectedValue(
new Error('something broke')
)
await expect(store.execute(basicTask)).rejects.toThrow()
expect(store.getRunner(basicTask).error).toBe('something broke')
})
it('clears runner error on successful execution', async () => {
vi.mocked(basicTask.execute).mockRejectedValue(new Error('fail'))
await expect(store.execute(basicTask)).rejects.toThrow()
vi.mocked(basicTask.execute).mockResolvedValue(true)
await store.execute(basicTask)
expect(store.getRunner(basicTask).error).toBeUndefined()
})
})
describe('isRunningTerminalCommand', () => {
it('returns true while a terminal task is executing', async () => {
let resolveTask!: (value: boolean) => void
vi.mocked(terminalTask.execute).mockReturnValue(
new Promise<boolean>((resolve) => {
resolveTask = resolve
})
)
const executePromise = store.execute(terminalTask)
expect(store.isRunningTerminalCommand).toBe(true)
resolveTask(true)
await executePromise
expect(store.isRunningTerminalCommand).toBe(false)
})
it('returns false when no terminal tasks are executing', () => {
expect(store.isRunningTerminalCommand).toBe(false)
})
})
describe('isRunningInstallationFix', () => {
it('returns true while an installation-fix task is executing', async () => {
let resolveTask!: (value: boolean) => void
vi.mocked(terminalTask.execute).mockReturnValue(
new Promise<boolean>((resolve) => {
resolveTask = resolve
})
)
const executePromise = store.execute(terminalTask)
expect(store.isRunningInstallationFix).toBe(true)
resolveTask(true)
await executePromise
expect(store.isRunningInstallationFix).toBe(false)
})
})
})

View File

@@ -66,7 +66,7 @@ class MaintenanceTaskRunner {
this.error = undefined
return true
} catch (error) {
this.error = error instanceof Error ? error.message : String(error)
this.error = (error as Error)?.message
throw error
} finally {
this.executing = false

View File

@@ -1 +0,0 @@
import '@testing-library/jest-dom/vitest'

View File

@@ -1,16 +0,0 @@
import { render } from '@testing-library/vue'
import { defineComponent } from 'vue'
export function withSetup<T>(composable: () => T): T {
let result!: T
render(
defineComponent({
setup() {
result = composable()
return {}
},
template: '<div />'
})
)
return result
}

View File

@@ -1,52 +0,0 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
const { mockElectron } = vi.hoisted(() => ({
mockElectron: {
NetWork: {
canAccessUrl: vi.fn<[url: string], Promise<boolean>>()
}
}
}))
vi.mock('@/utils/envUtil', () => ({
electronAPI: vi.fn(() => mockElectron)
}))
import { checkMirrorReachable } from '@/utils/electronMirrorCheck'
describe('checkMirrorReachable', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('returns false for an invalid URL without calling canAccessUrl', async () => {
const result = await checkMirrorReachable('not-a-url')
expect(result).toBe(false)
expect(mockElectron.NetWork.canAccessUrl).not.toHaveBeenCalled()
})
it('returns false when canAccessUrl returns false', async () => {
mockElectron.NetWork.canAccessUrl.mockResolvedValue(false)
const result = await checkMirrorReachable('https://example.com')
expect(result).toBe(false)
})
it('returns true when URL is valid and canAccessUrl returns true', async () => {
mockElectron.NetWork.canAccessUrl.mockResolvedValue(true)
const result = await checkMirrorReachable('https://example.com')
expect(result).toBe(true)
})
it('passes the mirror URL to canAccessUrl', async () => {
const url = 'https://pypi.org/simple/'
mockElectron.NetWork.canAccessUrl.mockResolvedValue(true)
await checkMirrorReachable(url)
expect(mockElectron.NetWork.canAccessUrl).toHaveBeenCalledWith(url)
})
it('returns false for empty string', async () => {
const result = await checkMirrorReachable('')
expect(result).toBe(false)
expect(mockElectron.NetWork.canAccessUrl).not.toHaveBeenCalled()
})
})

View File

@@ -1,72 +0,0 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { isElectron, isNativeWindow } from '@/utils/envUtil'
describe('isElectron', () => {
afterEach(() => {
vi.unstubAllGlobals()
})
it('returns true when window.electronAPI is an object', () => {
vi.stubGlobal('window', { ...window, electronAPI: {} })
expect(isElectron()).toBe(true)
})
it('returns false when window.electronAPI is undefined', () => {
vi.stubGlobal('window', { ...window, electronAPI: undefined })
expect(isElectron()).toBe(false)
})
it('returns false when window.electronAPI is absent', () => {
const copy = { ...window } as Record<string, unknown>
delete copy['electronAPI']
vi.stubGlobal('window', copy)
expect(isElectron()).toBe(false)
})
})
describe('isNativeWindow', () => {
afterEach(() => {
vi.unstubAllGlobals()
})
it('returns true when Electron and windowControlsOverlay.visible is true', () => {
vi.stubGlobal('window', {
...window,
electronAPI: {},
navigator: {
...window.navigator,
windowControlsOverlay: { visible: true }
}
})
expect(isNativeWindow()).toBe(true)
})
it('returns false when not in Electron', () => {
const copy = { ...window } as Record<string, unknown>
delete copy['electronAPI']
vi.stubGlobal('window', copy)
expect(isNativeWindow()).toBe(false)
})
it('returns false when windowControlsOverlay.visible is false', () => {
vi.stubGlobal('window', {
...window,
electronAPI: {},
navigator: {
...window.navigator,
windowControlsOverlay: { visible: false }
}
})
expect(isNativeWindow()).toBe(false)
})
it('returns false when windowControlsOverlay is absent', () => {
vi.stubGlobal('window', {
...window,
electronAPI: {},
navigator: { ...window.navigator, windowControlsOverlay: undefined }
})
expect(isNativeWindow()).toBe(false)
})
})

View File

@@ -1,102 +0,0 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { computed, nextTick, ref } from 'vue'
import { withSetup } from '@/test/withSetup'
import { useMinLoadingDurationRef } from '@/utils/refUtil'
describe('useMinLoadingDurationRef', () => {
beforeEach(() => {
vi.useFakeTimers()
})
afterEach(() => {
vi.useRealTimers()
})
it('reflects false when source is initially false', () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source))
expect(result.value).toBe(false)
})
it('reflects true when source is initially true', () => {
const source = ref(true)
const result = withSetup(() => useMinLoadingDurationRef(source))
expect(result.value).toBe(true)
})
it('becomes true immediately when source transitions to true', async () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source))
source.value = true
await nextTick()
expect(result.value).toBe(true)
})
it('stays true within minDuration after source returns to false', async () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source, 250))
source.value = true
await nextTick()
source.value = false
await nextTick()
vi.advanceTimersByTime(100)
await nextTick()
expect(result.value).toBe(true)
})
it('becomes false after minDuration has elapsed', async () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source, 250))
source.value = true
await nextTick()
source.value = false
await nextTick()
vi.advanceTimersByTime(250)
await nextTick()
expect(result.value).toBe(false)
})
it('remains true while source is true even after minDuration elapses', async () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source, 250))
source.value = true
await nextTick()
vi.advanceTimersByTime(500)
await nextTick()
expect(result.value).toBe(true)
})
it('works with a computed ref as input', async () => {
const raw = ref(false)
const source = computed(() => raw.value)
const result = withSetup(() => useMinLoadingDurationRef(source))
raw.value = true
await nextTick()
expect(result.value).toBe(true)
})
it('uses 250ms as default minDuration', async () => {
const source = ref(false)
const result = withSetup(() => useMinLoadingDurationRef(source))
source.value = true
await nextTick()
source.value = false
await nextTick()
vi.advanceTimersByTime(249)
await nextTick()
expect(result.value).toBe(true)
vi.advanceTimersByTime(1)
await nextTick()
expect(result.value).toBe(false)
})
})

View File

@@ -0,0 +1 @@
export { cn } from '@comfyorg/tailwind-utils'

View File

@@ -13,8 +13,7 @@
"src/**/*.ts",
"src/**/*.vue",
"src/**/*.d.ts",
"vite.config.mts",
"vitest.config.mts"
"vite.config.mts"
],
"references": []
}

View File

@@ -1,22 +0,0 @@
import vue from '@vitejs/plugin-vue'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import { defineConfig } from 'vitest/config'
const projectRoot = fileURLToPath(new URL('.', import.meta.url))
export default defineConfig({
plugins: [vue()],
resolve: {
alias: {
'@': path.resolve(projectRoot, 'src'),
'@frontend-locales': path.resolve(projectRoot, '../../src/locales')
}
},
test: {
globals: true,
environment: 'happy-dom',
include: ['src/**/*.{test,spec}.{ts,tsx}'],
setupFiles: ['./src/test/setup.ts']
}
})

View File

@@ -1,8 +1,2 @@
dist/
.astro/
test-results/
playwright-report/
results.json
# Platform-specific Playwright snapshots (CI runs Linux)
*-win32.png

View File

@@ -1,155 +0,0 @@
# @comfyorg/website
Marketing/brand website built with Astro + Vue.
## Ashby careers integration
`/careers` and `/zh-CN/careers` are rendered from Ashby's public job board
API at build time. Data flow:
1. `src/pages/careers.astro` awaits `fetchRolesForBuild()` during the
Astro build.
2. `src/utils/ashby.ts` calls
`GET https://api.ashbyhq.com/posting-api/job-board/{board}?includeCompensation=false`,
validates the envelope and each posting with Zod
(`src/utils/ashby.schema.ts`), and maps to the domain type in
`src/data/roles.ts`.
3. On any failure (network, HTTP 4xx/5xx, envelope schema drift),
the fetcher falls back to the committed JSON snapshot at
`src/data/ashby-roles.snapshot.json`.
4. `src/utils/ashby.ci.ts` emits GitHub Actions annotations and a
`$GITHUB_STEP_SUMMARY` block so stale fetches are visible on green
builds.
### Required environment variables
Both are build-time only. Never prefix with `PUBLIC_` (Astro would
inline that into the client bundle).
| Name | Purpose | Default (when unset) |
| ------------------------------ | --------------------------- | --------------------------------- |
| `WEBSITE_ASHBY_API_KEY` | Ashby API key (Basic auth) | Build uses the committed snapshot |
| `WEBSITE_ASHBY_JOB_BOARD_NAME` | Ashby public job board slug | Build uses the committed snapshot |
### CI wiring (manual step — required)
This repo's `.github/workflows/*.yaml` changes cannot be pushed by a
GitHub App. A maintainer must apply the following edits **once**:
**`.github/workflows/ci-website-build.yaml`** — pass the env into the
build step and run the unit tests before it:
```yaml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Run website unit tests
run: pnpm --filter @comfyorg/website test:unit
- name: Build website
env:
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ vars.WEBSITE_ASHBY_JOB_BOARD_NAME || 'comfy-org' }}
run: pnpm --filter @comfyorg/website build
- name: Verify API key is not leaked into build output
env:
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
run: |
set +x
if [ -z "${WEBSITE_ASHBY_API_KEY:-}" ]; then
echo "Secret not available in this run; skipping leak check."
exit 0
fi
# grep -rlF prints only file paths (never match content).
MATCHES=$(grep -rlF --exclude-dir=node_modules --null \
-e "$WEBSITE_ASHBY_API_KEY" apps/website/dist/ 2>/dev/null \
| tr '\0' '\n' || true)
if [ -n "$MATCHES" ]; then
echo "::error title=Ashby API key leaked into build output::$MATCHES"
exit 1
fi
```
**`.github/workflows/ci-vercel-website-preview.yaml`** — add the
two env vars to the top-level `env:` block so `vercel build` (both
`deploy-preview` and `deploy-production` jobs) sees them:
```yaml
env:
VERCEL_ORG_ID: ${{ secrets.VERCEL_WEBSITE_ORG_ID }}
VERCEL_PROJECT_ID: ${{ secrets.VERCEL_WEBSITE_PROJECT_ID }}
VERCEL_TOKEN: ${{ secrets.VERCEL_WEBSITE_TOKEN }}
VERCEL_SCOPE: comfyui
WEBSITE_ASHBY_API_KEY: ${{ secrets.WEBSITE_ASHBY_API_KEY }}
WEBSITE_ASHBY_JOB_BOARD_NAME: ${{ vars.WEBSITE_ASHBY_JOB_BOARD_NAME || 'comfy-org' }}
```
The secret must also be added to the Vercel project environment
(`vercel env add WEBSITE_ASHBY_API_KEY …` or via the Vercel UI) so
that `vercel build` in the preview job has access to it.
Fork PRs do not exercise this path: `ci-vercel-website-preview.yaml`
receives an empty `VERCEL_TOKEN` for forks and fails at `vercel pull`
before the build runs. Fork-safe PR interactions (the preview-URL
comment) are handled by `pr-vercel-website-preview.yaml`.
### Refreshing the snapshot
When a maintainer wants to update the committed snapshot (e.g. after
onboarding/offboarding roles):
```bash
WEBSITE_ASHBY_API_KEY=WEBSITE_ASHBY_JOB_BOARD_NAME=comfy-org \
pnpm --filter @comfyorg/website ashby:refresh-snapshot
git commit apps/website/src/data/ashby-roles.snapshot.json
```
The script exits non-zero on any non-fresh outcome so stale/empty
snapshots can't be accidentally committed.
## Cloud nodes integration
`/cloud/supported-nodes` (and `/zh-CN/`) lists custom-node packs preinstalled on Comfy Cloud, joined with public metadata from the [ComfyUI Custom Node Registry](https://registry.comfy.org) ([`api.comfy.org`](https://api.comfy.org)). See [`src/pages/cloud/supported-nodes/AGENTS.md`](src/pages/cloud/supported-nodes/AGENTS.md) for the build pipeline, source-file map, and key invariants.
Build-time env var: `WEBSITE_CLOUD_API_KEY` (Cloud `/api/object_info` auth; the build falls back to the committed snapshot when unset). Must also be set in the Vercel project environment.
## HubSpot contact form
The contact page uses HubSpot's hosted form embed for the interest form:
```html
<script
src="https://js-na2.hsforms.net/forms/embed/developer/244637579.js"
defer
></script>
<div
class="hs-form-html"
data-region="na2"
data-form-id="94e05eab-1373-47f7-ab5e-d84f9e6aa262"
data-portal-id="244637579"
></div>
```
The localized `/zh-CN/contact` page uses the same portal and script with form
ID `6885750c-02ef-4aa2-ba0d-213be9cccf93`.
This keeps submission handling, validation, anti-spam updates, and field
configuration in HubSpot. The local implementation in
`src/components/contact/HubspotFormEmbed.vue` only loads the hosted script and
renders the documented embed container.
## Scripts
- `pnpm dev` — Astro dev server
- `pnpm build` — production build to `dist/`
- `pnpm typecheck``astro check`
- `pnpm test:unit` — Vitest unit tests
- `pnpm test:e2e` — Playwright E2E tests (requires `pnpm build` first)
- `pnpm ashby:refresh-snapshot` — refresh the committed careers snapshot
- `pnpm cloud-nodes:refresh-snapshot` — refresh the committed cloud nodes snapshot

View File

@@ -1,56 +1,22 @@
import { defineConfig } from 'astro/config'
import sitemap from '@astrojs/sitemap'
import vue from '@astrojs/vue'
import tailwindcss from '@tailwindcss/vite'
const LOCALES = ['en', 'zh-CN'] as const
const DEFAULT_LOCALE = 'en'
const PAYMENT_STATUSES = ['success', 'failed'] as const
const LOCALE_PREFIXES = LOCALES.map((locale) =>
locale === DEFAULT_LOCALE ? '' : `/${locale}`
)
const SITEMAP_EXCLUDED_PATHNAMES = new Set(
LOCALE_PREFIXES.flatMap((prefix) =>
PAYMENT_STATUSES.map((status) => `${prefix}/payment/${status}`)
)
)
function isExcludedFromSitemap(page: string): boolean {
const pathname = new URL(page).pathname.replace(/\/$/, '')
return SITEMAP_EXCLUDED_PATHNAMES.has(pathname)
}
export default defineConfig({
site: 'https://comfy.org',
output: 'static',
prefetch: { prefetchAll: true },
redirects: {
'/cloud/enterprise-case-studies/comfyui-at-architectural-scale-how-moment-factory-reimagined-3d-projection-mapping':
'/customers/moment-factory/',
'/cloud/enterprise-case-studies/how-series-entertainment-rebuilt-game-and-video-production-with-comfyui':
'/customers/series-entertainment/'
integrations: [vue()],
vite: {
plugins: [tailwindcss()]
},
build: {
assets: '_website'
},
devToolbar: { enabled: !process.env.NO_TOOLBAR },
integrations: [
vue(),
sitemap({
filter: (page) => !isExcludedFromSitemap(page)
})
],
vite: {
plugins: [tailwindcss()],
server: {
watch: {
ignored: ['**/playwright-report/**']
}
}
assetsPrefix: process.env.VERCEL_URL
? `https://${process.env.VERCEL_URL}`
: undefined
},
i18n: {
locales: [...LOCALES],
defaultLocale: DEFAULT_LOCALE,
locales: ['en', 'zh-CN'],
defaultLocale: 'en',
routing: {
prefixDefaultLocale: false
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 B

View File

@@ -1,81 +0,0 @@
import { expect } from '@playwright/test'
import { test } from './fixtures/blockExternalMedia'
test.describe('Careers page @smoke', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/careers')
})
test('has correct title', async ({ page }) => {
await expect(page).toHaveTitle('Careers — Comfy')
})
test('Roles section heading is visible', async ({ page }) => {
await expect(
page.getByRole('heading', { name: 'Roles', level: 2 })
).toBeVisible()
})
test('renders at least one role from the snapshot', async ({ page }) => {
const roles = page.getByTestId('careers-role-link')
await expect(roles.first()).toBeVisible()
expect(await roles.count()).toBeGreaterThan(0)
})
test('clicking a department button scrolls to and activates that section', async ({
page
}) => {
const rolesSection = page.getByTestId('careers-roles')
await rolesSection.scrollIntoViewIfNeeded()
await expect(rolesSection).toBeVisible()
const allCount = await page.getByTestId('careers-role-link').count()
const engineeringButton = page.getByRole('button', {
name: 'ENGINEERING',
exact: true
})
// RolesSection is hydrated via `client:visible`. Once the button responds
// to a click by flipping aria-pressed, Vue is hydrated and the rest of
// the locator logic is in effect.
await expect(async () => {
await engineeringButton.click()
await expect(engineeringButton).toHaveAttribute('aria-pressed', 'true', {
timeout: 1_000
})
}).toPass({ timeout: 10_000 })
const engineeringSection = page.locator('#careers-dept-engineering')
await expect(engineeringSection).toBeInViewport()
expect(await page.getByTestId('careers-role-link').count()).toBe(allCount)
})
})
test.describe('Careers page role links', () => {
test('each role links to the Ashby job description page, not the application form', async ({
page
}) => {
await page.goto('/careers')
const roles = page.getByTestId('careers-role-link')
const count = await roles.count()
for (let i = 0; i < count; i++) {
const href = await roles.nth(i).getAttribute('href')
expect(href).toMatch(/^https:\/\/jobs\.ashbyhq\.com\//)
expect(href).not.toMatch(/\/application\/?$/)
}
})
})
test.describe('Careers page (zh-CN) @smoke', () => {
test('renders localized heading and roles', async ({ page }) => {
await page.goto('/zh-CN/careers')
await expect(page).toHaveTitle('招聘 — Comfy')
await expect(
page.getByRole('heading', { name: '职位', level: 2 })
).toBeVisible()
await expect(page.getByTestId('careers-role-link').first()).toBeVisible()
})
})

View File

@@ -1,169 +0,0 @@
import { expect } from '@playwright/test'
import { test } from './fixtures/blockExternalMedia'
test.describe('Cloud nodes page @smoke', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/cloud/supported-nodes')
})
test('has correct title', async ({ page }) => {
await expect(page).toHaveTitle(
'Custom-node packs on Comfy Cloud — supported by default'
)
})
test('renders at least one pack card', async ({ page }) => {
const cards = page.getByTestId('cloud-node-pack-card')
await expect(cards.first()).toBeVisible()
expect(await cards.count()).toBeGreaterThan(0)
})
test('search input filters cards down', async ({ page }) => {
const cards = page.getByTestId('cloud-node-pack-card')
const initialCount = await cards.count()
await page.getByTestId('cloud-nodes-search').fill('impact')
const filteredCards = page.getByTestId('cloud-node-pack-card')
await expect(filteredCards.first()).toBeVisible()
const filteredCount = await filteredCards.count()
expect(filteredCount).toBeGreaterThan(0)
expect(filteredCount).toBeLessThanOrEqual(initialCount)
})
test('renders banner image or fallback element', async ({ page }) => {
const banners = page.getByTestId('cloud-node-pack-banner')
await expect(banners.first()).toBeVisible()
})
test('opens pack detail page from first card', async ({ page }) => {
const firstCard = page.getByTestId('cloud-node-pack-card').first()
await expect(firstCard).toBeVisible()
await firstCard.locator('a').first().click()
await expect(page).toHaveURL(/\/cloud\/supported-nodes\/[a-z0-9-]+$/)
await expect(page.getByTestId('cloud-node-pack-detail')).toBeVisible()
})
test('direct pack detail route renders node entries', async ({ page }) => {
await page.goto('/cloud/supported-nodes/comfyui-impact-pack')
await expect(page.getByTestId('cloud-node-pack-detail')).toBeVisible()
await expect(
page.getByTestId('cloud-node-pack-detail-node').first()
).toBeVisible()
})
test('search with no matches shows empty state', async ({ page }) => {
await page
.getByTestId('cloud-nodes-search')
.fill('zzzz-this-pack-does-not-exist')
await expect(page.getByTestId('cloud-node-pack-card')).toHaveCount(0)
await expect(page.getByText('No matching packs')).toBeVisible()
})
test('clearing search restores the full list', async ({ page }) => {
const cards = page.getByTestId('cloud-node-pack-card')
const initialCount = await cards.count()
await page.getByTestId('cloud-nodes-search').fill('impact')
await expect(cards.first()).toBeVisible()
await page.getByTestId('cloud-nodes-search').fill('')
await expect(cards).toHaveCount(initialCount)
})
test('search matches against node display names, not just pack names', async ({
page
}) => {
await page.getByTestId('cloud-nodes-search').fill('FaceDetailer')
await expect(page.getByTestId('cloud-node-pack-card')).toHaveCount(1)
await expect(
page.getByTestId('cloud-node-pack-card-link').first()
).toContainText('Impact Pack')
})
test('switching sort to A → Z reorders cards alphabetically', async ({
page
}) => {
await page.locator('#cloud-nodes-sort').selectOption('az')
const firstName = await page
.getByTestId('cloud-node-pack-card-link')
.first()
.textContent()
expect(firstName?.trim().toLowerCase().charAt(0)).toMatch(/^[a-c]/)
})
test('list grid carries a localized aria-label', async ({ page }) => {
await expect(
page.getByRole('list', {
name: 'Custom-node packs supported on Comfy Cloud'
})
).toBeVisible()
})
test('clicking the back link returns to the index from a detail page', async ({
page
}) => {
await page.goto('/cloud/supported-nodes/comfyui-impact-pack')
await page.getByRole('link', { name: 'Back to all packs' }).click()
await expect(page).toHaveURL(/\/cloud\/supported-nodes\/?$/)
await expect(page.getByTestId('cloud-node-pack-card').first()).toBeVisible()
})
test('detail page renders publisher and external repo link', async ({
page
}) => {
await page.goto('/cloud/supported-nodes/comfyui-impact-pack')
const repoLink = page.getByRole('link', {
name: /github\.com\/ltdrdata\/ComfyUI-Impact-Pack/
})
await expect(repoLink).toBeVisible()
await expect(repoLink).toHaveAttribute('rel', /noopener/)
})
test('unknown pack slug 404s', async ({ page }) => {
const response = await page.goto(
'/cloud/supported-nodes/this-pack-does-not-exist'
)
expect(response?.status()).toBe(404)
})
test('JSON-LD ItemList is emitted on the index page', async ({ page }) => {
const jsonLd = page.locator('script[type="application/ld+json"]')
const ldBlocks = await jsonLd.allTextContents()
expect(ldBlocks.some((b) => b.includes('"@type":"ItemList"'))).toBeTruthy()
})
test('JSON-LD payload escapes <-sequences', async ({ page }) => {
const ldBlocks = await page
.locator('script[type="application/ld+json"]')
.allTextContents()
for (const block of ldBlocks) {
expect(block).not.toContain('</script')
}
})
})
test.describe('Cloud nodes page (zh-CN) @smoke', () => {
test('renders localized title and packs', async ({ page }) => {
await page.goto('/zh-CN/cloud/supported-nodes')
await expect(page).toHaveTitle('Comfy Cloud 自定义节点包合集——开箱即用')
await expect(page.getByTestId('cloud-node-pack-card').first()).toBeVisible()
await expect(
page.getByTestId('cloud-node-pack-banner').first()
).toBeVisible()
})
test('opens pack detail page from first card', async ({ page }) => {
await page.goto('/zh-CN/cloud/supported-nodes')
const firstCard = page.getByTestId('cloud-node-pack-card').first()
await expect(firstCard).toBeVisible()
await firstCard.locator('a').first().click()
await expect(page).toHaveURL(/\/zh-CN\/cloud\/supported-nodes\/[a-z0-9-]+$/)
await expect(page.getByTestId('cloud-node-pack-detail')).toBeVisible()
})
})

View File

@@ -1,143 +0,0 @@
import { expect } from '@playwright/test'
import { test } from './fixtures/blockExternalMedia'
test.describe('Cloud page @smoke', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/cloud')
})
test('has correct title', async ({ page }) => {
await expect(page).toHaveTitle('Comfy Cloud — AI in the Cloud')
})
test('HeroSection heading and subtitle are visible', async ({ page }) => {
await expect(
page.getByRole('heading', { name: /The full power of/i, level: 1 })
).toBeVisible()
await expect(
page.getByText(/The easiest way to start with ComfyUI/)
).toBeVisible()
})
test('HeroSection has CTA button linking to cloud', async ({ page }) => {
const cta = page.getByRole('link', { name: /TRY COMFY CLOUD FOR FREE/i })
await expect(cta).toBeVisible()
})
test('ReasonSection heading and reasons are visible', async ({ page }) => {
await expect(
page.getByRole('heading', { name: /Why.*professionals.*choose/i })
).toBeVisible()
for (const title of [
'Powerful GPUs',
'All models',
'More control',
'Community workflows'
]) {
await expect(page.getByText(title).first()).toBeVisible()
}
})
test('AIModelsSection heading and 5 model cards are visible', async ({
page
}) => {
const heading = page.getByRole('heading', { name: /leading AI models/i })
await expect(heading).toBeVisible()
const section = heading.locator('xpath=ancestor::section')
const grid = section.locator('.grid')
const modelCards = grid.locator('a[href="https://comfy.org/workflows"]')
await expect(modelCards).toHaveCount(5)
})
test('AIModelsSection CTA links to workflows', async ({ page }) => {
const cta = page.getByRole('link', {
name: /EXPLORE WORKFLOWS/i
})
await expect(cta.first()).toBeVisible()
await expect(cta.first()).toHaveAttribute(
'href',
'https://comfy.org/workflows'
)
})
test('AudienceSection heading and cards are visible', async ({ page }) => {
await expect(page.getByText(/creators/i).first()).toBeVisible()
for (const label of ['CREATORS', 'TEAMS & STUDIOS']) {
await expect(page.getByText(label).first()).toBeVisible()
}
})
test('PricingSection heading and CTA are visible', async ({ page }) => {
await expect(page.getByText(/Simple, credit-based pricing/)).toBeVisible()
const cta = page.getByRole('link', { name: /SEE PRICING PLANS/i })
await expect(cta).toBeVisible()
await expect(cta).toHaveAttribute('href', '/cloud/pricing')
})
test('ProductCardsSection has 3 product cards', async ({ page }) => {
const section = page.locator('section', {
has: page.getByRole('heading', { name: /The AI creation/ })
})
const cards = section.locator('a[href]')
await expect(cards).toHaveCount(3)
})
test('FAQSection heading is visible with 15 items', async ({ page }) => {
await expect(page.getByRole('heading', { name: /FAQ/i })).toBeVisible()
const faqButtons = page.locator('button[aria-controls^="faq-panel-"]')
await expect(faqButtons).toHaveCount(15)
})
})
test.describe('Cloud FAQ accordion @interaction', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/cloud')
})
test('all FAQs are collapsed by default', async ({ page }) => {
await expect(
page.getByText(/Comfy Cloud is a version of ComfyUI/i)
).toBeHidden()
})
test('clicking a collapsed FAQ expands it', async ({ page }) => {
const firstQuestion = page.getByRole('button', {
name: /What is Comfy Cloud/i
})
await firstQuestion.scrollIntoViewIfNeeded()
// Gate: wait for Vue hydration to bind aria-expanded
await expect(firstQuestion).toHaveAttribute('aria-expanded', 'false')
await firstQuestion.click()
await expect(
page.getByText(/Comfy Cloud is a version of ComfyUI/i)
).toBeVisible()
})
test('clicking an expanded FAQ collapses it again', async ({ page }) => {
const firstQuestion = page.getByRole('button', {
name: /What is Comfy Cloud/i
})
await firstQuestion.scrollIntoViewIfNeeded()
// Gate: wait for Vue hydration to bind aria-expanded
await expect(firstQuestion).toHaveAttribute('aria-expanded', 'false')
await firstQuestion.click()
await expect(firstQuestion).toHaveAttribute('aria-expanded', 'true')
await expect(
page.getByText(/Comfy Cloud is a version of ComfyUI/i)
).toBeVisible()
await firstQuestion.click()
await expect(firstQuestion).toHaveAttribute('aria-expanded', 'false')
await expect(
page.getByText(/Comfy Cloud is a version of ComfyUI/i)
).toBeHidden()
})
})

Some files were not shown because too many files have changed in this diff Show More