Compare commits

..

28 Commits

Author SHA1 Message Date
snomiao
ae19c039b3 [fix] Replace console.log with process.stdout.write
Use process.stdout.write instead of console.log to avoid no-console
lint rule violations in CLI scripts that output to stdout.
2026-03-14 06:24:46 +00:00
snomiao
132539fe49 [refactor] Improve API snapshot and changelog scripts
- Filter non-exported declarations in snapshot-api.ts to only capture
  the public API surface
- Remove redundant 'exported' field from snapshot output
- Return empty string from formatChangelog when no changes detected,
  preventing empty changelog PRs
- Remove dead commented-out Additions section code
- Remove unused deprecations field
- Update README example to match actual output format
2026-03-14 06:14:23 +00:00
snomiao
2312db5a2d [fix] Remove nonexistent demo-snapshots knip ignore entry
The demo-snapshots/ directory doesn't exist in the repository.
The workflows use .api-snapshots/ at runtime which is never committed.
2026-03-14 06:10:33 +00:00
snomiao
fb5f5bd90e [fix] Address workflow review feedback
- Add || true to git stash to handle empty working tree
- Use node-version-file: '.nvmrc' instead of lts/* to match repo
  Node 24 requirement
- Fix relative links in README (use ./ since README is in same dir)
- Fix pre-release version example to use valid X.Y.Z format
2026-03-14 06:10:17 +00:00
GitHub Action
24d8a9ce3a [automated] Apply ESLint and Oxfmt fixes 2026-03-13 02:07:53 +00:00
snomiao
5ef47de839 [fix] Pin GitHub Actions to specific commit SHAs
Fix validate-pins CI check by pinning all unpinned actions to their
exact commit SHA for security and reproducibility.
2026-03-13 02:04:43 +00:00
snomiao
3724ae7549 [fix] Remove debug push trigger from release-api-changelogs workflow
Remove the push trigger for sno-api-changelog branch that was added
for testing purposes. The workflow now only runs on successful completion
of the Release NPM Types workflow. Also simplify the PR base branch
to always target main.
2026-03-13 02:01:58 +00:00
snomiao
d32edbc986 Merge branch 'main' into sno-api-changelog
Resolve conflict in knip.config.ts by keeping demo-snapshots/** ignore
entry alongside new main entries.
2026-03-13 01:59:34 +00:00
snomiao
794829db6d [refactor] Clean up API changelog implementation
- Remove demo-snapshots folder
- Merge workflow documentation into main workflows README
- Convert scripts to TypeScript (.js → .ts)
- Revert eslint.config.ts changes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 09:35:35 +00:00
sno
e36e25ebd6 Merge branch 'main' into sno-api-changelog 2025-11-10 18:26:42 +09:00
snomiao
f844d3e95b [feat] Use source file links instead of dist .d.ts in API changelog
Update changelog link generation to prefer source file locations over
generated declaration file locations. Links now point to the actual
source code where APIs are defined (e.g., src/stores/commandStore.ts)
instead of the bundled dist/index.d.ts file.

This makes the changelog much more useful as developers can click
through to see the real implementation and context.

Falls back to .d.ts location if source mapping is unavailable.

Fixes the issue where links like:
https://github.com/.../dist/index.d.ts#L1247

Are now:
https://github.com/.../src/stores/commandStore.ts#L10

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 17:22:34 +00:00
snomiao
98ed124b02 [feat] Add source file location discovery to API snapshot tool
Enhance snapshot-api.js to discover source file locations for exported
declarations using grep search. This enables mapping from generated
.d.ts declarations back to their original source files.

For each declaration, the tool now searches the src directory for:
- export interface <Name>
- export class <Name>
- export type <Name>
- export enum <Name>
- export function <Name>
- export const <Name>

The source file path and line number are stored in the snapshot for use
in generating accurate GitHub permalinks in the changelog.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 17:22:11 +00:00
snomiao
e933b5c357 [feat] Enable declarationMap for source file mapping in API changelog
Generate declaration maps to enable mapping from generated .d.ts files
back to source files. This will allow the API changelog to link to actual
source code locations instead of generated declaration files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 17:21:30 +00:00
sno
cb32562f04 Merge branch 'main' into sno-api-changelog 2025-11-06 00:54:16 +08:00
snomiao
9c94a4818f [feat] Remove Additions section and add hyperlinks to API changelog
- Remove Additions section from changelog output (no longer needed)
- Add line number tracking in API snapshots
- Generate GitHub permalinks to referenced code in changelog
- Update workflows to pass git reference for link generation
- Breaking changes and modifications now link to source code

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 10:15:37 +00:00
sno
779f539b0e Merge branch 'main' into sno-api-changelog 2025-11-05 07:51:42 +08:00
snomiao
95c815f17c [docs] Add comprehensive documentation for manual API changelog workflow
- Usage examples via UI and CLI
- Detailed explanation of workflow steps
- Common use cases and troubleshooting
- Security and monitoring information

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 23:27:33 +00:00
snomiao
30dcf8c133 [feat] Add manual dispatch workflow for API changelog generation
- Allows comparing any two versions via workflow_dispatch
- Accepts version inputs (e.g., 1.29.0 and 1.30.2)
- Validates version format and tag existence
- Generates changelog and uploads as artifact (90-day retention)
- Optional PR creation for updating docs/API-CHANGELOG.md
- Provides detailed summary in workflow output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 23:21:56 +00:00
snomiao
f1193a2f86 [fix] Exclude demo-snapshots from knip checks
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-01 12:23:14 +00:00
snomiao
75daf2e4d2 [docs] Add API changelog generation demo
Add comprehensive demo showing API changelog generation across two versions (v1.29.0 → v1.30.2):

- Mock TypeScript definitions for both versions
- Generated JSON snapshots showing structured API surface
- Generated changelog with breaking changes, additions, and modifications
- Detailed README explaining the system and benefits

The demo showcases:
 Detection of breaking changes (queuePrompt → queuePromptAsync)
 Tracking of new additions (ExtensionMetadata interface, WorkflowId type)
 Identification of modifications (NodeDef enhancements, NodeStatus enum extensions)

This demonstrates the automated system that will run on every release to document API changes for extension developers.

Also exclude demo-snapshots directory from ESLint checking.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-01 12:21:37 +00:00
sno
11922709a9 Merge branch 'main' into sno-api-changelog 2025-10-30 16:57:22 +09:00
snomiao
3742a76cfb [fix] Use current branch as PR base when triggered by push event
- Set PR base to current branch (github.ref_name) when triggered by push
- Keeps main as base for workflow_run triggers
- Allows testing on feature branches without conflicts with main

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 11:05:37 +00:00
snomiao
c84144581d [fix] Remove copied scripts before git checkout to avoid conflicts
- Clean up scripts/snapshot-api.js and scripts/compare-api-snapshots.js before checkout
- Fixes untracked working tree files conflict when returning to current version

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 11:00:37 +00:00
snomiao
8b88f8ccae [fix] Preserve API changelog scripts when checking out previous version
- Copy snapshot and comparison scripts to /tmp before checking out previous version
- Restore scripts after checkout to ensure they're available in older versions
- Fixes MODULE_NOT_FOUND error when running snapshot on previous versions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:55:52 +00:00
snomiao
0aab7cba4b [feat] Configure API changelog workflow to run on push to sno-api-changelog branch
- Add push trigger for sno-api-changelog branch to allow testing
- Update job condition to handle both workflow_run and push events
- Limit version comparison to two most recent versions for efficiency

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 10:50:12 +00:00
sno
608874a312 Merge branch 'main' into sno-api-changelog 2025-10-29 19:44:28 +09:00
GitHub Action
5cf6ac07ac [automated] Apply ESLint and Prettier fixes 2025-10-23 03:06:37 +00:00
snomiao
ff60bdf1bc [feat] Add automated API changelog generation workflow
Implements a GitHub Actions workflow that automatically generates API
changelogs by comparing TypeScript type definitions between versions.

Changes:
- Add release-api-changelogs.yaml workflow triggered after npm types release
- Create snapshot-api.js script to extract API surface from TypeScript defs
- Create compare-api-snapshots.js to generate human-readable changelogs
- Initialize docs/API-CHANGELOG.md to track public API changes

The workflow:
1. Triggers after Release NPM Types workflow completes
2. Builds and snapshots current and previous API surfaces
3. Compares snapshots to detect additions, removals, and modifications
4. Generates formatted changelog with breaking changes highlighted
5. Creates draft PR for review before merging

This automates documentation of breaking changes for extension developers
without manual effort, supporting the large extension ecosystem.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 03:03:11 +00:00
447 changed files with 4046 additions and 31855 deletions

View File

@@ -1,9 +0,0 @@
{
"permissions": {
"allow": [
"Bash(pnpx vitest run --testPathPattern=\"draftCacheV2.property\")",
"Bash(pnpx vitest run \"draftCacheV2.property\")",
"Bash(node -e \"const fc = require\\(''fast-check''\\); console.log\\(Object.keys\\(fc\\).filter\\(k => k.includes\\(''string''\\)\\).join\\('', ''\\)\\)\")"
]
}
}

View File

@@ -1,358 +0,0 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using Playwright MCP, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Perform comprehensive quality assurance of the ComfyUI frontend application by navigating all routes, clicking interactive elements, and testing features. Generate a structured report and submit it as a draft PR.
## Prerequisites
- Node.js 18+
- `pnpm` package manager
- `gh` CLI (authenticated)
- Playwright MCP server (browser automation)
## Step 1: Environment Detection & Setup
Detect the runtime environment and ensure the app is accessible.
### CI Environment
If `CI=true` is set:
1. The ComfyUI backend is pre-configured in the CI container (`ghcr.io/comfy-org/comfyui-ci-container`)
2. Frontend dist is already built and served by the backend
3. Server runs at `http://127.0.0.1:8188`
4. Skip user prompts — run fully automated
### Local Environment
If `CI` is not set:
1. **Ask the user**: "Is a ComfyUI server already running? If so, what URL? (default: http://127.0.0.1:8188)"
- If yes: use the provided URL
- If no: offer to start one:
```bash
# Option A: Use existing ComfyUI installation
# Ask for the path to ComfyUI, then:
cd <comfyui_path>
python main.py --cpu --multi-user --front-end-root <frontend_dist_path> &
# Option B: Build frontend and use preview server (no backend features)
pnpm build && pnpm preview &
```
2. Wait for server readiness by polling the URL (retry with 2s intervals, 60s timeout)
### Browser Automation Setup
Use **Playwright MCP** for browser interaction. The agent should use whichever Playwright MCP is available in its tool set:
- **Claude Code**: Uses `remote-chrome` MCP or built-in browser tools
- **Interactive mode**: User can opt for `npx @anthropic-ai/mcp-server-playwright` or `npx @playwright/mcp@latest --port 3100` for headless
- **User's real browser**: `npx @playwright/mcp@latest --extension` — lets QA happen in the user's actual browser session
Determine the server URL. For local dev servers behind proxies, adjust the URL accordingly (e.g., `https://[port].stukivx.xyz` pattern if configured).
## Step 2: QA Test Plan
Navigate to the application URL and systematically test each area below. For each test, record:
- **Status**: pass / fail / skip (with reason)
- **Notes**: any issues, unexpected behavior, or visual glitches
- **Screenshots**: take screenshots of failures or notable states
### 2.1 Application Load & Routes
| Test | Steps |
| ----------------- | ------------------------------------------------------------ |
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| Default redirect | If multi-user mode, `/` redirects to `/user-select` first |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### 2.2 Canvas & Graph View
| Test | Steps |
| ------------------------- | -------------------------------------------------------------- |
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Fit view | Press `.` key — canvas fits to content |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Add node via search | Open search box, find and add a node |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Disconnect nodes | Right-click a link and remove, or drag from connected slot |
| Multi-select | Shift+click or drag-select multiple nodes |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Node context menu | Right-click a node — menu appears with all expected options |
| Canvas context menu | Right-click empty canvas — menu appears |
### 2.3 Node Operations
| Test | Steps |
| ------------------- | ---------------------------------------------------------- |
| Bypass node | Select node, Ctrl+B — node shows bypass state |
| Mute node | Select node, Ctrl+M — node shows muted state |
| Collapse node | Select node, Alt+C — node collapses |
| Pin node | Select node, press P — node becomes pinned |
| Rename node | Double-click node title — edit mode activates |
| Node color | Right-click > Color — color picker works |
| Group nodes | Select multiple nodes, Ctrl+G — group created |
| Ungroup | Right-click group > Ungroup |
| Widget interactions | Toggle checkboxes, adjust sliders, type in text fields |
| Combo widget | Click dropdown widgets — options appear and are selectable |
### 2.4 Sidebar Tabs
| Test | Steps |
| ---------------------- | ------------------------------------------------------ |
| Workflows tab | Press W — workflows sidebar opens with saved workflows |
| Node Library tab | Press N — node library opens with categories |
| Model Library tab | Press M — model library opens |
| Assets tab | Press A — assets browser opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
| Drag node from library | Drag a node from library onto canvas |
### 2.5 Topbar & Workflow Tabs
| Test | Steps |
| -------------------- | ------------------------------------------------------ |
| Workflow tab display | Current workflow name shown in tab bar |
| New workflow | Ctrl+N — new blank workflow created |
| Rename workflow | Double-click workflow tab |
| Tab context menu | Right-click workflow tab — menu with Close/Rename/etc. |
| Multiple tabs | Open multiple workflows, switch between them |
| Queue button | Click Queue/Run button — prompt queues |
| Batch count | Click batch count editor, change value |
| Menu hamburger | Click hamburger menu — options appear |
### 2.6 Settings Dialog
| Test | Steps |
| ---------------- | ---------------------------------------------------- |
| Open settings | Press Ctrl+, or click settings button |
| Settings tabs | Navigate through all setting categories |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Keybindings tab | Navigate to keybindings panel |
| About tab | Navigate to about panel — version info shown |
| Close settings | Press Escape or click close button |
### 2.7 Bottom Panel
| Test | Steps |
| ------------------- | -------------------------------------- |
| Toggle panel | Press Ctrl+` — bottom panel opens |
| Logs tab | Logs/terminal tab shows server output |
| Shortcuts tab | Shortcuts reference is displayed |
| Keybindings display | Press Ctrl+Shift+K — keybindings panel |
### 2.8 Execution & Queue
| Test | Steps |
| -------------- | ----------------------------------------------------- |
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
| Job history | Open job history sidebar — past executions listed |
| Clear history | Clear execution history via menu |
### 2.9 Workflow File Operations
| Test | Steps |
| --------------- | ------------------------------------------------- |
| Save workflow | Ctrl+S — workflow saves (check for prompt if new) |
| Open workflow | Ctrl+O — file picker or workflow browser opens |
| Export JSON | Menu > Export — workflow JSON downloads |
| Import workflow | Drag a .json workflow file onto canvas |
| Load default | Menu > Load Default — default workflow loads |
| Clear workflow | Menu > Clear — canvas clears (after confirmation) |
### 2.10 Advanced Features
| Test | Steps |
| --------------- | ------------------------------------------------- |
| Minimap | Alt+M — minimap toggle |
| Focus mode | Toggle focus mode |
| Canvas lock | Press H to lock, V to unlock |
| Link visibility | Ctrl+Shift+L — toggle links |
| Subgraph | Select nodes > Ctrl+Shift+E — convert to subgraph |
### 2.11 Error Handling
| Test | Steps |
| --------------------- | -------------------------------------------- |
| Missing nodes dialog | Load workflow with non-existent node types |
| Missing models dialog | Trigger missing model warning |
| Network error | Disconnect backend, verify graceful handling |
| Invalid workflow | Try loading malformed JSON |
### 2.12 Responsive & Accessibility
| Test | Steps |
| ------------------- | ------------------------------------- |
| Window resize | Resize browser window — layout adapts |
| Keyboard navigation | Tab through interactive elements |
| Sidebar resize | Drag sidebar edge to resize |
## Step 3: Generate Report
After completing all tests, generate a markdown report file.
### Report Location
```
docs/qa/YYYY-MM-DD-NNN-report.md
```
Where:
- `YYYY-MM-DD` is today's date
- `NNN` is a zero-padded increment index (001, 002, etc.)
To determine the increment, check existing files:
```bash
ls docs/qa/ | grep "$(date +%Y-%m-%d)" | wc -l
```
### Report Template
```markdown
# QA Report: ComfyUI Frontend
**Date**: YYYY-MM-DD
**Environment**: CI / Local (OS, Browser)
**Frontend Version**: (git sha or version)
**Agent**: Claude / Codex / Other
**Server URL**: http://...
## Summary
| Category | Pass | Fail | Skip | Total |
| --------------- | ---- | ---- | ---- | ----- |
| Routes & Load | | | | |
| Canvas | | | | |
| Node Operations | | | | |
| Sidebar | | | | |
| Topbar | | | | |
| Settings | | | | |
| Bottom Panel | | | | |
| Execution | | | | |
| File Operations | | | | |
| Advanced | | | | |
| Error Handling | | | | |
| Responsive | | | | |
| **Total** | | | | |
## Results
### Routes & Load
- [x] Root route loads — pass
- [ ] ...
### Canvas & Graph View
- [x] Canvas renders — pass
- [ ] ...
(repeat for each category)
## Issues Found
### Issue 1: [Title]
- **Severity**: critical / major / minor / cosmetic
- **Steps to reproduce**: ...
- **Expected**: ...
- **Actual**: ...
- **Screenshot**: (if available)
## Notes
Any additional observations, performance notes, or suggestions.
```
## Step 4: Commit and Push Report
### In CI (when `CI=true`)
Commit and push the report directly to the current branch. Do **not** create
a new PR — the CI workflow handles posting results as a PR comment.
```bash
git add docs/qa/
git commit -m "docs: add QA report $(date +%Y-%m-%d)"
git push
```
### Local / interactive use
When running locally, create a draft PR after committing:
```bash
# Ensure on a feature branch
BRANCH_NAME="qa/$(date +%Y-%m-%d)-$(git rev-parse --short HEAD)"
git checkout -b "$BRANCH_NAME" 2>/dev/null || git checkout "$BRANCH_NAME"
git add docs/qa/
git commit -m "docs: add QA report $(date +%Y-%m-%d)
Automated QA report covering all frontend routes and features."
git push -u origin "$BRANCH_NAME"
# Create draft PR assigned to comfy-pr-bot
gh pr create \
--draft \
--title "QA Report: $(date +%Y-%m-%d)" \
--body "## QA Report
Automated frontend QA run covering all routes and interactive features.
See \`docs/qa/\` for the full report.
/cc @comfy-pr-bot" \
--assignee comfy-pr-bot
```
## Execution Notes
### Cross-Platform Considerations
- **Windows**: Use `pwsh` or `cmd` equivalents for shell commands. `gh` CLI works on all platforms.
- **macOS**: Keyboard shortcuts use Cmd instead of Ctrl in the actual app, but Playwright sends OS-appropriate keys.
- **Linux**: Primary CI platform. Screenshot baselines are Linux-only.
### Agent Compatibility
This skill is written to be agent-agnostic:
- **Claude Code**: Has native Playwright MCP via `remote-chrome` or configured MCP servers
- **Codex**: Can use `npx @playwright/mcp@latest` for browser automation
- **Other agents**: Any agent with browser automation tools can follow these instructions
The key requirement is the ability to:
1. Navigate to URLs
2. Take snapshots/screenshots
3. Click elements
4. Type text
5. Press keyboard shortcuts
6. Read page content
### Tips for Reliable QA
1. **Wait for page stability** before interacting — check that elements are visible and enabled
2. **Take a snapshot after each major navigation** to verify state
3. **Don't use fixed timeouts** — poll for expected conditions
4. **Record the full page snapshot** at the start for baseline comparison
5. **If a test fails**, document it and continue — don't abort the entire QA run
6. **Group related tests** — complete one category before moving to the next

View File

@@ -1,179 +0,0 @@
---
name: perf-fix-with-proof
description: 'Ships performance fixes with CI-proven improvement using stacked PRs. PR1 adds a @perf test (establishes baseline on main), PR2 adds the fix (CI shows delta). Use when implementing a perf optimization and wanting to prove it in CI.'
---
# Performance Fix with Proof
Ships perf fixes as two stacked PRs so CI automatically proves the improvement.
## Why Two PRs
The `ci-perf-report.yaml` workflow compares PR metrics against the **base branch baseline**. If you add a new `@perf` test in the same PR as the fix, that test doesn't exist on main yet — no baseline, no delta, no proof. Stacking solves this:
1. **PR1 (test-only)** — adds the `@perf` test that exercises the bottleneck. Merges to main. CI runs it on main → baseline established.
2. **PR2 (fix)** — adds the optimization. CI runs the same test → compares against PR1's baseline → delta shows improvement.
## Workflow
### Step 1: Create the test branch
```bash
git worktree add <worktree-path> -b perf/test-<name> origin/main
```
### Step 2: Write the `@perf` test
Add a test to `browser_tests/tests/performance.spec.ts` (or a new file with `@perf` tag). The test should stress the specific bottleneck.
**Test structure:**
```typescript
test('<descriptive name>', async ({ comfyPage }) => {
// 1. Load a workflow that exercises the bottleneck
await comfyPage.workflow.loadWorkflow('<workflow>')
// 2. Start measuring
await comfyPage.perf.startMeasuring()
// 3. Perform the action that triggers the bottleneck (at scale)
for (let i = 0; i < N; i++) {
// ... stress the hot path ...
await comfyPage.nextFrame()
}
// 4. Stop measuring and record
const m = await comfyPage.perf.stopMeasuring('<metric-name>')
recordMeasurement(m)
console.log(`<name>: ${m.styleRecalcs} recalcs, ${m.layouts} layouts`)
})
```
**Available metrics** (from `PerformanceHelper`):
- `m.styleRecalcs` / `m.styleRecalcDurationMs` — style recalculation count and time
- `m.layouts` / `m.layoutDurationMs` — forced layout count and time
- `m.taskDurationMs` — total main-thread JS execution time
- `m.heapDeltaBytes` — memory pressure delta
**Key helpers** (from `ComfyPage`):
- `comfyPage.perf.startMeasuring()` / `.stopMeasuring(name)` — CDP metrics capture
- `comfyPage.nextFrame()` — wait one animation frame
- `comfyPage.workflow.loadWorkflow(name)` — load a test workflow from `browser_tests/assets/`
- `comfyPage.canvas` — the canvas locator
- `comfyPage.page.mouse.move(x, y)` — mouse interaction
### Step 3: Add test workflow asset (if needed)
If the bottleneck needs a specific workflow (e.g., 50+ nodes, many DOM widgets), add it to `browser_tests/assets/`. Keep it minimal — only the structure needed to trigger the bottleneck.
### Step 4: Verify locally
```bash
pnpm exec playwright test --project=performance --grep "<test name>"
```
Confirm the test runs and produces reasonable metric values.
### Step 5: Create PR1 (test-only)
```bash
pnpm typecheck:browser
pnpm lint
git add browser_tests/
git commit -m "test: add perf test for <bottleneck description>"
git push -u origin perf/test-<name>
gh pr create --title "test: add perf test for <bottleneck>" \
--body "Adds a @perf test to establish a baseline for <bottleneck>.
This is PR 1 of 2. The fix will follow in a separate PR once this baseline is established on main.
## What
Adds \`<test-name>\` to the performance test suite measuring <metric> during <action>.
## Why
Needed to prove the improvement from the upcoming fix for backlog item #<N>." \
--base main
```
### Step 6: Get PR1 merged
Once PR1 merges, CI runs the test on main → baseline artifact saved.
### Step 7: Create PR2 (fix) on top of main
```bash
git worktree add <worktree-path> -b perf/fix-<name> origin/main
```
Implement the fix. The `@perf` test from PR1 is now on main and will run automatically. CI will:
1. Run the test on the PR branch
2. Download the baseline from main (which includes PR1's test results)
3. Post a PR comment showing the delta
### Step 8: Verify the improvement shows in CI
The `ci-perf-report.yaml` posts a comment like:
```markdown
## ⚡ Performance Report
| Metric | Baseline | PR (n=3) | Δ | Sig |
| --------------------- | -------- | -------- | ---- | --- |
| <name>: style recalcs | 450 | 12 | -97% | 🟢 |
```
If Δ is negative for the target metric, the fix is proven.
## Test Design Guidelines
1. **Stress the specific bottleneck** — don't measure everything, isolate the hot path
2. **Use enough iterations** — the test should run long enough that the metric difference is clear (100+ frames for idle tests, 50+ interactions for event tests)
3. **Keep it deterministic** — avoid timing-dependent assertions; measure counts not durations when possible
4. **Match the backlog entry** — reference the backlog item number in the test name or PR description
## Examples
**Testing DOM widget reactive mutations (backlog #8):**
```typescript
test('DOM widget positioning recalculations', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('default')
await comfyPage.perf.startMeasuring()
// Idle for 120 frames — DOM widgets update position every frame
for (let i = 0; i < 120; i++) {
await comfyPage.nextFrame()
}
const m = await comfyPage.perf.stopMeasuring('dom-widget-idle')
recordMeasurement(m)
})
```
**Testing measureText caching (backlog #4):**
```typescript
test('canvas text rendering with many nodes', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('large-workflow-50-nodes')
await comfyPage.perf.startMeasuring()
for (let i = 0; i < 60; i++) {
await comfyPage.nextFrame()
}
const m = await comfyPage.perf.stopMeasuring('text-rendering-50-nodes')
recordMeasurement(m)
})
```
## Reference
| Resource | Path |
| ----------------- | ----------------------------------------------------- |
| Perf test file | `browser_tests/tests/performance.spec.ts` |
| PerformanceHelper | `browser_tests/fixtures/helpers/PerformanceHelper.ts` |
| Perf reporter | `browser_tests/helpers/perfReporter.ts` |
| CI workflow | `.github/workflows/ci-perf-report.yaml` |
| Report generator | `scripts/perf-report.ts` |
| Stats utilities | `scripts/perf-stats.ts` |
| Backlog | `docs/perf/BACKLOG.md` (local only, not committed) |
| Playbook | `docs/perf/PLAYBOOK.md` (local only, not committed) |

View File

@@ -1,221 +0,0 @@
---
name: red-green-fix
description: 'Bug fix workflow that proves test validity with a red-then-green CI sequence. Commits a failing test first (CI red), then the minimal fix (CI green). Use when fixing a bug, writing a regression test, or when asked to prove a fix works.'
---
# Red-Green Fix
Fixes bugs as two commits so CI automatically proves the test catches the bug.
## Why Two Commits
If you commit the test and fix together, the test always passes — reviewers cannot tell whether the test actually detects the bug or is a no-op. Splitting into two commits creates a verifiable CI trail:
1. **Commit 1 (test-only)** — adds a test that exercises the bug. CI runs it → test fails → red X.
2. **Commit 2 (fix)** — adds the minimal fix. CI runs the same test → test passes → green check.
The red-then-green sequence in the commit history proves the test is valid.
## Input
The user provides a bug description as an argument. If no description is given, ask the user to describe the bug before proceeding.
Bug description: $ARGUMENTS
## Step 0 — Setup
Create an isolated branch from main:
```bash
git fetch origin main
git checkout -b fix/<bug-name> origin/main
```
## Step 1 — Red: Failing Test Only
Write a test that reproduces the bug. **Do NOT write any fix code.**
### Choosing the Test Framework
| Bug type | Framework | File location |
| --------------------------------- | ---------- | ------------------------------- |
| Logic, utils, stores, composables | Vitest | `src/**/*.test.ts` (colocated) |
| UI interaction, canvas, workflows | Playwright | `browser_tests/tests/*.spec.ts` |
For Playwright tests, follow the `/writing-playwright-tests` skill for patterns, fixtures, and tags.
### Rules
- The test MUST fail against the current codebase (this is the whole point)
- Do NOT modify any source code outside of test files
- Do NOT include any fix, workaround, or behavioral change
- Do NOT add unrelated tests or refactor existing tests
- Keep the test minimal — only what is needed to reproduce the bug
- Avoid common anti-patterns — see `reference/testing-anti-patterns.md`
### Vitest Example
```typescript
// src/utils/pathUtil.test.ts
import { describe, expect, it } from 'vitest'
import { resolveModelPath } from './pathUtil'
describe('resolveModelPath', () => {
it('handles absolute paths from folder_paths API', () => {
const result = resolveModelPath(
'/absolute/models',
'/absolute/models/checkpoints'
)
expect(result).toBe('/absolute/models/checkpoints')
})
})
```
### Playwright Example
```typescript
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test.describe('Model Download', { tag: ['@smoke'] }, () => {
test('downloads model when path is absolute', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('missing-model')
const downloadBtn = comfyPage.page.getByTestId('download-model-button')
await downloadBtn.click()
await expect(comfyPage.page.getByText('Download complete')).toBeVisible()
})
})
```
### Verify Locally First
Run the test locally before pushing to confirm it fails for the right reason:
```bash
# Vitest
pnpm test:unit -- <test-file>
# Playwright
pnpm test:browser:local -- --grep "<test name>"
```
If the test passes locally, it does not reproduce the bug — revisit your test before pushing.
### Quality Checks and Commit
```bash
pnpm typecheck
pnpm lint
pnpm format:check
git add <test-files-only>
git commit -m "test: add failing test for <concise bug description>"
git push -u origin HEAD
```
### Verify CI Failure
```bash
gh run list --branch $(git branch --show-current) --limit 1
```
**STOP HERE.** Inform the user of the CI status and wait for confirmation before proceeding to Step 2.
- If CI passes: the test does not catch the bug. Revisit the test.
- If CI fails for unrelated reasons: investigate and fix the test setup, not the bug.
- If CI fails because the test correctly catches the bug: proceed to Step 2.
## Step 2 — Green: Minimal Fix
Write the minimum code change needed to make the failing test pass.
### Rules
- Do NOT modify, weaken, or delete the test from Step 1 — it is immutable. If the test needs changes, restart from Step 1 and re-prove the red.
- Do NOT add new tests (tests were finalized in Step 1)
- Do NOT refactor, clean up, or make "drive-by" improvements
- Do NOT modify code unrelated to the bug
- The fix should be the smallest correct change
### Quality Checks and Commit
```bash
pnpm typecheck
pnpm lint
pnpm format
git add <fix-files-only>
git commit -m "fix: <concise bug description>"
git push
```
### Verify CI Pass
```bash
gh run list --branch $(git branch --show-current) --limit 1
```
- If CI passes: the fix is verified. Proceed to PR creation.
- If CI fails: investigate and fix. Do NOT change the test from Step 1.
## Step 3 — Open Pull Request
```bash
gh pr create --title "fix: <description>" --body "$(cat <<'EOF'
## Summary
<Brief explanation of the bug and root cause>
- Fixes #<issue-number>
## Red-Green Verification
| Commit | CI Status | Purpose |
|--------|-----------|---------|
| `test: ...` | :red_circle: Red | Proves the test catches the bug |
| `fix: ...` | :green_circle: Green | Proves the fix resolves the bug |
## Test Plan
- [ ] CI red on test-only commit
- [ ] CI green on fix commit
- [ ] Added/updated E2E regression under `browser_tests/` or explained why not applicable
- [ ] Manual verification (if applicable)
EOF
)"
```
## Gotchas
### CI fails on test commit for unrelated reasons
Lint, typecheck, or other tests may fail — not just your new test. Check the CI logs carefully. If the failure is unrelated, fix it in a separate commit before the `test:` commit so the red X is clearly attributable to your test.
### Test passes when it should fail
The bug may only manifest under specific conditions (e.g., Windows paths, external model directories, certain workflow structures). Make sure your test setup matches the actual bug scenario. Check that you're not accidentally testing the happy path.
### Flaky Playwright tests
If your e2e test is intermittent, it doesn't prove anything. Use retrying assertions (`toBeVisible`, `toHaveText`) instead of `waitForTimeout`. See the `/writing-playwright-tests` skill for anti-patterns.
### Pre-existing CI failures on main
If main itself is red, branch from the last green commit or fix the pre-existing failure first. A red-green proof is meaningless if the baseline is already red.
## Reference
| Resource | Path |
| --------------------- | -------------------------------------------------- |
| Unit test framework | Vitest (`src/**/*.test.ts`) |
| E2E test framework | Playwright (`browser_tests/tests/*.spec.ts`) |
| E2E fixtures | `browser_tests/fixtures/` |
| E2E assets | `browser_tests/assets/` |
| Playwright skill | `.claude/skills/writing-playwright-tests/SKILL.md` |
| Unit CI | `.github/workflows/ci-tests-unit.yaml` |
| E2E CI | `.github/workflows/ci-tests-e2e.yaml` |
| Lint CI | `.github/workflows/ci-lint-format.yaml` |
| Testing anti-patterns | `reference/testing-anti-patterns.md` |
| Related skill | `.claude/skills/perf-fix-with-proof/SKILL.md` |

View File

@@ -1,214 +0,0 @@
# Testing Anti-Patterns for Red-Green Fixes
Common mistakes that undermine the red-green proof. Avoid these when writing the test commit (Step 1).
## Testing Implementation Details
Test observable behavior, not internal state.
**Bad** — coupling to internals:
```typescript
it('uses cache internally', () => {
const service = new UserService()
service.getUser(1)
expect(service._cache.has(1)).toBe(true) // Implementation detail
})
```
**Good** — testing through the public interface:
```typescript
it('returns same user on repeated calls', async () => {
const service = new UserService()
const user1 = await service.getUser(1)
const user2 = await service.getUser(1)
expect(user1).toBe(user2) // Behavior, not implementation
})
```
Why this matters for red-green: if your test is coupled to internals, a valid fix that changes the implementation may break the test — even though the bug is fixed. The green commit should only require changing source code, not rewriting the test.
## Assertion-Free Tests
Every test must assert something meaningful. A test without assertions always passes — it cannot produce the red X needed in Step 1.
**Bad**:
```typescript
it('processes the download', () => {
processDownload('/models/checkpoints', 'model.safetensors')
// No expect()!
})
```
**Good**:
```typescript
it('processes the download to correct path', () => {
const result = processDownload('/models/checkpoints', 'model.safetensors')
expect(result.savePath).toBe('/models/checkpoints/model.safetensors')
})
```
## Over-Mocking
Mock only system boundaries (network, filesystem, Electron APIs). If you mock the module under test, you are testing your mocks — the test will not detect the real bug.
**Bad** — mocking everything:
```typescript
vi.mock('./pathResolver')
vi.mock('./validator')
vi.mock('./downloader')
it('downloads model', () => {
// This only tests that mocks were called, not that the bug exists
})
```
**Good** — mock only the boundary:
```typescript
vi.mock('./electronAPI') // Boundary: Electron IPC
it('resolves absolute path correctly', () => {
const result = resolveModelPath('/root/models', '/root/models/checkpoints')
expect(result).toBe('/root/models/checkpoints')
})
```
See also: [Don't Mock What You Don't Own](https://hynek.me/articles/what-to-mock-in-5-mins/)
## Giant Tests
A test that covers the entire flow makes it hard to pinpoint which part catches the bug. Keep it focused — one concept per test.
**Bad**:
```typescript
it('full model download flow', async () => {
// 80 lines: load workflow, open dialog, select model,
// click download, verify path, check progress, confirm completion
})
```
**Good**:
```typescript
it('resolves absolute savePath without nesting under modelsDirectory', () => {
const result = getLocalSavePath(
'/models',
'/models/checkpoints',
'file.safetensors'
)
expect(result).toBe('/models/checkpoints/file.safetensors')
})
```
If the bug is in path resolution, test path resolution — not the entire download flow.
## Test Duplication
Duplicated test code hides what actually differs between cases. Use parameterized tests.
**Bad**:
```typescript
it('resolves checkpoints path', () => {
expect(resolve('/models', '/models/checkpoints', 'a.safetensors')).toBe(
'/models/checkpoints/a.safetensors'
)
})
it('resolves loras path', () => {
expect(resolve('/models', '/models/loras', 'b.safetensors')).toBe(
'/models/loras/b.safetensors'
)
})
```
**Good**:
```typescript
it.each([
['/models/checkpoints', 'a.safetensors', '/models/checkpoints/a.safetensors'],
['/models/loras', 'b.safetensors', '/models/loras/b.safetensors']
])('resolves %s/%s to %s', (dir, file, expected) => {
expect(resolve('/models', dir, file)).toBe(expected)
})
```
## Flaky Tests
A flaky test cannot prove anything — it may show red for reasons unrelated to the bug, or green despite the bug still existing.
**Common causes in this codebase:**
| Cause | Fix |
| -------------------------------------- | --------------------------------------- |
| Missing `nextFrame()` after canvas ops | Add `await comfyPage.nextFrame()` |
| `waitForTimeout` instead of assertions | Use `toBeVisible()`, `toHaveText()` |
| Shared state between tests | Isolate with `afterEach` / `beforeEach` |
| Timing-dependent logic | Use `expect.poll()` or `toPass()` |
## Gaming the Red-Green Process
These are ways the red-green proof gets invalidated during Step 2 (the fix commit). The test from Step 1 is immutable — if any of these happen, restart from Step 1.
**Weakening the assertion to make it pass:**
```typescript
// Step 1 (red) — strict assertion
expect(result).toBe('/external/drive/models/checkpoints/file.safetensors')
// Step 2 (green) — weakened to pass without a real fix
expect(result).toBeDefined() // This proves nothing
```
**Updating snapshots to bless the bug:**
```bash
# Instead of fixing the code, just updating the snapshot to match buggy output
pnpm test:unit -- --update
```
If a snapshot needs updating, the fix should change the code behavior, not the expected output.
**Adding mocks in Step 2 that hide the failure:**
```typescript
// Step 2 adds a mock that didn't exist in Step 1
vi.mock('./pathResolver', () => ({
resolve: () => '/expected/path' // Hardcoded to pass
}))
```
Step 2 should only change source code — not test infrastructure.
## Testing the Happy Path Only
The red-green pattern specifically requires the test to exercise the **broken path**. If you only test the case that already works, the test will pass (green) on Step 1 — defeating the purpose.
**Bad** — testing the default case that works:
```typescript
it('downloads to default models directory', () => {
// This already works — it won't produce a red X
const result = resolve('/models', 'checkpoints', 'file.safetensors')
expect(result).toBe('/models/checkpoints/file.safetensors')
})
```
**Good** — testing the case that is actually broken:
```typescript
it('downloads to external models directory configured via extra_model_paths', () => {
// This is the broken case — absolute path from folder_paths API
const result = resolve(
'/models',
'/external/drive/models/checkpoints',
'file.safetensors'
)
expect(result).toBe('/external/drive/models/checkpoints/file.safetensors')
})
```

View File

@@ -1,361 +0,0 @@
---
name: ticket-intake
description: 'Parse ticket URL (Notion or GitHub), extract all data, initialize pipeline run. Use when starting work on a new ticket or when asked to pick up a ticket.'
---
# Ticket Intake
Parses a ticket URL from supported sources (Notion or GitHub), extracts all relevant information, and creates a ticket in the pipeline API.
> **🚨 CRITICAL REQUIREMENT**: This skill MUST register the ticket in the Pipeline API and update the source (Notion/GitHub). If these steps are skipped, the entire pipeline breaks. See [Mandatory API Calls](#mandatory-api-calls-execute-all-three) below.
## Supported Sources
| Source | URL Pattern | Provider File |
| ------ | --------------------------------------------------- | --------------------- |
| Notion | `https://notion.so/...` `https://www.notion.so/...` | `providers/notion.md` |
| GitHub | `https://github.com/{owner}/{repo}/issues/{n}` | `providers/github.md` |
## Quick Start
When given a ticket URL:
1. **Detect source type** from URL pattern
2. **Load provider-specific logic** from `providers/` directory
3. Fetch ticket content via appropriate API
4. Extract and normalize properties to common schema
5. **Register ticket in pipeline API** ← MANDATORY
6. **Update source** (Notion status / GitHub comment) ← MANDATORY
7. **Run verification script** to confirm API registration
8. Output summary and handoff to `research-orchestrator`
## Configuration
Uses the **production API** by default. No configuration needed for read operations.
**Defaults (no setup required):**
- API URL: `https://api-gateway-856475788601.us-central1.run.app`
- Read-only endpoints at `/public/*` require no authentication
**For write operations** (transitions, creating tickets), set:
```bash
export PIPELINE_API_KEY="..." # Get from GCP Secret Manager or ask admin
```
**Optional (for local working artifacts):**
```bash
PIPELINE_DIR="${PIPELINE_DIR:-$HOME/repos/ticket-to-pr-pipeline}"
```
## Mandatory API Calls (Execute ALL Three)
**⚠️ These three API calls are the ENTIRE POINT of this skill. Without them, the ticket is invisible to the pipeline, downstream skills will fail, and Notion status won't update.**
**You MUST make these HTTP requests.** Use `curl` from bash — do not just read this as documentation.
### Call 1: Create Ticket
```bash
API_URL="${PIPELINE_API_URL:-https://api-gateway-856475788601.us-central1.run.app}"
API_KEY="${PIPELINE_API_KEY}"
curl -s -X POST "${API_URL}/v1/tickets" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"notion_page_id": "NOTION_PAGE_UUID_HERE",
"title": "TICKET_TITLE_HERE",
"source": "notion",
"metadata": {
"description": "DESCRIPTION_HERE",
"priority": "High",
"labels": [],
"acceptanceCriteria": []
}
}'
```
Save the returned `id` — you need it for the next two calls.
### Call 2: Transition to RESEARCH
```bash
TICKET_ID="id-from-step-1"
curl -s -X POST "${API_URL}/v1/tickets/${TICKET_ID}/transition" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"to_state": "RESEARCH",
"reason": "Intake complete, starting research"
}'
```
### Call 3: Queue Source Update
```bash
curl -s -X POST "${API_URL}/v1/sync/queue" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"ticket_id": "TICKET_ID_HERE",
"action": "update_status",
"payload": { "status": "In Progress" },
"priority": "normal"
}'
```
> **Note:** The action MUST be `"update_status"` (not `"UPDATE_NOTION_STATUS"`). Valid actions: `update_status`, `update_pr_url`, `mark_done`.
### TypeScript Equivalent (if using pipeline client)
```typescript
import { PipelineClient } from '@pipeline/client'
const client = new PipelineClient({
apiUrl:
process.env.PIPELINE_API_URL ||
'https://api-gateway-856475788601.us-central1.run.app',
agentId: process.env.AGENT_ID!
})
const ticket = await client.createTicket({
notion_page_id: pageId,
title: ticketTitle,
source: 'notion',
metadata: { description, priority, labels, acceptanceCriteria }
})
await client.transitionState(
ticket.id,
'RESEARCH',
'Intake complete, starting research'
)
await client.queueSync(ticket.id, 'update_status', { status: 'In Progress' })
```
## Workflow
### Step 1: Detect Source Type
Parse the URL to determine source:
```javascript
if (url.includes('notion.so')) {
source = 'notion'
// Load providers/notion.md
} else if (url.match(/github\.com\/[^\/]+\/[^\/]+\/issues\/\d+/)) {
source = 'github'
// Load providers/github.md
} else {
// Error: Unsupported URL format
}
```
### Step 2: Load Provider and Fetch Data
Read the appropriate provider file for source-specific instructions:
- **Notion**: `providers/notion.md` - Uses Notion MCP, handles Slack links
- **GitHub**: `providers/github.md` - Uses `gh` CLI, handles Dosu comments
Follow the provider's instructions for:
- Fetching content
- Extracting properties
- **Updating the source** (Notion status → "In Progress", Assignee → pipeline owner)
### Step 3: Normalize to Common Schema
All providers must extract normalized ticket data following `schema.md`:
```json
{
"id": "abc12345",
"url": "https://...",
"source": "notion | github",
"title": "Ticket title",
"description": "Full description",
"status": "Not Started",
"assignee": "username",
"priority": "High",
"area": "UI",
"labels": ["bug", "frontend"],
"acceptanceCriteria": ["Criterion 1", "Criterion 2"],
"fetchedAt": "2024-01-15T10:30:00Z"
}
```
### Step 4: Register Ticket in Pipeline API (MANDATORY — DO NOT SKIP)
**Execute all three API calls from [Mandatory API Calls](#mandatory-api-calls-execute-all-three) above.**
This is not optional. This is not documentation. You MUST make these HTTP requests right now.
1. `createTicket()` → save the returned ticket ID
2. `transitionState(id, 'RESEARCH')` → confirm state changed
3. `queueSync(id, 'update_status', { status: 'In Progress' })` → confirm queued
**If any call fails**, retry once. If it still fails, report the error prominently — do NOT silently continue.
### Step 5: Run Verification Script
After making the API calls, run the verification script to confirm everything worked:
```bash
bash scripts/verify-intake.sh TICKET_ID_OR_NOTION_PAGE_ID
```
**If the script is not available locally**, verify manually via the public API:
```bash
curl -s "${API_URL}/public/tickets/${TICKET_ID}" | jq '{id, state, title, notion_page_id}'
```
Expected output:
```json
{
"id": "...",
"state": "RESEARCH",
"title": "...",
"notion_page_id": "..."
}
```
**If `state` is not `RESEARCH`, go back to Step 4 and complete the missing calls.**
### Step 6: Output Summary and Handoff
Print a clear summary:
```markdown
## Ticket Intake Complete
**Source:** Notion | GitHub
**Title:** [Ticket title]
**ID:** abc12345
**Status:** In Progress (queued)
**Priority:** High
**Area:** UI
### Description
[Brief description or first 200 chars]
### Acceptance Criteria
- [ ] Criterion 1
- [ ] Criterion 2
### Links
- **Ticket:** [Original URL]
- **Slack:** [Slack thread content fetched via slackdump] (Notion only)
### Pipeline
- **API Ticket ID:** abc12345
- **State:** RESEARCH
- **Verified:** ✅ (via verify-intake.sh or public API)
```
**After printing the summary, immediately handoff** to continue the pipeline. Use the `handoff` tool with all necessary context (ticket ID, source, title, description, slack context if any):
> **Handoff goal:** "Continue pipeline for ticket {ID} ({title}). Ticket is in RESEARCH state. Load skill: `research-orchestrator` to begin research phase. Ticket data: source={source}, notion_page_id={pageId}, priority={priority}. {slack context summary if available}"
**Do NOT wait for human approval to proceed.** The intake phase is complete — handoff immediately.
## Error Handling
### Unsupported URL
```
❌ Unsupported ticket URL format.
Supported formats:
- Notion: https://notion.so/... or https://www.notion.so/...
- GitHub: https://github.com/{owner}/{repo}/issues/{number}
Received: [provided URL]
```
### Provider-Specific Errors
See individual provider files for source-specific error handling:
- `providers/notion.md` - Authentication, page not found
- `providers/github.md` - Auth, rate limits, issue not found
### Missing Properties
Continue with available data and note what's missing:
```
⚠️ Some properties unavailable:
- Priority: not found (using default: Medium)
- Area: not found
Proceeding with available data...
```
### API Call Failures
```
❌ Pipeline API call failed: {method} {endpoint}
Status: {status}
Error: {message}
Retrying once...
❌ Retry also failed. INTAKE IS INCOMPLETE.
The ticket was NOT registered in the pipeline.
Downstream skills will not work until this is fixed.
```
## Notes
- This skill focuses ONLY on intake — it does not do research
- Slack thread content is fetched automatically via the `slackdump` skill — no manual copy-paste needed
- ALL API calls (createTicket, transitionState, queueSync) are MANDATORY — never skip them
- The `queueSync` action must be `"update_status"`, NOT `"UPDATE_NOTION_STATUS"`
- Pipeline state is tracked via the API, not local files
- Working artifacts (research-report.md, plan.md) can be saved locally to `$PIPELINE_DIR/runs/{ticket-id}/`
- The `source` field in the ticket determines which research strategies to use
## API Client Reference
### Available Methods
| Method | Description |
| ----------------------------------------------------------- | ------------------------------------------------------------------- |
| `createTicket({ notion_page_id, title, source, metadata })` | Create a new ticket in the API |
| `getTicket(id)` | Retrieve a ticket by ID |
| `findByNotionId(notionPageId)` | Look up a ticket by its Notion page ID |
| `listTickets({ state, agent_id, limit, offset })` | List tickets with optional filters |
| `transitionState(id, state, reason)` | Move ticket to a new state (e.g., `'RESEARCH'`) |
| `setPRCreated(id, prUrl)` | Mark ticket as having a PR created |
| `queueSync(id, action, payload)` | Queue a sync action (`update_status`, `update_pr_url`, `mark_done`) |
| `registerBranch(id, branch, repo)` | Register working branch for automatic PR detection |
### Error Handling
```typescript
import { PipelineClient, PipelineAPIError } from '@pipeline/client';
try {
await client.createTicket({ ... });
} catch (error) {
if (error instanceof PipelineAPIError) {
console.error(`API Error ${error.status}: ${error.message}`);
}
throw error;
}
```

View File

@@ -1,194 +0,0 @@
# GitHub Provider - Ticket Intake
Provider-specific logic for ingesting tickets from GitHub Issues.
## URL Pattern
```
https://github.com/{owner}/{repo}/issues/{number}
https://www.github.com/{owner}/{repo}/issues/{number}
```
Extract: `owner`, `repo`, `issue_number` from URL.
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
- Access to the repository
## Fetch Issue Content
Use `gh` CLI to fetch issue details:
```bash
# Get issue details in JSON
gh issue view {number} --repo {owner}/{repo} --json title,body,state,labels,assignees,milestone,author,createdAt,comments,linkedPRs
# Get comments separately if needed
gh issue view {number} --repo {owner}/{repo} --comments
```
## Extract Ticket Data
Map GitHub issue fields to normalized ticket data (stored via API):
| GitHub Field | ticket.json Field | Notes |
| ------------ | ----------------- | -------------------------- |
| title | title | Direct mapping |
| body | description | Issue body/description |
| state | status | Map: open → "Not Started" |
| labels | labels | Array of label names |
| assignees | assignee | First assignee login |
| author | author | Issue author login |
| milestone | milestone | Milestone title if present |
| comments | comments | Array of comment objects |
| linkedPRs | linkedPRs | PRs linked to this issue |
### Priority Mapping
Infer priority from labels:
- `priority:critical`, `P0` → "Critical"
- `priority:high`, `P1` → "High"
- `priority:medium`, `P2` → "Medium"
- `priority:low`, `P3` → "Low"
- No priority label → "Medium" (default)
### Area Mapping
Infer area from labels:
- `area:ui`, `frontend`, `component:*` → "UI"
- `area:api`, `backend` → "API"
- `area:docs`, `documentation` → "Docs"
- `bug`, `fix` → "Bug"
- `enhancement`, `feature` → "Feature"
## Update Source
**For GitHub issues, update is optional but recommended.**
Add a comment to indicate work has started:
```bash
gh issue comment {number} --repo {owner}/{repo} --body "🤖 Pipeline started processing this issue."
```
Optionally assign to self:
```bash
gh issue edit {number} --repo {owner}/{repo} --add-assignee @me
```
Log any updates via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
githubWrites: [
...(ticket.metadata?.githubWrites || []),
{
action: 'comment',
issueNumber: 123,
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
## GitHub-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "github",
"githubOwner": "Comfy-Org",
"githubRepo": "ComfyUI_frontend",
"githubIssueNumber": 123,
"githubIssueUrl": "https://github.com/Comfy-Org/ComfyUI_frontend/issues/123",
"labels": ["bug", "area:ui", "priority:high"],
"linkedPRs": [456, 789],
"dosuComment": "..." // Extracted Dosu bot analysis if present
}
```
## Dosu Bot Detection
Many repositories use Dosu bot for automated issue analysis. Check comments for Dosu:
```bash
gh issue view {number} --repo {owner}/{repo} --comments | grep -A 100 "dosu"
```
Look for comments from:
- `dosu[bot]`
- `dosu-bot`
Extract Dosu analysis which typically includes:
- Root cause analysis
- Suggested files to modify
- Related issues/PRs
- Potential solutions
Store in ticket data via API:
```json
{
"dosuComment": {
"found": true,
"analysis": "...",
"suggestedFiles": ["src/file1.ts", "src/file2.ts"],
"relatedIssues": [100, 101]
}
}
```
## Extract Linked Issues/PRs
Parse issue body and comments for references:
- `#123` → Issue or PR reference
- `fixes #123`, `closes #123` → Linked issue
- `https://github.com/.../issues/123` → Full URL reference
Store in ticket data via API for research phase:
```json
{
"referencedIssues": [100, 101, 102],
"referencedPRs": [200, 201]
}
```
## Error Handling
### Authentication Error
```
⚠️ GitHub CLI not authenticated.
Run: gh auth login
```
### Issue Not Found
```
❌ GitHub issue not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this repository
- Run: gh auth status
```
### Rate Limiting
```
⚠️ GitHub API rate limited.
Wait a few minutes and try again.
Check status: gh api rate_limit
```

View File

@@ -1,202 +0,0 @@
# Notion Provider - Ticket Intake
Provider-specific logic for ingesting tickets from Notion.
## URL Pattern
```
https://www.notion.so/workspace/Page-Title-abc123def456...
https://notion.so/Page-Title-abc123def456...
https://www.notion.so/abc123def456...
```
Page ID is the 32-character hex string (with or without hyphens).
## Prerequisites
- Notion MCP connected and authenticated
- If not setup: `claude mcp add --transport http notion https://mcp.notion.com/mcp`
- Authenticate via `/mcp` command if prompted
## Fetch Ticket Content
Use `Notion:notion-fetch` with the page URL or ID:
```
Fetch the full page content including all properties
```
## Extract Ticket Data
Extract these properties (names may vary):
| Property | Expected Name | Type |
| ------------- | ------------------------- | ------------ |
| Title | Name / Title | Title |
| Status | Status | Select |
| Assignee | Assignee / Assigned To | Person |
| Description | - | Page content |
| Slack Link | Slack Link / Slack Thread | URL |
| GitHub PR | GitHub PR / PR Link | URL |
| Priority | Priority | Select |
| Area | Area / Category | Select |
| Related Tasks | Related Tasks | Relation |
**If properties are missing**: Note what's unavailable and continue with available data.
## Update Source (REQUIRED)
**⚠️ DO NOT SKIP THIS STEP. This is a required action, not optional.**
**⚠️ Notion Write Safety rules apply (see `$PIPELINE_DIR/docs/notion-write-safety.md` for full reference):**
- **Whitelist**: Only `Status`, `GitHub PR`, and `Assignee` fields may be written
- **Valid transitions**: Not Started → In Progress, In Progress → In Review, In Review → Done
- **Logging**: Every write attempt MUST be logged with timestamp, field, value, previous value, skill name, and success status
Use `Notion:notion-update-page` to update the ticket:
1. **Status**: Set to "In Progress" (only valid from "Not Started")
2. **Assignee**: Assign to pipeline owner (Notion ID: `175d872b-594c-81d4-ba5a-0002911c5966`)
```json
{
"page_id": "{page_id_from_ticket}",
"command": "update_properties",
"properties": {
"Status": "In Progress",
"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"
}
}
```
**After the update succeeds**, log the write via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
notionWrites: [
...(ticket.metadata?.notionWrites || []),
{
field: 'Status',
value: 'In Progress',
previousValue: 'Not Started',
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
If update fails, log with `success: false` and continue.
## Notion-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "notion",
"notionPageId": "abc123def456...",
"slackLink": "https://slack.com/...",
"relatedTasks": ["page-id-1", "page-id-2"]
}
```
## Slack Thread Handling
If a Slack link exists, use the `slackdump` skill to fetch the thread content programmatically.
### Slack URL Conversion
Notion stores Slack links in `slackMessage://` format:
```
slackMessage://comfy-organization.slack.com/CHANNEL_ID/THREAD_TS/MESSAGE_TS
```
Convert to browser-clickable format:
```
https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS_NO_DOT
```
**Example:**
- Input: `slackMessage://comfy-organization.slack.com/C075ANWQ8KS/1766022478.450909/1764772881.854829`
- Output: `https://comfy-organization.slack.com/archives/C075ANWQ8KS/p1764772881854829`
(Remove the dot from the last timestamp and prefix with `p`)
### Fetching Thread Content
Load the `slackdump` skill and use the **export-thread** workflow:
```bash
# Export thread by URL
slackdump dump "https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS"
# Or by colon notation (channel_id:thread_ts)
slackdump dump CHANNEL_ID:THREAD_TS
```
Save the thread content to `$RUN_DIR/slack-context.md` and include it in the ticket metadata.
> **No manual action required.** The slackdump CLI handles authentication via stored credentials at `~/.cache/slackdump/comfy-organization.bin`.
## Database Reference: Comfy Tasks
The "Comfy Tasks" database has these properties (verify via `notion-search`):
- **Status values**: Not Started, In Progress, In Review, Done
- **Team assignment**: "Frontend Team" for unassigned tickets
- **Filtering note**: Team filtering in Notion may have quirks - handle gracefully
### Pipeline Owner Details
When assigning tickets, use these identifiers:
| Platform | Identifier |
| --------------- | -------------------------------------- |
| Notion User ID | `175d872b-594c-81d4-ba5a-0002911c5966` |
| Notion Name | Christian Byrne |
| Notion Email | cbyrne@comfy.org |
| Slack User ID | U087MJCDHHC |
| GitHub Username | christian-byrne |
**To update Assignee**, use the Notion User ID (not name):
```
properties: {"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"}
```
### Finding Active Tickets
To list your active tickets:
```
Use Notion:notion-search for "Comfy Tasks"
Filter by Assignee = current user OR Team = "Frontend Team"
```
## Error Handling
### Authentication Error
```
⚠️ Notion authentication required.
Run: claude mcp add --transport http notion https://mcp.notion.com/mcp
Then authenticate via /mcp command.
```
### Page Not Found
```
❌ Notion page not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this page
- Try re-authenticating via /mcp
```

View File

@@ -1,81 +0,0 @@
# Ticket Schema
Common schema for normalized ticket data across all sources. This data is stored and retrieved via the Pipeline API, not local files.
## Ticket Data Schema
```json
{
// Required fields (all sources)
"id": "string", // Unique identifier (short form)
"url": "string", // Original URL
"source": "notion | github", // Source type
"title": "string", // Ticket title
"description": "string", // Full description/body
"fetchedAt": "ISO8601", // When ticket was fetched
// Common optional fields
"status": "string", // Current status
"assignee": "string", // Assigned user
"priority": "string", // Priority level
"area": "string", // Category/area
"labels": ["string"], // Tags/labels
"acceptanceCriteria": ["string"] // List of AC items
// Source-specific fields (see providers)
// Notion: notionPageId, slackLink, relatedTasks, notionWrites
// GitHub: githubOwner, githubRepo, githubIssueNumber, linkedPRs, dosuComment, referencedIssues
}
```
## Ticket State Schema (via API)
State is managed via the Pipeline API using `client.transitionState()`:
```json
{
"ticketId": "string",
"state": "intake | research | planning | implementation | pr_created | done | failed",
"stateChangedAt": "ISO8601",
// Timestamps tracked by API
"createdAt": "ISO8601",
"updatedAt": "ISO8601"
}
```
## Priority Normalization
All sources should normalize to these values:
| Normalized | Description |
| ---------- | ------------------------- |
| Critical | Production down, security |
| High | Blocking work, urgent |
| Medium | Normal priority (default) |
| Low | Nice to have, backlog |
## Status Normalization
Pipeline tracks these statuses internally:
| Status | Description |
| -------------- | ---------------------------- |
| research | Gathering context |
| planning | Creating implementation plan |
| implementation | Writing code |
| review | Code review in progress |
| qa | Quality assurance |
| done | PR merged or completed |
## ID Generation
IDs are generated by the API when creating tickets. For reference:
- **Notion**: First 8 characters of page ID
- **GitHub**: `gh-{owner}-{repo}-{issue_number}` (sanitized)
Examples:
- Notion: `abc12345`
- GitHub: `gh-comfy-org-frontend-123`

View File

@@ -1,143 +0,0 @@
---
name: writing-storybook-stories
description: 'Write or update Storybook stories for Vue components in ComfyUI_frontend. Use when adding, modifying, reviewing, or debugging `.stories.ts` files, Storybook docs, component demos, or visual catalog entries in `src/` or `apps/desktop-ui/`.'
---
# Write Storybook Stories for ComfyUI_frontend
## Workflow
1. !!!!IMPORTANT Confirm the worktree is on a `feat/*` or `fix/*` branch. Base PRs on the local `main`, not a fork branch.
2. Read the component source first. Understand props, emits, slots, exposed methods, and any supporting types or composables.
3. Read nearby stories before writing anything.
- Search stories: `rg --files src apps | rg '\.stories\.ts$'`
- Inspect title patterns: `rg -n "title:\\s*'" src apps --glob '*.stories.ts'`
4. If a Figma link is provided, list the states you need to cover before writing stories.
5. Co-locate the story file with the component: `ComponentName.stories.ts`.
6. Add each variation on separate stories, except hover state. this should be automatically applied by the implementation and not require a separate story.
7. Run Storybook and validation checks before handing off.
## Match Local Conventions
- Copy the closest neighboring story instead of forcing one universal template.
- Most repo stories use `@storybook/vue3-vite`. Some stories under `apps/desktop-ui` still use `@storybook/vue3`; keep the local convention for that area.
- Add `tags: ['autodocs']` unless the surrounding stories in that area intentionally omit it.
- Use `ComponentPropsAndSlots<typeof Component>` when it helps with prop and slot typing.
- Keep `render` functions stateful when needed. Use `ref()`, `computed()`, and `toRefs(args)` instead of mutating Storybook args directly.
- Use `args.default` or other slot-shaped args when the component content is provided through slots.
- Use `ComponentExposed` only when a component's exposed API breaks the normal Storybook typing.
- Add decorators for realistic width or background context when the component needs it.
## Title Patterns
Do not invent titles from scratch when a close sibling story already exists. Match the nearest domain pattern.
| Component area | Typical title pattern |
| ------------------------------------------------------- | ------------------------------------ |
| `src/components/ui/button/Button.vue` | `Components/Button/Button` |
| `src/components/ui/input/Input.vue` | `Components/Input` |
| `src/components/ui/search-input/SearchInput.vue` | `Components/Input/SearchInput` |
| `src/components/common/SearchBox.vue` | `Components/Input/SearchBox` |
| `src/renderer/extensions/vueNodes/widgets/components/*` | `Widgets/<WidgetName>` |
| `src/platform/assets/components/*` | `Platform/Assets/<ComponentName>` |
| `apps/desktop-ui/src/components/*` | `Desktop/Components/<ComponentName>` |
| `apps/desktop-ui/src/views/*` | `Desktop/Views/<ViewName>` |
If multiple patterns seem plausible, follow the closest sibling story in the same folder tree.
## Common Story Shapes
### Stateful input or `v-model`
```typescript
export const Default: Story = {
render: (args) => ({
components: { MyComponent },
setup() {
const { disabled, size } = toRefs(args)
const value = ref('Hello world')
return { value, disabled, size }
},
template:
'<MyComponent v-model="value" :disabled="disabled" :size="size" />'
})
}
```
### Slot-driven content
```typescript
const meta: Meta<ComponentPropsAndSlots<typeof Button>> = {
argTypes: {
default: { control: 'text' }
},
args: {
default: 'Button'
}
}
export const SingleButton: Story = {
render: (args) => ({
components: { Button },
setup() {
return { args }
},
template: '<Button v-bind="args">{{ args.default }}</Button>'
})
}
```
### Variants or edge cases grid
```typescript
export const AllVariants: Story = {
render: () => ({
components: { MyComponent },
template: `
<div class="grid gap-4 sm:grid-cols-2">
<MyComponent />
<MyComponent disabled />
<MyComponent loading />
<MyComponent invalid />
</div>
`
})
}
```
## Figma Mapping
- Extract the named states from the design first.
- Prefer explicit prop-driven stories such as `Disabled`, `Loading`, `Invalid`, `WithPlaceholder`, `AllSizes`, or `EdgeCases`.
- Add an aggregate story such as `AllVariants`, `AllSizes`, or `EdgeCases` when side-by-side comparison is useful.
- Use pseudo-state parameters only if the addon is already configured in this repo.
- If a Figma state cannot be represented exactly, capture the closest prop-driven version and explain the gap in the story docs.
## Component-Specific Notes
- Widget components often need a minimal `SimplifiedWidget` object. Build it in `setup()` and use `computed()` when `args` change `widget.options`.
- Input and search components often need a width-constrained wrapper so they render at realistic sizes.
- Asset and platform cards often need background decorators such as `bg-base-background` and fixed-width containers.
- Desktop installer stories may need custom `backgrounds` parameters and may intentionally keep the older Storybook import style used by neighboring files.
- Use semantic tokens such as `bg-base-background` and `bg-node-component-surface` instead of `dark:` variants or hardcoded theme assumptions.
## Checklist
- [ ] Read the component source and any supporting types or composables
- [ ] Match the nearest local title pattern and story style
- [ ] Include a baseline story; name it `Default` only when that matches nearby conventions
- [ ] Add focused stories for meaningful states
- [ ] Add `tags: ['autodocs']`
- [ ] Keep the story co-located with the component
- [ ] Run `pnpm storybook`
- [ ] Run `pnpm typecheck`
- [ ] Run `pnpm lint`
## Avoid
- Do not guess props, emits, slots, or exposed methods.
- Do not force one generic title convention across the repo.
- Do not mutate Storybook args directly for `v-model` components.
- Do not introduce `dark:` Tailwind variants in story wrappers.
- Do not create barrel files.
- Do not assume every story needs `layout: 'centered'` or a `Default` export; follow the nearest existing pattern.

View File

@@ -1,4 +0,0 @@
interface:
display_name: 'ComfyUI Storybook Stories'
short_description: 'Write Vue Storybook stories for ComfyUI'
default_prompt: 'Use $writing-storybook-stories to add or update a Storybook story for this ComfyUI_frontend component.'

View File

@@ -3,7 +3,6 @@ issue_enrichment:
enabled: true
reviews:
high_level_summary: false
request_changes_workflow: true
auto_review:
drafts: true
ignore_title_keywords:
@@ -13,18 +12,3 @@ reviews:
- comfy-pr-bot
- github-actions
- github-actions[bot]
pre_merge_checks:
override_requested_reviewers_only: true
custom_checks:
- name: End-to-end regression coverage for fixes
mode: error
instructions: |
Use only PR metadata already available in the review context: the PR title, commit subjects in this PR, the files changed in this PR relative to the PR base (equivalent to `base...head`), and the PR description.
Do not rely on shell commands. Do not inspect reverse diffs, files changed only on the base branch, or files outside this PR. If the changed-file list or commit subjects are unavailable, mark the check inconclusive instead of guessing.
Pass if at least one of the following is true:
1. Neither the PR title nor any commit subject in the PR uses bug-fix language such as `fix`, `fixed`, `fixes`, `fixing`, `bugfix`, or `hotfix`.
2. The PR changes at least one file under `browser_tests/`.
3. The PR description includes a concrete, non-placeholder explanation of why an end-to-end regression test was not added.
Fail otherwise. When failing, mention which bug-fix signal you found and ask the author to either add or update a Playwright regression test under `browser_tests/` or add a concrete explanation in the PR description of why an end-to-end regression test is not practical.

2
.gitattributes vendored
View File

@@ -3,6 +3,4 @@
# Generated files
packages/registry-types/src/comfyRegistryTypes.ts linguist-generated=true
packages/ingest-types/src/types.gen.ts linguist-generated=true
packages/ingest-types/src/zod.gen.ts linguist-generated=true
src/workbench/extensions/manager/types/generatedManagerTypes.ts linguist-generated=true

View File

@@ -44,6 +44,7 @@ runs:
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Start ComfyUI server
if: ${{ inputs.launch_server == 'true' }}
@@ -51,8 +52,4 @@ runs:
working-directory: ComfyUI
run: |
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
for i in $(seq 1 300); do
curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1 && echo "Server ready" && exit 0
sleep 2
done
echo "::error::ComfyUI server did not start within 600s" && exit 1
wait-for-it --service 127.0.0.1:8188 -t 600

View File

@@ -19,7 +19,7 @@ runs:
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
node-version: 'lts/*'
cache: 'pnpm'
cache-dependency-path: './pnpm-lock.yaml'

View File

@@ -19,3 +19,217 @@ Workflow files follow a consistent naming pattern: `<prefix>-<descriptive-name>.
Each workflow file contains comments explaining its purpose, triggers, and behavior. For specific details about what each workflow does, refer to the comments at the top of each `.yaml` file.
For GitHub Actions documentation, see [Events that trigger workflows](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows).
## Manual API Changelog Generation
The `manual-api-changelog.yaml` workflow allows you to generate API changelogs by comparing any two versions of the ComfyUI Frontend package.
### Usage
#### Via GitHub Actions UI
1. Go to **Actions** tab in the repository
2. Select **Manual API Changelog Generation** from the workflows list
3. Click **Run workflow** button
4. Fill in the inputs:
- **Previous version**: The earlier version (e.g., `1.29.0` or `v1.29.0`)
- **Current version**: The later version (e.g., `1.30.2` or `v1.30.2`)
- **Create PR**: Check this to automatically create a pull request with the changelog
#### Via GitHub CLI
```bash
# Basic usage - just generate changelog
gh workflow run manual-api-changelog.yaml \
-f from_version=1.29.0 \
-f to_version=1.30.2 \
-f create_pr=false
# Generate changelog and create PR
gh workflow run manual-api-changelog.yaml \
-f from_version=1.29.0 \
-f to_version=1.30.2 \
-f create_pr=true
```
### What It Does
1. **Validates Inputs**: Checks that version formats are valid (X.Y.Z) and tags exist
2. **Builds Both Versions**: Checks out each version tag, installs dependencies, and builds TypeScript types
3. **Generates Snapshots**: Creates structured JSON snapshots of the public API surface for each version
4. **Compares APIs**: Analyzes differences and categorizes as:
- ⚠️ **Breaking changes** (removals, signature changes)
- 🔄 **Modifications** (member additions, type changes)
5. **Uploads Artifact**: Saves the changelog and snapshots as a workflow artifact (90-day retention)
6. **Creates PR** (optional): Generates a draft PR to update `docs/API-CHANGELOG.md`
### Output
#### Workflow Artifacts
Every run produces an artifact containing:
- `CHANGELOG-{from}-to-{to}.md` - Human-readable changelog
- `from.json` - API snapshot of the earlier version
- `to.json` - API snapshot of the later version
**Retention**: 90 days
#### Pull Request (Optional)
If `create_pr` is enabled and changes are detected:
- Creates a draft PR with title: `[docs] API Changelog: v{from} → v{to}`
- Updates `docs/API-CHANGELOG.md` with the new changelog entry
- Includes detailed metadata and review instructions
- Labeled with `documentation`
### Example Changelog Output
```markdown
## v1.30.2 (2025-11-04)
Comparing v1.29.0 → v1.30.2. This changelog documents changes to the public
API surface that third-party extensions and custom nodes depend on.
### ⚠️ Breaking Changes
**Interfaces**
- **Removed**: `ComfyApi.queuePrompt`
### 🔄 Modifications
> **Note**: Some modifications may be breaking changes.
**Interfaces**
- `ComfyApi`
- ✨ Added member: `queuePromptAsync`
- ✨ Added member: `cancelPrompt`
**Enums**
- `NodeStatus`
- ✨ Added enum value: `ERROR`
- ✨ Added enum value: `COMPLETED`
```
### Use Cases
#### 1. Generate Changelog for Missed Releases
If the automatic workflow failed or was skipped for a release:
```bash
gh workflow run manual-api-changelog.yaml \
-f from_version=1.28.0 \
-f to_version=1.29.0 \
-f create_pr=true
```
#### 2. Compare Non-Adjacent Versions
To see cumulative changes across multiple releases:
```bash
gh workflow run manual-api-changelog.yaml \
-f from_version=1.25.0 \
-f to_version=1.30.2 \
-f create_pr=false
```
#### 3. Test Upcoming Changes
Compare current `main` branch against the latest release (requires creating a temporary tag with valid semver format):
```bash
# Create temporary tag for current main
git tag v1.31.0
git push origin v1.31.0
# Run comparison
gh workflow run manual-api-changelog.yaml \
-f from_version=1.30.2 \
-f to_version=1.31.0 \
-f create_pr=false
# Clean up temporary tag
git tag -d v1.31.0
git push origin :refs/tags/v1.31.0
```
#### 4. Audit Historical Changes
Generate changelogs for documentation purposes:
```bash
# Compare multiple version pairs
for from in 1.26.0 1.27.0 1.28.0 1.29.0; do
to=$(echo "$from" | awk -F. '{print $1"."$2+1".0"}')
gh workflow run manual-api-changelog.yaml \
-f from_version=$from \
-f to_version=$to \
-f create_pr=false
done
```
### Validation
The workflow validates:
- ✅ Version format matches semantic versioning (X.Y.Z)
- ✅ Both version tags exist in the repository
- ✅ Tags reference valid commits with buildable code
If validation fails, the workflow exits early with a clear error message.
### Limitations
- **Tag requirement**: Both versions must have corresponding `vX.Y.Z` git tags
- **Build requirement**: Both versions must have functional build processes
- **Type files**: Requires `dist/index.d.ts` to exist after building
- **Scripts**: Requires `scripts/snapshot-api.ts` and `scripts/compare-api-snapshots.ts` to be present
### Related Workflows
- **[Release API Changelogs](./release-api-changelogs.yaml)**: Automatic changelog generation triggered by NPM releases
- **[Release NPM Types](./release-npm-types.yaml)**: Publishes type definitions and triggers automatic changelog
### Troubleshooting
#### "Tag does not exist" error
Ensure the version exists as a git tag:
```bash
git tag -l 'v*' | grep 1.29.0
```
If missing, the version may not have been released yet.
#### "Build failed" error
Check that the version can be built successfully:
```bash
git checkout v1.29.0
pnpm install
pnpm build:types
```
#### No changes detected
If the workflow reports no changes but you expect some:
1. Check the artifact snapshots to verify they're different
2. Ensure you're comparing the correct versions
3. Review the comparison script logic in `scripts/compare-api-snapshots.ts`
#### PR not created
PR creation requires:
- `create_pr` input set to `true`
- Significant changes detected (more than just headers)
- `PR_GH_TOKEN` secret configured with appropriate permissions

View File

@@ -61,22 +61,6 @@ jobs:
git commit -m "[automated] Apply ESLint and Oxfmt fixes"
git push
- name: Fail for fork PRs with unfixed lint/format issues
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name != github.repository
run: |
echo "::error::Linting/formatting issues found. Since this PR is from a fork, auto-fix cannot be applied automatically."
echo ""
echo "Please run these commands locally and push the changes:"
echo " pnpm lint:fix"
echo " pnpm stylelint:fix"
echo " pnpm format"
echo ""
echo "Or set up pre-commit hooks to automatically format on every commit:"
echo " pnpm prepare"
echo ""
echo "See CONTRIBUTING.md for more details."
exit 1
- name: Final validation
run: |
pnpm lint
@@ -100,3 +84,16 @@ jobs:
repo: context.repo.repo,
body: '## 🔧 Auto-fixes Applied\n\nThis PR has been automatically updated to fix linting and formatting issues.\n\n**⚠️ Important**: Your local branch is now behind. Run `git pull` before making additional changes to avoid conflicts.\n\n### Changes made:\n- ESLint auto-fixes\n- Oxfmt formatting'
})
- name: Comment on PR about manual fix needed
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name != github.repository
continue-on-error: true
uses: actions/github-script@v8
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '## ⚠️ Linting/Formatting Issues Found\n\nThis PR has linting or formatting issues that need to be fixed.\n\n**Since this PR is from a fork, auto-fix cannot be applied automatically.**\n\n### Option 1: Set up pre-commit hooks (recommended)\nRun this once to automatically format code on every commit:\n```bash\npnpm prepare\n```\n\n### Option 2: Fix manually\nRun these commands and push the changes:\n```bash\npnpm lint:fix\npnpm format\n```\n\nSee [CONTRIBUTING.md](https://github.com/Comfy-Org/ComfyUI_frontend/blob/main/CONTRIBUTING.md#git-pre-commit-hooks) for more details.'
})

View File

@@ -10,7 +10,7 @@ on:
concurrency:
group: perf-${{ github.ref }}
cancel-in-progress: false
cancel-in-progress: true
permissions:
contents: read
@@ -26,15 +26,12 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
permissions:
contents: write
contents: read
packages: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Setup frontend
uses: ./.github/actions/setup-frontend
@@ -71,44 +68,3 @@ jobs:
with:
name: perf-meta
path: temp/perf-meta/
- name: Save perf baseline to perf-data branch
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.perf.outcome == 'success'
continue-on-error: true
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git config url."https://x-access-token:${GH_TOKEN}@github.com/".insteadOf "https://github.com/"
cp test-results/perf-metrics.json /tmp/perf-metrics.json
git fetch origin perf-data || {
echo "Creating perf-data branch"
git checkout --orphan perf-data
git rm -rf . 2>/dev/null || true
echo "# Performance Baselines" > README.md
mkdir -p baselines
git add README.md baselines
git commit -m "Initialize perf-data branch"
git push origin perf-data
git fetch origin perf-data
}
git worktree add /tmp/perf-data origin/perf-data
TIMESTAMP=$(date -u +%Y%m%dT%H%M%SZ)
SHA=$(echo "${{ github.sha }}" | cut -c1-8)
mkdir -p /tmp/perf-data/baselines
cp /tmp/perf-metrics.json "/tmp/perf-data/baselines/perf-${TIMESTAMP}-${SHA}.json"
# Keep only last 20 baselines
cd /tmp/perf-data
ls -t baselines/perf-*.json 2>/dev/null | tail -n +21 | xargs -r rm
git -C /tmp/perf-data add baselines/
git -C /tmp/perf-data commit -m "perf: add baseline for ${SHA}" || echo "No changes to commit"
git -C /tmp/perf-data push origin HEAD:perf-data
git worktree remove /tmp/perf-data --force 2>/dev/null || true
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -0,0 +1,255 @@
name: Manual API Changelog Generation
on:
workflow_dispatch:
inputs:
from_version:
description: 'Previous version (e.g., 1.29.0 or v1.29.0)'
required: true
type: string
to_version:
description: 'Current version (e.g., 1.30.2 or v1.30.2)'
required: true
type: string
create_pr:
description: 'Create a pull request with the changelog'
required: false
type: boolean
default: false
concurrency:
group: manual-api-changelog-${{ github.run_id }}
cancel-in-progress: false
jobs:
generate_changelog:
name: Generate API Changelog
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
with:
fetch-depth: 0 # Fetch all history for comparing versions
- name: Validate version inputs
id: validate_versions
run: |
# Normalize version strings (remove 'v' prefix if present)
FROM_VERSION="${{ github.event.inputs.from_version }}"
TO_VERSION="${{ github.event.inputs.to_version }}"
FROM_VERSION=${FROM_VERSION#v}
TO_VERSION=${TO_VERSION#v}
# Validate version format (semantic versioning)
if ! [[ "$FROM_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Error: Invalid from_version format: $FROM_VERSION"
echo "Expected format: X.Y.Z (e.g., 1.29.0)"
exit 1
fi
if ! [[ "$TO_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Error: Invalid to_version format: $TO_VERSION"
echo "Expected format: X.Y.Z (e.g., 1.30.2)"
exit 1
fi
# Check if tags exist
if ! git rev-parse "v$FROM_VERSION" >/dev/null 2>&1; then
echo "Error: Tag v$FROM_VERSION does not exist"
exit 1
fi
if ! git rev-parse "v$TO_VERSION" >/dev/null 2>&1; then
echo "Error: Tag v$TO_VERSION does not exist"
exit 1
fi
echo "from_version=$FROM_VERSION" >> $GITHUB_OUTPUT
echo "to_version=$TO_VERSION" >> $GITHUB_OUTPUT
echo "from_tag=v$FROM_VERSION" >> $GITHUB_OUTPUT
echo "to_tag=v$TO_VERSION" >> $GITHUB_OUTPUT
echo "✅ Validated versions: v$FROM_VERSION → v$TO_VERSION"
- name: Install pnpm
uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4.3.0
with:
version: 10
- name: Setup Node.js
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
- name: Create snapshots directory
run: mkdir -p .api-snapshots
- name: Preserve scripts
run: |
# Copy scripts to temporary location
mkdir -p /tmp/api-changelog-scripts
cp scripts/snapshot-api.ts scripts/compare-api-snapshots.ts /tmp/api-changelog-scripts/
- name: Build and snapshot TO version
run: |
echo "Building types for v${{ steps.validate_versions.outputs.to_version }}"
git checkout ${{ steps.validate_versions.outputs.to_tag }}
# Restore scripts
mkdir -p scripts
cp /tmp/api-changelog-scripts/*.ts scripts/
pnpm install --frozen-lockfile
pnpm build:types
# Generate snapshot
node --loader tsx scripts/snapshot-api.ts dist/index.d.ts > /tmp/api-snapshots-to.json
echo "✅ Created snapshot for v${{ steps.validate_versions.outputs.to_version }}"
- name: Build and snapshot FROM version
run: |
echo "Building types for v${{ steps.validate_versions.outputs.from_version }}"
git checkout ${{ steps.validate_versions.outputs.from_tag }}
# Restore scripts
mkdir -p scripts
cp /tmp/api-changelog-scripts/*.ts scripts/
pnpm install --frozen-lockfile
pnpm build:types
# Generate snapshot
node --loader tsx scripts/snapshot-api.ts dist/index.d.ts > /tmp/api-snapshots-from.json
echo "✅ Created snapshot for v${{ steps.validate_versions.outputs.from_version }}"
- name: Return to original branch
run: |
git checkout ${{ github.ref_name }}
# Restore scripts
mkdir -p scripts
cp /tmp/api-changelog-scripts/*.ts scripts/
# Copy snapshots to working directory
cp /tmp/api-snapshots-from.json .api-snapshots/from.json
cp /tmp/api-snapshots-to.json .api-snapshots/to.json
- name: Compare API snapshots and generate changelog
id: generate_changelog
run: |
# Get git ref for TO version
GIT_REF=$(git rev-parse ${{ steps.validate_versions.outputs.to_tag }})
# Run the comparison script
CHANGELOG_OUTPUT=$(node --loader tsx scripts/compare-api-snapshots.ts \
.api-snapshots/from.json \
.api-snapshots/to.json \
${{ steps.validate_versions.outputs.from_version }} \
${{ steps.validate_versions.outputs.to_version }} \
Comfy-Org \
ComfyUI_frontend \
"$GIT_REF")
# Save changelog to file for artifact
echo "$CHANGELOG_OUTPUT" > .api-snapshots/CHANGELOG-${{ steps.validate_versions.outputs.from_version }}-to-${{ steps.validate_versions.outputs.to_version }}.md
# Also output to step summary
echo "## 📊 Generated API Changelog" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "$CHANGELOG_OUTPUT" >> $GITHUB_STEP_SUMMARY
# Check if changelog is empty or just header
if [ $(echo "$CHANGELOG_OUTPUT" | wc -l) -lt 5 ]; then
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "⚠️ No significant API changes detected" >> $GITHUB_STEP_SUMMARY
else
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "✅ API changes detected and documented" >> $GITHUB_STEP_SUMMARY
fi
echo "✅ Changelog generated successfully"
- name: Upload changelog artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: api-changelog-v${{ steps.validate_versions.outputs.from_version }}-to-v${{ steps.validate_versions.outputs.to_version }}
path: |
.api-snapshots/CHANGELOG-*.md
.api-snapshots/from.json
.api-snapshots/to.json
retention-days: 90
- name: Create Pull Request
if: github.event.inputs.create_pr == 'true' && steps.generate_changelog.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e
with:
token: ${{ secrets.PR_GH_TOKEN }}
commit-message: '[docs] Update API changelog for v${{ steps.validate_versions.outputs.from_version }} → v${{ steps.validate_versions.outputs.to_version }}'
title: '[docs] API Changelog: v${{ steps.validate_versions.outputs.from_version }} → v${{ steps.validate_versions.outputs.to_version }}'
body: |
## API Changelog Update (Manual)
This PR documents public API changes between v${{ steps.validate_versions.outputs.from_version }} and v${{ steps.validate_versions.outputs.to_version }}.
The changelog has been manually generated by comparing TypeScript type definitions between versions.
### Version Comparison
- **From:** v${{ steps.validate_versions.outputs.from_version }}
- **To:** v${{ steps.validate_versions.outputs.to_version }}
- **Requested by:** @${{ github.actor }}
- **Workflow run:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
### Review Instructions
- Review the changes in `docs/API-CHANGELOG.md`
- Verify accuracy of breaking changes
- Add any additional context or migration notes if needed
- Merge when ready to publish changelog
### Artifacts
The full changelog and snapshots are available as workflow artifacts for 90 days.
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
branch: api-changelog-manual-${{ steps.validate_versions.outputs.from_version }}-to-${{ steps.validate_versions.outputs.to_version }}
base: ${{ github.ref_name }}
labels: documentation
delete-branch: true
draft: true
add-paths: |
docs/API-CHANGELOG.md
- name: Summary
run: |
echo "## 🎉 Workflow Complete" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- **From version:** v${{ steps.validate_versions.outputs.from_version }}" >> $GITHUB_STEP_SUMMARY
echo "- **To version:** v${{ steps.validate_versions.outputs.to_version }}" >> $GITHUB_STEP_SUMMARY
echo "- **Changes detected:** ${{ steps.generate_changelog.outputs.has_changes }}" >> $GITHUB_STEP_SUMMARY
echo "- **Create PR:** ${{ github.event.inputs.create_pr }}" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### 📦 Artifact" >> $GITHUB_STEP_SUMMARY
echo "The generated changelog and API snapshots have been uploaded as artifacts." >> $GITHUB_STEP_SUMMARY
echo "Retention: 90 days" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ github.event.inputs.create_pr }}" == "true" ] && [ "${{ steps.generate_changelog.outputs.has_changes }}" == "true" ]; then
echo "### 🔀 Pull Request" >> $GITHUB_STEP_SUMMARY
echo "A draft pull request has been created with the changelog updates." >> $GITHUB_STEP_SUMMARY
elif [ "${{ github.event.inputs.create_pr }}" == "true" ]; then
echo "### No PR Created" >> $GITHUB_STEP_SUMMARY
echo "No significant changes were detected, so no PR was created." >> $GITHUB_STEP_SUMMARY
else
echo "### PR Creation Skipped" >> $GITHUB_STEP_SUMMARY
echo "Pull request creation was not requested. Enable 'Create PR' option to automatically create a PR." >> $GITHUB_STEP_SUMMARY
fi

View File

@@ -10,7 +10,6 @@ permissions:
contents: read
pull-requests: write
issues: write
actions: read
jobs:
comment:
@@ -74,28 +73,7 @@ jobs:
core.setOutput('number', String(pr.number));
core.setOutput('base', trustedBase);
- name: Check if results are still current
id: sha-check
uses: actions/github-script@v8
with:
script: |
const prNumber = Number('${{ steps.pr-meta.outputs.number }}');
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
const runSha = context.payload.workflow_run.head_sha;
const currentSha = pr.head.sha;
if (runSha !== currentSha) {
core.info(`Skipping stale report: run SHA ${runSha} != current PR SHA ${currentSha}`);
core.setOutput('stale', 'true');
} else {
core.setOutput('stale', 'false');
}
- name: Download PR perf metrics
if: steps.sha-check.outputs.stale != 'true'
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
name: perf-metrics
@@ -103,7 +81,6 @@ jobs:
path: test-results/
- name: Download baseline perf metrics
if: steps.sha-check.outputs.stale != 'true'
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
branch: ${{ steps.pr-meta.outputs.base }}
@@ -113,33 +90,10 @@ jobs:
path: temp/perf-baseline/
if_no_artifact_found: warn
- name: Load historical baselines from perf-data branch
if: steps.sha-check.outputs.stale != 'true'
continue-on-error: true
run: |
mkdir -p temp/perf-history
git fetch origin perf-data 2>/dev/null || {
echo "perf-data branch not found, skipping historical data"
exit 0
}
INDEX=0
for file in $(git ls-tree --name-only origin/perf-data baselines/ 2>/dev/null | sort -r | head -5); do
DIR="temp/perf-history/$INDEX"
mkdir -p "$DIR"
git show "origin/perf-data:${file}" > "$DIR/perf-metrics.json" 2>/dev/null || true
INDEX=$((INDEX + 1))
done
echo "Loaded $INDEX historical baselines"
- name: Generate perf report
if: steps.sha-check.outputs.stale != 'true'
run: npx --yes tsx scripts/perf-report.ts > perf-report.md
- name: Post PR comment
if: steps.sha-check.outputs.stale != 'true'
uses: ./.github/actions/post-pr-report-comment
with:
pr-number: ${{ steps.pr-meta.outputs.number }}

View File

@@ -1,564 +0,0 @@
# Automated QA of ComfyUI frontend using Claude CLI + Playwright MCP.
# Two modes:
# Focused (qa-changes label): Linux-only, tests areas affected by PR changes
# Full (qa-full label): 3-OS matrix, full test plan
name: 'PR: QA'
on:
pull_request:
types: [labeled]
branches: [main]
workflow_dispatch:
inputs:
mode:
description: 'QA mode'
type: choice
options: [focused, full]
default: focused
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
resolve-matrix:
runs-on: ubuntu-latest
outputs:
os: ${{ steps.set.outputs.os }}
mode: ${{ steps.set.outputs.mode }}
skip: ${{ steps.set.outputs.skip }}
steps:
- name: Determine QA mode
id: set
env:
LABEL: ${{ github.event.label.name }}
EVENT_ACTION: ${{ github.event.action }}
EVENT_NAME: ${{ github.event_name }}
INPUT_MODE: ${{ inputs.mode }}
run: |
FULL=false
# Only run on label events if it's one of our labels
if [ "$EVENT_ACTION" = "labeled" ] && \
[ "$LABEL" != "qa-changes" ] && [ "$LABEL" != "qa-full" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
fi
# Full QA triggers
if [ "$EVENT_NAME" = "workflow_dispatch" ] && \
[ "$INPUT_MODE" = "full" ]; then
FULL=true
fi
if [ "$LABEL" = "qa-full" ]; then
FULL=true
fi
if [ "$FULL" = "true" ]; then
echo 'os=["ubuntu-latest","macos-latest","windows-latest"]' >> "$GITHUB_OUTPUT"
echo "mode=full" >> "$GITHUB_OUTPUT"
else
echo 'os=["ubuntu-latest"]' >> "$GITHUB_OUTPUT"
echo "mode=focused" >> "$GITHUB_OUTPUT"
fi
echo "Mode: $([ "$FULL" = "true" ] && echo full || echo focused)"
qa:
needs: resolve-matrix
if: needs.resolve-matrix.outputs.skip != 'true'
strategy:
fail-fast: false
matrix:
os: ${{ fromJson(needs.resolve-matrix.outputs.os) }}
runs-on: ${{ matrix.os }}
timeout-minutes: 60
permissions:
contents: write
pull-requests: write
env:
QA_MODE: ${{ needs.resolve-matrix.outputs.mode }}
steps:
- name: Set QA artifacts path
shell: bash
run: echo "QA_ARTIFACTS=$RUNNER_TEMP/qa-artifacts" >> "$GITHUB_ENV"
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
fetch-depth: 0
ref: ${{ github.head_ref || github.ref }}
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup frontend
uses: ./.github/actions/setup-frontend
with:
include_build_step: true
- name: Setup and start ComfyUI server
uses: ./.github/actions/setup-comfyui-server
with:
launch_server: 'true'
- name: Wait for ComfyUI server
shell: bash
run: |
echo "Waiting for ComfyUI server..."
for i in $(seq 1 60); do
if curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1; then
echo "Server ready"; exit 0
fi; sleep 2
done
echo "::error::Server timeout"; exit 1
- name: Install Playwright for MCP
shell: bash
run: pnpm exec playwright install chromium --with-deps
- name: Install Claude Code
shell: bash
run: npm install -g @anthropic-ai/claude-code@2.1.71
# --- Virtual display + recording: Linux ---
- name: Setup Xvfb (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get update -qq && sudo apt-get install -y -qq xvfb ffmpeg x11-xserver-utils >/dev/null 2>&1
Xvfb :99 -screen 0 1280x720x24 -ac -nolisten tcp &
echo $! > "${{ runner.temp }}/xvfb.pid"
sleep 2
echo "DISPLAY=:99" >> "$GITHUB_ENV"
# Set black background and normal cursor (removes the default X cursor)
DISPLAY=:99 xsetroot -solid black -cursor_name left_ptr 2>/dev/null || true
- name: Start recording (Linux)
if: runner.os == 'Linux'
continue-on-error: true
run: |
mkdir -p "$QA_ARTIFACTS"
ffmpeg -y -f x11grab -video_size 1280x720 -framerate 10 \
-i :99.0 -c:v libx264 -preset ultrafast -crf 28 \
-pix_fmt yuv420p "$QA_ARTIFACTS/qa-session.mp4" &
echo $! > "${{ runner.temp }}/ffmpeg.pid"
sleep 1
# --- Recording: macOS (avfoundation) ---
- name: Install ffmpeg (macOS)
if: runner.os == 'macOS'
run: brew install ffmpeg
- name: Grant screen recording permission (macOS)
if: runner.os == 'macOS'
run: |
# Grant kTCCServiceScreenCapture to ffmpeg and bash so avfoundation
# recording works without the blocking permission dialog.
FFMPEG_PATH=$(which ffmpeg)
echo "Granting screen recording permission to: $FFMPEG_PATH"
# macOS 14+ (Sonoma/Sequoia) TCC.db schema has 17 columns
for BIN in "$FFMPEG_PATH" "/bin/bash"; do
sudo sqlite3 "/Library/Application Support/com.apple.TCC/TCC.db" \
"INSERT OR REPLACE INTO access VALUES('kTCCServiceScreenCapture','${BIN}',1,2,0,1,NULL,NULL,0,'UNUSED',NULL,0,$(date +%s),NULL,NULL,NULL,NULL);" 2>/dev/null \
|| sudo sqlite3 "/Library/Application Support/com.apple.TCC/TCC.db" \
"INSERT OR REPLACE INTO access VALUES('kTCCServiceScreenCapture','${BIN}',1,2,0,1,NULL,NULL,NULL,'UNUSED',NULL,0,$(date +%s));" 2>/dev/null \
|| echo "Warning: Could not grant TCC permission to ${BIN}"
done
- name: Start recording (macOS)
if: runner.os == 'macOS'
continue-on-error: true
run: |
mkdir -p "$QA_ARTIFACTS"
ffmpeg -f avfoundation -list_devices true -i "" 2>&1 || true
ffmpeg -y -f avfoundation -framerate 10 -capture_cursor 1 \
-i "Capture screen 0:none" -c:v libx264 -preset ultrafast -crf 28 \
-pix_fmt yuv420p "$QA_ARTIFACTS/qa-session.mp4" &
echo $! > "${{ runner.temp }}/ffmpeg.pid"
sleep 2
# --- Recording: Windows (gdigrab) ---
- name: Install ffmpeg (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: choco install ffmpeg -y --no-progress
- name: Start recording (Windows)
if: runner.os == 'Windows'
continue-on-error: true
shell: bash
run: |
mkdir -p "$QA_ARTIFACTS"
ffmpeg -y -f gdigrab -framerate 10 -i desktop \
-c:v libx264 -preset ultrafast -crf 28 \
-pix_fmt yuv420p "$QA_ARTIFACTS/qa-session.mp4" &
echo $! > "${{ runner.temp }}/ffmpeg.pid"
sleep 1
- name: Create MCP config
shell: bash
run: |
cat > "${{ runner.temp }}/mcp-config.json" <<EOF
{"mcpServers":{"playwright":{"command":"npx","args":["@playwright/mcp@0.0.68"]}}}
EOF
- name: Get PR diff for focused QA
if: needs.resolve-matrix.outputs.mode == 'focused'
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh pr diff ${{ github.event.pull_request.number || '' }} \
--repo ${{ github.repository }} > "${{ runner.temp }}/pr-diff.txt" 2>/dev/null || \
git diff origin/main...HEAD > "${{ runner.temp }}/pr-diff.txt"
# Summarize changed files for the prompt
echo "Changed files:"
grep '^diff --git' "${{ runner.temp }}/pr-diff.txt" | \
sed 's|diff --git a/||;s| b/.*||' | sort -u | tee "${{ runner.temp }}/changed-files.txt"
- name: Write QA prompt
shell: bash
env:
BRANCH: ${{ github.head_ref || github.ref_name }}
PR_NUM: ${{ github.event.pull_request.number || 'N/A' }}
SHA: ${{ github.sha }}
run: |
OS_LOWER=$(echo "$RUNNER_OS" | tr '[:upper:]' '[:lower:]')
if [ "$QA_MODE" = "full" ]; then
cat > "${{ runner.temp }}/qa-prompt.txt" <<PROMPT
You are running a FULL automated QA pass on the ComfyUI frontend.
Read the file .claude/skills/comfy-qa/SKILL.md and follow the FULL QA test plan.
Environment: CI=true, OS=${{ runner.os }}
Server URL: http://127.0.0.1:8188
Branch: ${BRANCH}
PR: #${PR_NUM}
Commit: ${SHA}
1. Use playwright MCP tools to navigate http://127.0.0.1:8188
2. Run the FULL QA test plan from the skill file
3. Take screenshots of failures or notable states
4. Save report to docs/qa/ as YYYY-MM-DD-NNN-${OS_LOWER}-report.md
5. Commit and push the report to this branch
Do NOT create a new PR. Do NOT post PR comments.
Skip tests not available in CI (file dialogs, GPU execution).
PROMPT
else
cat > "${{ runner.temp }}/qa-prompt.txt" <<PROMPT
You are running a FOCUSED QA pass on a pull request to the ComfyUI frontend.
Your goal is to verify that the changes in this PR work correctly and don't break related functionality.
Environment: CI=true, OS=${{ runner.os }}
Server URL: http://127.0.0.1:8188
Branch: ${BRANCH}
PR: #${PR_NUM}
Commit: ${SHA}
CHANGED FILES:
$(cat "${{ runner.temp }}/changed-files.txt" 2>/dev/null || echo "Unknown")
DIFF (truncated to 500 lines):
$(head -500 "${{ runner.temp }}/pr-diff.txt" 2>/dev/null || echo "No diff available")
Instructions:
1. Read the diff above to understand what changed in this PR
2. Use playwright MCP tools to navigate http://127.0.0.1:8188
3. Test the specific UI areas affected by these changes
4. Also do a quick smoke test of core functionality (app loads, canvas renders, sidebar works)
5. Take screenshots of any failures or the areas you tested
6. Save a concise report to docs/qa/ as YYYY-MM-DD-NNN-${OS_LOWER}-report.md
7. Commit and push the report to this branch
Focus on:
- Does the changed functionality work as expected?
- Are there visual regressions in affected areas?
- Do related features still work?
Do NOT run the full QA test plan. Do NOT create a new PR. Do NOT post PR comments.
Skip tests not available in CI (file dialogs, GPU execution).
PROMPT
fi
- name: Run Claude QA
shell: bash
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
CI: 'true'
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
MAX_TURNS=128
if [ "$QA_MODE" = "focused" ]; then MAX_TURNS=30; fi
cat "${{ runner.temp }}/qa-prompt.txt" | claude --print --verbose \
--max-turns "$MAX_TURNS" \
--mcp-config "${{ runner.temp }}/mcp-config.json" \
--allowedTools "mcp__playwright__browser_navigate,mcp__playwright__browser_snapshot,mcp__playwright__browser_click,mcp__playwright__browser_type,mcp__playwright__browser_press_key,mcp__playwright__browser_take_screenshot,mcp__playwright__browser_hover,mcp__playwright__browser_drag,mcp__playwright__browser_select_option,mcp__playwright__browser_handle_dialog,mcp__playwright__browser_tab_list,mcp__playwright__browser_tab_new,mcp__playwright__browser_tab_select,mcp__playwright__browser_tab_close,mcp__playwright__browser_console_messages,mcp__playwright__browser_resize,mcp__playwright__browser_wait_for,Bash(git add:*),Bash(git commit:*),Bash(git push:*),Bash(git status:*),Bash(git log:*),Bash(git diff:*),Bash(date:*),Bash(ls:*),Bash(mkdir:*),Read,Write,Edit,Glob,Grep"
- name: Stop recording
if: always()
shell: bash
run: |
PID_FILE="${{ runner.temp }}/ffmpeg.pid"
if [ -f "$PID_FILE" ]; then
if [ "$RUNNER_OS" = "Windows" ]; then
# Graceful stop: taskkill without /F sends WM_CLOSE so ffmpeg
# can finalize the mp4 (write moov atom). Force-kill as fallback.
taskkill //PID $(cat "$PID_FILE") 2>/dev/null || true
sleep 5
taskkill //F //PID $(cat "$PID_FILE") 2>/dev/null || true
else
kill -INT $(cat "$PID_FILE") 2>/dev/null || true
sleep 3; kill $(cat "$PID_FILE") 2>/dev/null || true
fi
fi
[ -f "$QA_ARTIFACTS/qa-session.mp4" ] && \
echo "Video: $(du -h "$QA_ARTIFACTS/qa-session.mp4" | cut -f1)" || \
echo "No video (non-fatal)"
- name: Collect artifacts
if: always()
shell: bash
run: |
mkdir -p "$QA_ARTIFACTS"
cp -r docs/qa/* "$QA_ARTIFACTS/" 2>/dev/null || true
ls -la "$QA_ARTIFACTS/" || true
- name: Upload QA artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v6.2.0
with:
name: qa-report-${{ runner.os }}-${{ github.run_id }}
path: ${{ env.QA_ARTIFACTS }}/
retention-days: 14
- name: Cleanup (Linux)
if: always() && runner.os == 'Linux'
run: kill $(cat "${{ runner.temp }}/xvfb.pid") 2>/dev/null || true
report:
needs: [resolve-matrix, qa]
if: always() && github.event.pull_request.number && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Download QA artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
path: qa-artifacts
pattern: qa-report-*
- name: Install ffmpeg
run: sudo apt-get update -qq && sudo apt-get install -y -qq ffmpeg >/dev/null 2>&1
- name: Deploy videos to Cloudflare Pages
id: deploy-videos
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
RAW_BRANCH: ${{ github.head_ref || github.ref_name }}
run: |
npm install -g wrangler@^4.0.0 >/dev/null 2>&1
DEPLOY_DIR=$(mktemp -d)
mkdir -p "$DEPLOY_DIR"
for os in Linux macOS Windows; do
VID="qa-artifacts/qa-report-${os}-${{ github.run_id }}/qa-session.mp4"
if [ -f "$VID" ]; then
cp "$VID" "$DEPLOY_DIR/qa-${os}.mp4"
echo "Found ${os} video ($(du -h "$VID" | cut -f1))"
# Generate GIF thumbnail: 8s starting at 10s, 480px wide, 8fps
ffmpeg -y -ss 10 -i "$VID" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| ffmpeg -y -i "$VID" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| echo "GIF generation failed for ${os} (non-fatal)"
fi
done
# Build video cards HTML
CARDS=""
ICONS_Linux="&#x1F427;" ICONS_macOS="&#x1F34E;" ICONS_Windows="&#x1FA9F;"
for os in Linux macOS Windows; do
eval "ICON=\$ICONS_${os}"
if [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
CARDS="${CARDS}<div class=card><video controls autoplay muted loop preload=metadata><source src=qa-${os}.mp4 type=video/mp4></video><div class=card-body><span class=platform><span class=icon>${ICON}</span> ${os}</span><a class=download href=qa-${os}.mp4 download>Download</a></div></div>"
else
CARDS="${CARDS}<div class=card><div class=empty-card>No recording available</div><div class=card-body><span class=platform><span class=icon>${ICON}</span> ${os}</span><span class='badge missing'>Missing</span></div></div>"
fi
done
cat > "$DEPLOY_DIR/index.html" <<INDEXEOF
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
<style>
*{margin:0;padding:0;box-sizing:border-box}body{background:#0d1117;color:#e6edf3;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,Arial,sans-serif;min-height:100vh;padding:2rem 1rem}.container{max-width:1200px;margin:0 auto}header{display:flex;align-items:center;gap:.75rem;margin-bottom:2rem;padding-bottom:1rem;border-bottom:1px solid #30363d}h1{font-size:1.5rem;font-weight:600}.meta{color:#8b949e;font-size:.875rem;margin-top:.25rem}.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(340px,1fr));gap:1.25rem}.card{background:#161b22;border:1px solid #30363d;border-radius:.5rem;overflow:hidden;transition:border-color .15s}.card:hover{border-color:#58a6ff}.card video{width:100%;display:block;background:#010409;aspect-ratio:16/9;object-fit:contain}.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}.platform{display:flex;align-items:center;gap:.5rem;font-weight:500}.icon{font-size:1.25rem}.badge{font-size:.75rem;padding:.125rem .5rem;border-radius:999px;background:#1f6feb33;color:#58a6ff;border:1px solid #1f6feb55}.badge.missing{background:#da363333;color:#f85149;border-color:#da363355}.empty-card{display:flex;align-items:center;justify-content:center;min-height:200px;color:#484f58;font-size:.875rem}a.download{color:#58a6ff;text-decoration:none;font-size:.8125rem}a.download:hover{text-decoration:underline}
</style></head><body><div class=container>
<header><svg width=28 height=28 viewBox="0 0 24 24" fill=none stroke=#58a6ff stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend &middot; Automated QA</div></div></header>
<div class=grid>${CARDS}</div>
</div></body></html>
INDEXEOF
# 404 page so Cloudflare Pages returns proper 404 for missing files
# (instead of SPA fallback serving index.html)
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
<!DOCTYPE html><html><head><meta charset=utf-8><title>404</title>
<style>body{background:#0d1117;color:#8b949e;font-family:sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh;margin:0}div{text-align:center}h1{color:#f85149;font-size:3rem;margin-bottom:.5rem}p{font-size:1rem}</style>
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
ERROREOF
# Sanitize branch name for Cloudflare Pages URL (same rules CF uses)
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
URL=$(wrangler pages deploy "$DEPLOY_DIR" \
--project-name="comfyui-qa-videos" \
--branch="$BRANCH" 2>&1 \
| grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1)
echo "url=${URL:-https://${BRANCH}.comfyui-qa-videos.pages.dev}" >> "$GITHUB_OUTPUT"
echo "Deployed to: ${URL}"
- name: Post QA comment on PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VIDEO_BASE: ${{ steps.deploy-videos.outputs.url }}
QA_MODE: ${{ needs.resolve-matrix.outputs.mode }}
run: |
RUN="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
COMMENT_MARKER="<!-- QA_REPORT_COMMENT -->"
MODE_BADGE="🔍 Focused"
if [ "$QA_MODE" = "full" ]; then MODE_BADGE="🔬 Full (3-OS)"; fi
# Build video section with GIF thumbnails linking to full videos
VIDEO_SECTION=""
for os in Linux macOS Windows; do
GIF_URL="${VIDEO_BASE}/qa-${os}-thumb.gif"
VID_URL="${VIDEO_BASE}/qa-${os}.mp4"
if curl -sf --head "$VID_URL" >/dev/null 2>&1; then
if curl -sf --head "$GIF_URL" >/dev/null 2>&1; then
VIDEO_SECTION="${VIDEO_SECTION}[![${os} QA](${GIF_URL})](${VID_URL})"$'\n'
else
VIDEO_SECTION="${VIDEO_SECTION}[${os} video](${VID_URL})"$'\n'
fi
fi
done
BODY=$(cat <<EOF
${COMMENT_MARKER}
## QA ${MODE_BADGE}
${VIDEO_SECTION}
**Run**: [${RUN}](${RUN}) · [Download artifacts](${RUN}#artifacts) · [All videos](${VIDEO_BASE})
EOF
)
EXISTING=$(gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" \
--jq ".[] | select(.body | contains(\"${COMMENT_MARKER}\")) | .id" | head -1)
if [ -n "$EXISTING" ]; then
gh api --method PATCH "repos/${{ github.repository }}/issues/comments/${EXISTING}" \
--field body="$BODY"
else
gh pr comment ${{ github.event.pull_request.number }} \
--repo ${{ github.repository }} --body "$BODY"
fi
- name: Remove QA label
if: >-
github.event.label.name == 'qa-changes' ||
github.event.label.name == 'qa-full'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
LABEL: ${{ github.event.label.name }}
run: |
gh pr edit ${{ github.event.pull_request.number }} \
--repo ${{ github.repository }} --remove-label "$LABEL"
video-review:
needs: [qa, report]
if: always() && github.event.pull_request.number && github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Setup Node.js
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version: 24
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
with:
version: 10
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Install ffmpeg
run: sudo apt-get update -qq && sudo apt-get install -y -qq ffmpeg >/dev/null 2>&1
- name: Download QA artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
path: qa-artifacts
pattern: qa-report-*
- name: Run video review
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
mkdir -p video-reviews
for vid in qa-artifacts/qa-report-*/qa-session.mp4; do
[ -f "$vid" ] || continue
echo "::group::Reviewing $vid"
pnpm exec tsx scripts/qa-video-review.ts \
--artifacts-dir qa-artifacts \
--output-dir video-reviews \
--video-file "$vid" \
--model gpt-4o || true
echo "::endgroup::"
done
- name: Post video review comment
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
MARKER="<!-- QA_VIDEO_REVIEW_COMMENT -->"
RUN="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
# Combine per-platform reports
REPORT=""
for f in video-reviews/*-qa-video-report.md; do
[ -f "$f" ] || continue
[ -n "$REPORT" ] && REPORT="${REPORT}
---
"
REPORT="${REPORT}$(cat "$f")"
done
[ -z "$REPORT" ] && REPORT="No video reports were generated."
BODY="${MARKER}
## QA Video Report
${REPORT}
---
**Run**: [${RUN}](${RUN})"
EXISTING=$(gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" \
--jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" | head -1)
if [ -n "$EXISTING" ]; then
gh api --method PATCH "repos/${{ github.repository }}/issues/comments/${EXISTING}" \
--field body="$BODY"
else
gh pr comment ${{ github.event.pull_request.number }} \
--repo ${{ github.repository }} --body "$BODY"
fi

View File

@@ -0,0 +1,200 @@
name: Release API Changelogs
on:
workflow_run:
workflows: ['Release NPM Types']
types:
- completed
concurrency:
group: release-api-changelogs-${{ github.workflow }}
cancel-in-progress: false
jobs:
generate_changelog:
name: Generate API Changelog
runs-on: ubuntu-latest
# Only run on successful completion of the Release NPM Types workflow
if: ${{ github.event.workflow_run.conclusion == 'success' }}
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5.0.1
with:
fetch-depth: 0 # Fetch all history for comparing versions
- name: Install pnpm
uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4.3.0
with:
version: 10
- name: Setup Node.js
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
- name: Get current version
id: current_version
run: |
VERSION=$(node -p "require('./package.json').version")
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "Current version: $VERSION"
- name: Get previous version
id: previous_version
run: |
# Get the two most recent version tags sorted
CURRENT_VERSION="${{ steps.current_version.outputs.version }}"
TAGS=$(git tag --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -2)
# Find the previous version tag (skip current if it exists)
PREVIOUS_TAG=""
for tag in $TAGS; do
TAG_VERSION=${tag#v}
if [ "$TAG_VERSION" != "$CURRENT_VERSION" ]; then
PREVIOUS_TAG=$tag
break
fi
done
if [ -z "$PREVIOUS_TAG" ]; then
echo "No previous version found, this may be the first release"
echo "version=" >> $GITHUB_OUTPUT
echo "tag=" >> $GITHUB_OUTPUT
else
echo "version=${PREVIOUS_TAG#v}" >> $GITHUB_OUTPUT
echo "tag=$PREVIOUS_TAG" >> $GITHUB_OUTPUT
echo "Previous version: ${PREVIOUS_TAG#v}"
fi
- name: Build current types
run: pnpm build:types
- name: Snapshot current API
id: current_snapshot
run: |
# Create snapshots directory
mkdir -p .api-snapshots
# Generate snapshot of current types
node --loader tsx scripts/snapshot-api.ts dist/index.d.ts > .api-snapshots/current.json
echo "Current API snapshot created"
- name: Preserve scripts for previous version
if: steps.previous_version.outputs.tag != ''
run: |
# Copy scripts to temporary location to use with previous version
mkdir -p /tmp/api-changelog-scripts
cp scripts/snapshot-api.ts scripts/compare-api-snapshots.ts /tmp/api-changelog-scripts/
- name: Checkout previous version
if: steps.previous_version.outputs.tag != ''
run: |
# Stash current changes (may be empty)
git stash || true
# Checkout previous version
git checkout ${{ steps.previous_version.outputs.tag }}
# Restore scripts
mkdir -p scripts
cp /tmp/api-changelog-scripts/*.ts scripts/
- name: Build previous types
if: steps.previous_version.outputs.tag != ''
run: |
pnpm install --frozen-lockfile
pnpm build:types
- name: Snapshot previous API
if: steps.previous_version.outputs.tag != ''
run: |
# Generate snapshot of previous types
node --loader tsx scripts/snapshot-api.ts dist/index.d.ts > .api-snapshots/previous.json
echo "Previous API snapshot created"
- name: Return to current version
if: steps.previous_version.outputs.tag != ''
run: |
# Remove copied scripts to avoid conflicts
rm -f scripts/snapshot-api.ts scripts/compare-api-snapshots.ts
git checkout -
git stash pop || true
- name: Compare API snapshots and generate changelog
id: generate_changelog
run: |
# Create docs directory if it doesn't exist
mkdir -p docs
# Get current git ref (commit SHA)
GIT_REF=$(git rev-parse HEAD)
# Run the comparison script
if [ -f .api-snapshots/previous.json ]; then
node --loader tsx scripts/compare-api-snapshots.ts \
.api-snapshots/previous.json \
.api-snapshots/current.json \
${{ steps.previous_version.outputs.version }} \
${{ steps.current_version.outputs.version }} \
Comfy-Org \
ComfyUI_frontend \
"$GIT_REF" \
>> docs/API-CHANGELOG.md
else
# First release - just document the initial API surface
echo "## v${{ steps.current_version.outputs.version }} ($(date +%Y-%m-%d))" >> docs/API-CHANGELOG.md
echo "" >> docs/API-CHANGELOG.md
echo "Initial API release." >> docs/API-CHANGELOG.md
echo "" >> docs/API-CHANGELOG.md
fi
# Check if there are any changes
if git diff --quiet docs/API-CHANGELOG.md; then
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No API changes detected"
else
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "API changes detected"
fi
- name: Create Pull Request
if: steps.generate_changelog.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e
with:
token: ${{ secrets.PR_GH_TOKEN }}
commit-message: '[docs] Update API changelog for v${{ steps.current_version.outputs.version }}'
title: '[docs] API Changelog for v${{ steps.current_version.outputs.version }}'
body: |
## API Changelog Update
This PR documents public API changes between v${{ steps.previous_version.outputs.version }} and v${{ steps.current_version.outputs.version }}.
The changelog has been automatically generated by comparing TypeScript type definitions between versions.
### Review Instructions
- Review the changes in `docs/API-CHANGELOG.md`
- Verify accuracy of breaking changes
- Add any additional context or migration notes if needed
- Merge when ready to publish changelog
---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
branch: api-changelog-v${{ steps.current_version.outputs.version }}
base: main
labels: documentation
delete-branch: true
draft: true
add-paths: |
docs/API-CHANGELOG.md

View File

@@ -1 +0,0 @@
.claude/worktrees

File diff suppressed because it is too large Load Diff

View File

@@ -8,7 +8,7 @@
"id": 11,
"type": "422723e8-4bf6-438c-823f-881ca81acead",
"pos": [791.59912109375, 386.13336181640625],
"size": [400, 200],
"size": [140, 26],
"flags": {},
"order": 0,
"mode": 0,

View File

@@ -25,11 +25,9 @@ import {
import { Topbar } from './components/Topbar'
import { CanvasHelper } from './helpers/CanvasHelper'
import { PerformanceHelper } from './helpers/PerformanceHelper'
import { QueueHelper } from './helpers/QueueHelper'
import { ClipboardHelper } from './helpers/ClipboardHelper'
import { CommandHelper } from './helpers/CommandHelper'
import { DragDropHelper } from './helpers/DragDropHelper'
import { FeatureFlagHelper } from './helpers/FeatureFlagHelper'
import { KeyboardHelper } from './helpers/KeyboardHelper'
import { NodeOperationsHelper } from './helpers/NodeOperationsHelper'
import { SettingsHelper } from './helpers/SettingsHelper'
@@ -186,11 +184,9 @@ export class ComfyPage {
public readonly contextMenu: ContextMenu
public readonly toast: ToastHelper
public readonly dragDrop: DragDropHelper
public readonly featureFlags: FeatureFlagHelper
public readonly command: CommandHelper
public readonly bottomPanel: BottomPanel
public readonly perf: PerformanceHelper
public readonly queue: QueueHelper
/** Worker index to test user ID */
public readonly userIds: string[] = []
@@ -231,11 +227,9 @@ export class ComfyPage {
this.contextMenu = new ContextMenu(page)
this.toast = new ToastHelper(page)
this.dragDrop = new DragDropHelper(page, this.assetPath.bind(this))
this.featureFlags = new FeatureFlagHelper(page)
this.command = new CommandHelper(page)
this.bottomPanel = new BottomPanel(page)
this.perf = new PerformanceHelper(page)
this.queue = new QueueHelper(page)
}
get visibleToasts() {
@@ -437,9 +431,9 @@ export const comfyPageFixture = base.extend<{
// Disable toast warning about version compatibility, as they may or
// may not appear - depending on upstream ComfyUI dependencies
'Comfy.VersionCompatibility.DisableWarnings': true,
// Disable errors tab to prevent missing model detection from
// rendering error indicators on nodes during unrelated tests.
'Comfy.RightSidePanel.ShowErrorsTab': false
// Browser tests should opt into missing-model warnings explicitly so
// workflows do not render differently based on models present on disk.
'Comfy.Workflow.ShowMissingModelsWarning': false
})
} catch (e) {
console.error(e)

View File

@@ -1,73 +0,0 @@
import type { Page, Route } from '@playwright/test'
export class FeatureFlagHelper {
private featuresRouteHandler: ((route: Route) => void) | null = null
constructor(private readonly page: Page) {}
/**
* Seed feature flags via `addInitScript` so they are available in
* localStorage before the app JS executes on first load.
* Must be called before `comfyPage.setup()` / `page.goto()`.
*
* Note: Playwright init scripts persist for the page lifetime and
* cannot be removed. Call this once per test, before navigation.
*/
async seedFlags(flags: Record<string, unknown>): Promise<void> {
await this.page.addInitScript((flagMap: Record<string, unknown>) => {
for (const [key, value] of Object.entries(flagMap)) {
localStorage.setItem(`ff:${key}`, JSON.stringify(value))
}
}, flags)
}
/**
* Set feature flags at runtime via localStorage. Uses the `ff:` prefix
* that devFeatureFlagOverride.ts reads in dev mode.
* For flags needed before page init, use `seedFlags()` instead.
*/
async setFlags(flags: Record<string, unknown>): Promise<void> {
await this.page.evaluate((flagMap: Record<string, unknown>) => {
for (const [key, value] of Object.entries(flagMap)) {
localStorage.setItem(`ff:${key}`, JSON.stringify(value))
}
}, flags)
}
async setFlag(name: string, value: unknown): Promise<void> {
await this.setFlags({ [name]: value })
}
async clearFlags(): Promise<void> {
await this.page.evaluate(() => {
const keysToRemove: string[] = []
for (let i = 0; i < localStorage.length; i++) {
const key = localStorage.key(i)
if (key?.startsWith('ff:')) keysToRemove.push(key)
}
keysToRemove.forEach((k) => {
localStorage.removeItem(k)
})
})
}
/**
* Mock server feature flags via route interception on /api/features.
*/
async mockServerFeatures(features: Record<string, unknown>): Promise<void> {
this.featuresRouteHandler = (route: Route) =>
route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify(features)
})
await this.page.route('**/api/features', this.featuresRouteHandler)
}
async clearMocks(): Promise<void> {
if (this.featuresRouteHandler) {
await this.page.unroute('**/api/features', this.featuresRouteHandler)
this.featuresRouteHandler = null
}
}
}

View File

@@ -8,10 +8,6 @@ interface PerfSnapshot {
TaskDuration: number
JSHeapUsedSize: number
Timestamp: number
Nodes: number
JSHeapTotalSize: number
ScriptDuration: number
JSEventListeners: number
}
export interface PerfMeasurement {
@@ -23,12 +19,6 @@ export interface PerfMeasurement {
layoutDurationMs: number
taskDurationMs: number
heapDeltaBytes: number
domNodes: number
jsHeapTotalBytes: number
scriptDurationMs: number
eventListeners: number
totalBlockingTimeMs: number
frameDurationMs: number
}
export class PerformanceHelper {
@@ -69,100 +59,16 @@ export class PerformanceHelper {
LayoutDuration: get('LayoutDuration'),
TaskDuration: get('TaskDuration'),
JSHeapUsedSize: get('JSHeapUsedSize'),
Timestamp: get('Timestamp'),
Nodes: get('Nodes'),
JSHeapTotalSize: get('JSHeapTotalSize'),
ScriptDuration: get('ScriptDuration'),
JSEventListeners: get('JSEventListeners')
Timestamp: get('Timestamp')
}
}
/**
* Collect longtask entries from PerformanceObserver and compute TBT.
* TBT = sum of (duration - 50ms) for every task longer than 50ms.
*/
private async collectTBT(): Promise<number> {
return this.page.evaluate(() => {
const state = (window as unknown as Record<string, unknown>)
.__perfLongtaskState as
| { observer: PerformanceObserver; tbtMs: number }
| undefined
if (!state) return 0
// Flush any queued-but-undelivered entries into our accumulator
for (const entry of state.observer.takeRecords()) {
if (entry.duration > 50) state.tbtMs += entry.duration - 50
}
const result = state.tbtMs
state.tbtMs = 0
return result
})
}
/**
* Measure average frame duration via rAF timing over a sample window.
* Returns average ms per frame (lower = better, 16.67 = 60fps).
*/
private async measureFrameDuration(sampleFrames = 10): Promise<number> {
return this.page.evaluate((frames) => {
return new Promise<number>((resolve) => {
const timeout = setTimeout(() => resolve(0), 5000)
const timestamps: number[] = []
let count = 0
function tick(ts: number) {
timestamps.push(ts)
count++
if (count <= frames) {
requestAnimationFrame(tick)
} else {
clearTimeout(timeout)
if (timestamps.length < 2) {
resolve(0)
return
}
const total = timestamps[timestamps.length - 1] - timestamps[0]
resolve(total / (timestamps.length - 1))
}
}
requestAnimationFrame(tick)
})
}, sampleFrames)
}
async startMeasuring(): Promise<void> {
if (this.snapshot) {
throw new Error(
'Measurement already in progress — call stopMeasuring() first'
)
}
// Install longtask observer if not already present, then reset the
// accumulator so old longtasks don't bleed into the new measurement window.
await this.page.evaluate(() => {
const win = window as unknown as Record<string, unknown>
if (!win.__perfLongtaskState) {
const state: { observer: PerformanceObserver; tbtMs: number } = {
observer: new PerformanceObserver((list) => {
const self = (window as unknown as Record<string, unknown>)
.__perfLongtaskState as {
observer: PerformanceObserver
tbtMs: number
}
for (const entry of list.getEntries()) {
if (entry.duration > 50) self.tbtMs += entry.duration - 50
}
}),
tbtMs: 0
}
state.observer.observe({ type: 'longtask', buffered: true })
win.__perfLongtaskState = state
}
const state = win.__perfLongtaskState as {
observer: PerformanceObserver
tbtMs: number
}
state.tbtMs = 0
state.observer.takeRecords()
})
this.snapshot = await this.getSnapshot()
}
@@ -176,11 +82,6 @@ export class PerformanceHelper {
return after[key] - before[key]
}
const [totalBlockingTimeMs, frameDurationMs] = await Promise.all([
this.collectTBT(),
this.measureFrameDuration()
])
return {
name,
durationMs: delta('Timestamp') * 1000,
@@ -189,13 +90,7 @@ export class PerformanceHelper {
layouts: delta('LayoutCount'),
layoutDurationMs: delta('LayoutDuration') * 1000,
taskDurationMs: delta('TaskDuration') * 1000,
heapDeltaBytes: delta('JSHeapUsedSize'),
domNodes: delta('Nodes'),
jsHeapTotalBytes: delta('JSHeapTotalSize'),
scriptDurationMs: delta('ScriptDuration') * 1000,
eventListeners: delta('JSEventListeners'),
totalBlockingTimeMs,
frameDurationMs
heapDeltaBytes: delta('JSHeapUsedSize')
}
}
}

View File

@@ -1,79 +0,0 @@
import type { Page, Route } from '@playwright/test'
export class QueueHelper {
private queueRouteHandler: ((route: Route) => void) | null = null
private historyRouteHandler: ((route: Route) => void) | null = null
constructor(private readonly page: Page) {}
/**
* Mock the /api/queue endpoint to return specific queue state.
*/
async mockQueueState(
running: number = 0,
pending: number = 0
): Promise<void> {
this.queueRouteHandler = (route: Route) =>
route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
queue_running: Array.from({ length: running }, (_, i) => [
i,
`running-${i}`,
{},
{},
[]
]),
queue_pending: Array.from({ length: pending }, (_, i) => [
i,
`pending-${i}`,
{},
{},
[]
])
})
})
await this.page.route('**/api/queue', this.queueRouteHandler)
}
/**
* Mock the /api/history endpoint with completed/failed job entries.
*/
async mockHistory(
jobs: Array<{ promptId: string; status: 'success' | 'error' }>
): Promise<void> {
const history: Record<string, unknown> = {}
for (const job of jobs) {
history[job.promptId] = {
prompt: [0, job.promptId, {}, {}, []],
outputs: {},
status: {
status_str: job.status === 'success' ? 'success' : 'error',
completed: true
}
}
}
this.historyRouteHandler = (route: Route) =>
route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify(history)
})
await this.page.route('**/api/history**', this.historyRouteHandler)
}
/**
* Clear all route mocks set by this helper.
*/
async clearMocks(): Promise<void> {
if (this.queueRouteHandler) {
await this.page.unroute('**/api/queue', this.queueRouteHandler)
this.queueRouteHandler = null
}
if (this.historyRouteHandler) {
await this.page.unroute('**/api/history**', this.historyRouteHandler)
this.historyRouteHandler = null
}
}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 132 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 154 KiB

After

Width:  |  Height:  |  Size: 158 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 142 KiB

After

Width:  |  Height:  |  Size: 151 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 141 KiB

After

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 140 KiB

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 126 KiB

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 107 KiB

After

Width:  |  Height:  |  Size: 115 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 97 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 100 KiB

View File

@@ -1,3 +1,4 @@
import type { Locator } from '@playwright/test'
import { expect } from '@playwright/test'
import type { Keybinding } from '../../src/platform/keybindings/types'
@@ -71,10 +72,6 @@ test('Does not report warning on undo/redo', async ({ comfyPage }) => {
test.describe('Execution error', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.settings.setSetting(
'Comfy.RightSidePanel.ShowErrorsTab',
true
)
await comfyPage.setup()
})
@@ -91,58 +88,117 @@ test.describe('Execution error', () => {
})
})
test.describe('Missing models in Error Tab', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.settings.setSetting(
'Comfy.RightSidePanel.ShowErrorsTab',
true
)
const cleanupOk = await comfyPage.page.evaluate(async (url: string) => {
const response = await fetch(`${url}/api/devtools/cleanup_fake_model`)
return response.ok
}, comfyPage.url)
expect(cleanupOk).toBeTruthy()
})
test('Should show error overlay with missing models when workflow has missing models', async ({
test.describe('Missing models warning', () => {
test('Should be disabled by default in browser tests', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow('missing/missing_models')
const errorOverlay = comfyPage.page.locator('[data-testid="error-overlay"]')
await expect(errorOverlay).toBeVisible()
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
await expect(missingModelsTitle).toBeVisible()
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).not.toBeVisible()
})
test('Should show missing models from node properties', async ({
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
'Comfy.Workflow.ShowMissingModelsWarning',
true
)
await comfyPage.page.evaluate((url: string) => {
return fetch(`${url}/api/devtools/cleanup_fake_model`)
}, comfyPage.url)
})
test('Should display a warning when missing models are found', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow('missing/missing_models')
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).toBeVisible()
const downloadAllButton = comfyPage.page.getByText('Download all')
await expect(downloadAllButton).toBeVisible()
})
test('Should display a warning when missing models are found in node properties', async ({
comfyPage
}) => {
// Load workflow that has a node with models metadata at the node level
await comfyPage.workflow.loadWorkflow(
'missing/missing_models_from_node_properties'
)
const errorOverlay = comfyPage.page.locator('[data-testid="error-overlay"]')
await expect(errorOverlay).toBeVisible()
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).toBeVisible()
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
await expect(missingModelsTitle).toBeVisible()
const downloadAllButton = comfyPage.page.getByText('Download all')
await expect(downloadAllButton).toBeVisible()
})
test('Should not show missing models when widget values have changed', async ({
test('Should not display a warning when no missing models are found', async ({
comfyPage
}) => {
const modelFoldersRes = {
status: 200,
body: JSON.stringify([
{
name: 'text_encoders',
folders: ['ComfyUI/models/text_encoders']
}
])
}
await comfyPage.page.route(
'**/api/experiment/models',
(route) => route.fulfill(modelFoldersRes),
{ times: 1 }
)
// Reload page to trigger indexing of model folders
await comfyPage.setup()
const clipModelsRes = {
status: 200,
body: JSON.stringify([
{
name: 'fake_model.safetensors',
pathIndex: 0
}
])
}
await comfyPage.page.route(
'**/api/experiment/models/text_encoders',
(route) => route.fulfill(clipModelsRes),
{ times: 1 }
)
await comfyPage.workflow.loadWorkflow('missing/missing_models')
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).not.toBeVisible()
})
test('Should not display warning when model metadata exists but widget values have changed', async ({
comfyPage
}) => {
// This tests the scenario where outdated model metadata exists in the workflow
// but the actual selected models (widget values) have changed
await comfyPage.workflow.loadWorkflow(
'missing/model_metadata_widget_mismatch'
)
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
await expect(missingModelsTitle).not.toBeVisible()
const errorOverlay = comfyPage.page.locator('[data-testid="error-overlay"]')
await expect(errorOverlay).not.toBeVisible()
// The missing models warning should NOT appear
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).not.toBeVisible()
})
// Flaky test after parallelization
@@ -150,10 +206,14 @@ test.describe('Missing models in Error Tab', () => {
test.skip('Should download missing model when clicking download button', async ({
comfyPage
}) => {
// The fake_model.safetensors is served by
// https://github.com/Comfy-Org/ComfyUI_devtools/blob/main/__init__.py
await comfyPage.workflow.loadWorkflow('missing/missing_models')
const errorOverlay = comfyPage.page.locator('[data-testid="error-overlay"]')
await expect(errorOverlay).toBeVisible()
const dialogTitle = comfyPage.page.getByText(
'This workflow is missing models'
)
await expect(dialogTitle).toBeVisible()
const downloadAllButton = comfyPage.page.getByText('Download all')
await expect(downloadAllButton).toBeVisible()
@@ -163,6 +223,50 @@ test.describe('Missing models in Error Tab', () => {
const download = await downloadPromise
expect(download.suggestedFilename()).toBe('fake_model.safetensors')
})
test.describe('Do not show again checkbox', () => {
let checkbox: Locator
let closeButton: Locator
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
'Comfy.Workflow.ShowMissingModelsWarning',
true
)
await comfyPage.workflow.loadWorkflow('missing/missing_models')
checkbox = comfyPage.page.getByLabel("Don't show this again")
closeButton = comfyPage.page.getByLabel('Close')
})
test('Should disable warning dialog when checkbox is checked', async ({
comfyPage
}) => {
const changeSettingPromise = comfyPage.page.waitForRequest(
'**/api/settings/Comfy.Workflow.ShowMissingModelsWarning'
)
await checkbox.click()
await changeSettingPromise
await closeButton.click()
const settingValue = await comfyPage.settings.getSetting(
'Comfy.Workflow.ShowMissingModelsWarning'
)
expect(settingValue).toBe(false)
})
test('Should keep warning dialog enabled when checkbox is unchecked', async ({
comfyPage
}) => {
await closeButton.click()
const settingValue = await comfyPage.settings.getSetting(
'Comfy.Workflow.ShowMissingModelsWarning'
)
expect(settingValue).toBe(true)
})
})
})
test.describe('Settings', () => {
@@ -224,11 +328,9 @@ test.describe('Settings', () => {
})
await newBlankWorkflowRow.click()
// Click add keybinding button (New Blank Workflow has no default keybinding)
const addKeybindingButton = newBlankWorkflowRow.locator(
'.icon-\\[lucide--plus\\]'
)
await addKeybindingButton.click()
// Click edit button
const editKeybindingButton = newBlankWorkflowRow.locator('.pi-pencil')
await editKeybindingButton.click()
// Set new keybinding
const input = comfyPage.page.getByPlaceholder('Enter your keybind')

View File

@@ -9,10 +9,6 @@ test.beforeEach(async ({ comfyPage }) => {
test.describe('Execution', { tag: ['@smoke', '@workflow'] }, () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.settings.setSetting(
'Comfy.RightSidePanel.ShowErrorsTab',
true
)
await comfyPage.setup()
})

View File

@@ -60,15 +60,6 @@ test.describe('Graph Canvas Menu', { tag: ['@screenshot', '@canvas'] }, () => {
await comfyPage.nextFrame()
})
test('Fit view button is present and clickable', async ({ comfyPage }) => {
const fitViewButton = comfyPage.page
.locator('button')
.filter({ has: comfyPage.page.locator('.icon-\\[lucide--focus\\]') })
await expect(fitViewButton).toBeVisible()
await fitViewButton.click()
await comfyPage.nextFrame()
})
test('Zoom controls popup opens and closes', async ({ comfyPage }) => {
// Find the zoom button by its percentage text content
const zoomButton = comfyPage.page.locator('button').filter({

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 95 KiB

View File

@@ -1,44 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
test.describe('Group Copy Paste', { tag: ['@canvas'] }, () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.canvasOps.resetView()
})
test('Pasted group is offset from original position', async ({
comfyPage
}) => {
await comfyPage.workflow.loadWorkflow('groups/single_group_only')
const titlePos = await comfyPage.page.evaluate(() => {
const app = window.app!
const group = app.graph.groups[0]
const clientPos = app.canvasPosToClientPos([
group.pos[0] + 50,
group.pos[1] + 15
])
return { x: clientPos[0], y: clientPos[1] }
})
await comfyPage.canvas.click({ position: titlePos })
await comfyPage.nextFrame()
await comfyPage.clipboard.copy()
await comfyPage.clipboard.paste()
await comfyPage.nextFrame()
const positions = await comfyPage.page.evaluate(() =>
window.app!.graph.groups.map((g: { pos: number[] }) => ({
x: g.pos[0],
y: g.pos[1]
}))
)
expect(positions).toHaveLength(2)
const dx = Math.abs(positions[0].x - positions[1].x)
const dy = Math.abs(positions[0].y - positions[1].y)
expect(dx).toBeCloseTo(50, 0)
expect(dy).toBeCloseTo(15, 0)
})
})

View File

@@ -1,123 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
import type { ComfyPage } from '../fixtures/ComfyPage'
/**
* Returns the client-space position of a group's title bar (for clicking).
*/
async function getGroupTitlePosition(
comfyPage: ComfyPage,
title: string
): Promise<{ x: number; y: number }> {
const pos = await comfyPage.page.evaluate((title) => {
const app = window.app!
const group = app.graph.groups.find(
(g: { title: string }) => g.title === title
)
if (!group) return null
const clientPos = app.canvasPosToClientPos([
group.pos[0] + 50,
group.pos[1] + 15
])
return { x: clientPos[0], y: clientPos[1] }
}, title)
if (!pos) throw new Error(`Group "${title}" not found`)
return pos
}
/**
* Returns {selectedNodeCount, selectedGroupCount, selectedItemCount}
* from the canvas in the browser.
*/
async function getSelectionCounts(comfyPage: ComfyPage) {
return comfyPage.page.evaluate(() => {
const canvas = window.app!.canvas
let selectedNodeCount = 0
let selectedGroupCount = 0
for (const item of canvas.selectedItems) {
if ('inputs' in item || 'outputs' in item) selectedNodeCount++
else selectedGroupCount++
}
return {
selectedNodeCount,
selectedGroupCount,
selectedItemCount: canvas.selectedItems.size
}
})
}
test.describe('Group Select Children', { tag: ['@canvas', '@node'] }, () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.canvasOps.resetView()
})
test('Setting enabled: clicking outer group selects nested group and inner node', async ({
comfyPage
}) => {
await comfyPage.settings.setSetting(
'LiteGraph.Group.SelectChildrenOnClick',
true
)
await comfyPage.workflow.loadWorkflow('groups/nested-groups-1-inner-node')
await comfyPage.nextFrame()
const outerPos = await getGroupTitlePosition(comfyPage, 'Outer Group')
await comfyPage.canvas.click({ position: outerPos })
await comfyPage.nextFrame()
const counts = await getSelectionCounts(comfyPage)
// Outer Group + Inner Group + 1 node = 3 items
expect(counts.selectedItemCount).toBe(3)
expect(counts.selectedGroupCount).toBe(2)
expect(counts.selectedNodeCount).toBe(1)
})
test('Setting disabled: clicking outer group selects only the group', async ({
comfyPage
}) => {
await comfyPage.settings.setSetting(
'LiteGraph.Group.SelectChildrenOnClick',
false
)
await comfyPage.workflow.loadWorkflow('groups/nested-groups-1-inner-node')
await comfyPage.nextFrame()
const outerPos = await getGroupTitlePosition(comfyPage, 'Outer Group')
await comfyPage.canvas.click({ position: outerPos })
await comfyPage.nextFrame()
const counts = await getSelectionCounts(comfyPage)
expect(counts.selectedItemCount).toBe(1)
expect(counts.selectedGroupCount).toBe(1)
expect(counts.selectedNodeCount).toBe(0)
})
test('Deselecting outer group deselects all children', async ({
comfyPage
}) => {
await comfyPage.settings.setSetting(
'LiteGraph.Group.SelectChildrenOnClick',
true
)
await comfyPage.workflow.loadWorkflow('groups/nested-groups-1-inner-node')
await comfyPage.nextFrame()
// Select the outer group (cascades to children)
const outerPos = await getGroupTitlePosition(comfyPage, 'Outer Group')
await comfyPage.canvas.click({ position: outerPos })
await comfyPage.nextFrame()
let counts = await getSelectionCounts(comfyPage)
expect(counts.selectedItemCount).toBe(3)
// Deselect all via page.evaluate to avoid UI overlay interception
await comfyPage.page.evaluate(() => {
window.app!.canvas.deselectAll()
})
await comfyPage.nextFrame()
counts = await getSelectionCounts(comfyPage)
expect(counts.selectedItemCount).toBe(0)
})
})

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 106 KiB

After

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 85 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 100 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Some files were not shown because too many files have changed in this diff Show More