mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-04-13 19:20:37 +00:00
Compare commits
43 Commits
codex/clou
...
sno-qa-106
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1dd66315ed | ||
|
|
83de5a222e | ||
|
|
2faadaeab0 | ||
|
|
cb921ada71 | ||
|
|
884270c46f | ||
|
|
51e48b55b3 | ||
|
|
07f6611cc8 | ||
|
|
55acfdd43b | ||
|
|
09ea4ae302 | ||
|
|
d698347dfd | ||
|
|
7129c2d702 | ||
|
|
2d8c71d683 | ||
|
|
231878918d | ||
|
|
5228245561 | ||
|
|
03f2e1d183 | ||
|
|
a3271cfde8 | ||
|
|
3eb8ed0748 | ||
|
|
faac186ebe | ||
|
|
d96d582ce2 | ||
|
|
76ef88f18b | ||
|
|
b42fd03213 | ||
|
|
e22daf415a | ||
|
|
86e61d7f16 | ||
|
|
218e44d112 | ||
|
|
14187df785 | ||
|
|
d821a94c59 | ||
|
|
4fd9d723e0 | ||
|
|
448c76c318 | ||
|
|
691242afcd | ||
|
|
49efcfe36a | ||
|
|
60bf788050 | ||
|
|
14e194ab34 | ||
|
|
8664d2201e | ||
|
|
c1f7e03c1c | ||
|
|
10d6e93197 | ||
|
|
cc85e699ae | ||
|
|
2394016036 | ||
|
|
72abaefe42 | ||
|
|
d5b6ceead7 | ||
|
|
94561270ef | ||
|
|
e04d050ad3 | ||
|
|
c930bbaf8e | ||
|
|
7520a47b5a |
278
.claude/skills/comfy-qa/REPRODUCE.md
Normal file
278
.claude/skills/comfy-qa/REPRODUCE.md
Normal file
@@ -0,0 +1,278 @@
|
||||
---
|
||||
name: reproduce-issue
|
||||
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
|
||||
---
|
||||
|
||||
# Issue Reproduction Skill
|
||||
|
||||
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two videos are produced:
|
||||
|
||||
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
|
||||
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
|
||||
|
||||
```
|
||||
Phase 1: Research → Read issue, understand prerequisites
|
||||
Phase 2: Environment → Install custom nodes, load workflows, configure settings
|
||||
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
|
||||
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
|
||||
Phase 5: Report → Generate a structured reproduction report
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
|
||||
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
|
||||
- `gh` CLI (authenticated, for reading issues)
|
||||
- ComfyUI backend with Python environment (for installing custom nodes)
|
||||
|
||||
## Phase 1: Research the Issue
|
||||
|
||||
1. Fetch the issue details:
|
||||
|
||||
```bash
|
||||
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
|
||||
```
|
||||
|
||||
2. Extract from the issue body:
|
||||
- **Reproduction steps** (the exact sequence)
|
||||
- **Prerequisites**: specific workflows, custom nodes, settings, models
|
||||
- **Environment**: OS, browser, ComfyUI version
|
||||
- **Media**: screenshots or videos showing the bug
|
||||
|
||||
3. Search the codebase for related code:
|
||||
- Find the feature/component mentioned in the issue
|
||||
- Understand how it works currently
|
||||
- Identify what state the UI needs to be in
|
||||
|
||||
## Phase 2: Environment Setup
|
||||
|
||||
Set up everything the issue requires BEFORE attempting reproduction.
|
||||
|
||||
### Custom Nodes
|
||||
|
||||
If the issue mentions custom nodes:
|
||||
|
||||
```bash
|
||||
# Find the custom node repo
|
||||
# Clone into ComfyUI's custom_nodes directory
|
||||
cd <comfyui_path>/custom_nodes
|
||||
git clone <custom_node_repo_url>
|
||||
|
||||
# Install dependencies if needed
|
||||
cd <custom_node_name>
|
||||
pip install -r requirements.txt 2>/dev/null || true
|
||||
|
||||
# Restart ComfyUI server to load the new nodes
|
||||
```
|
||||
|
||||
### Workflows
|
||||
|
||||
If the issue references a specific workflow:
|
||||
|
||||
```bash
|
||||
# Download workflow JSON if a URL is provided
|
||||
curl -L "<workflow_url>" -o /tmp/test-workflow.json
|
||||
|
||||
# Load it via the API
|
||||
curl -X POST http://127.0.0.1:8188/api/workflow \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @/tmp/test-workflow.json
|
||||
```
|
||||
|
||||
Or load via playwright-cli:
|
||||
|
||||
```bash
|
||||
playwright-cli goto "http://127.0.0.1:8188"
|
||||
# Drag-and-drop or use File > Open to load the workflow
|
||||
```
|
||||
|
||||
### Settings
|
||||
|
||||
If the issue requires specific settings:
|
||||
|
||||
```bash
|
||||
# Use playwright-cli to open settings and change them
|
||||
playwright-cli press "Control+,"
|
||||
playwright-cli snapshot
|
||||
# Find and modify the relevant setting
|
||||
```
|
||||
|
||||
## Phase 3: Interactive Exploration — Research Video
|
||||
|
||||
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
|
||||
|
||||
```bash
|
||||
# Open browser and start video recording
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
|
||||
# Take a snapshot to see current state
|
||||
playwright-cli snapshot
|
||||
|
||||
# Interact based on what you see
|
||||
playwright-cli click <ref>
|
||||
playwright-cli fill <ref> "text"
|
||||
playwright-cli press "Control+s"
|
||||
|
||||
# Check results
|
||||
playwright-cli snapshot
|
||||
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
|
||||
```
|
||||
|
||||
### Key Principles
|
||||
|
||||
- **Observe before acting**: Always `snapshot` before interacting
|
||||
- **Retry and adapt**: If a step fails, try a different approach
|
||||
- **Document what works**: Keep notes on which steps trigger the bug
|
||||
- **Don't give up**: Try multiple approaches if the first doesn't work
|
||||
- **Establish prerequisites**: Many bugs require specific UI state:
|
||||
- Save a workflow first (File > Save)
|
||||
- Make changes to dirty the workflow
|
||||
- Open multiple tabs
|
||||
- Add specific node types
|
||||
- Change settings
|
||||
- Resize the window
|
||||
|
||||
### Common ComfyUI Interactions via playwright-cli
|
||||
|
||||
| Action | Command |
|
||||
| ------------------- | -------------------------------------------------------------- |
|
||||
| Open hamburger menu | `playwright-cli click` on the C logo button |
|
||||
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
|
||||
| Add node | Double-click canvas → type node name → select from results |
|
||||
| Connect nodes | Drag from output slot to input slot |
|
||||
| Save workflow | `playwright-cli press "Control+s"` |
|
||||
| Save As | Menu > File > Save As |
|
||||
| Select node | Click on the node |
|
||||
| Delete node | Select → `playwright-cli press "Delete"` |
|
||||
| Right-click menu | `playwright-cli click <ref> --button right` |
|
||||
| Keyboard shortcut | `playwright-cli press "Control+z"` |
|
||||
|
||||
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
|
||||
|
||||
Once the bug is confirmed, **stop the research video** and **close the research browser**:
|
||||
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
Now start a **fresh browser session** for the clean reproduce video (Video 2).
|
||||
|
||||
**IMPORTANT constraints:**
|
||||
|
||||
- **Max 5 minutes** — the reproduce video must be short and focused
|
||||
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
|
||||
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
|
||||
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
|
||||
|
||||
1. **Open browser and start recording**:
|
||||
|
||||
```bash
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
```
|
||||
|
||||
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
|
||||
|
||||
3. **Take key screenshots** at critical moments:
|
||||
|
||||
```bash
|
||||
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
|
||||
# ... trigger the bug ...
|
||||
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
|
||||
```
|
||||
|
||||
4. **Stop recording and close** immediately after the bug is demonstrated:
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
## Phase 5: Generate Report
|
||||
|
||||
Create a reproduction report at `tmp/qa/reproduce-report.md`:
|
||||
|
||||
```markdown
|
||||
# Issue Reproduction Report
|
||||
|
||||
- **Issue**: <issue_url>
|
||||
- **Title**: <issue_title>
|
||||
- **Date**: <today>
|
||||
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
|
||||
|
||||
## Environment
|
||||
|
||||
- ComfyUI Server: <url>
|
||||
- OS: <os>
|
||||
- Custom Nodes Installed: <list or "none">
|
||||
- Settings Changed: <list or "none">
|
||||
|
||||
## Prerequisites
|
||||
|
||||
List everything that had to be set up before the bug could be triggered:
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
## Reproduction Steps
|
||||
|
||||
Minimal steps to reproduce (the clean sequence):
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
3. ...
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
<from the issue>
|
||||
|
||||
## Actual Behavior
|
||||
|
||||
<what actually happened>
|
||||
|
||||
## Evidence
|
||||
|
||||
- Research video: `research-video/video.webm` (full exploration session)
|
||||
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
|
||||
- Screenshots: `before-bug.png`, `bug-visible.png`
|
||||
|
||||
## Root Cause Analysis (if identified)
|
||||
|
||||
<code pointers, hypothesis about what's going wrong>
|
||||
|
||||
## Notes
|
||||
|
||||
<any additional observations, workarounds discovered, related issues>
|
||||
```
|
||||
|
||||
## Handling Failures
|
||||
|
||||
If the bug **cannot be reproduced**:
|
||||
|
||||
1. Document what you tried and why it didn't work
|
||||
2. Check if the issue was already fixed (search git log for related commits)
|
||||
3. Check if it's environment-specific (OS, browser, specific version)
|
||||
4. Set report status to "Not Reproduced" with detailed notes
|
||||
5. The report is still valuable — it saves others from repeating the same investigation
|
||||
|
||||
## CI Integration
|
||||
|
||||
In CI, this skill runs as a Claude Code agent with:
|
||||
|
||||
- `ANTHROPIC_API_KEY` for Claude
|
||||
- `GEMINI_API_KEY` for initial issue analysis (optional)
|
||||
- ComfyUI server pre-started in the container
|
||||
- `playwright-cli` pre-installed
|
||||
|
||||
The CI workflow:
|
||||
|
||||
1. Gemini generates a reproduce guide (markdown) from the issue
|
||||
2. Claude agent receives the guide and runs this skill
|
||||
3. Claude explores interactively, installs dependencies, retries
|
||||
4. Claude records a clean demo once reproduced
|
||||
5. Video and report are uploaded as artifacts
|
||||
133
.claude/skills/comfy-qa/SKILL.md
Normal file
133
.claude/skills/comfy-qa/SKILL.md
Normal file
@@ -0,0 +1,133 @@
|
||||
---
|
||||
name: comfy-qa
|
||||
description: 'Comprehensive QA of ComfyUI frontend. Reproduces bugs via E2E tests, records narrated demo videos, deploys reports. Works in CI and locally via `pnpm qa`.'
|
||||
---
|
||||
|
||||
# ComfyUI Frontend QA Skill
|
||||
|
||||
Automated quality assurance pipeline that reproduces reported bugs using Playwright E2E tests, records narrated demo videos with demowright, and deploys reports to Cloudflare Pages.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Reproduce an issue
|
||||
pnpm qa 10253
|
||||
|
||||
# Test a PR
|
||||
pnpm qa 10270
|
||||
|
||||
# Test PR base (reproduce bug)
|
||||
pnpm qa 10270 -t base
|
||||
|
||||
# Test both base + head
|
||||
pnpm qa 10270 -t both
|
||||
|
||||
# Test local uncommitted changes
|
||||
pnpm qa --uncommitted
|
||||
|
||||
# Help
|
||||
pnpm qa -h
|
||||
```
|
||||
|
||||
Auto-loads `.env.local` / `.env` for `GEMINI_API_KEY` and `ANTHROPIC_API_KEY`.
|
||||
Auto-detects issue vs PR via GitHub API. Auto-starts ComfyUI if not running.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three-phase pipeline:
|
||||
|
||||
1. **RESEARCH** — Claude Sonnet 4.6 (Agent SDK) writes Playwright E2E tests to reproduce bugs
|
||||
2. **RECORD** — Re-runs test with demowright for narrated demo video (title cards, TTS, subtitles)
|
||||
3. **REPORT** — Gemini reviews video, deploys to Cloudflare Pages with badge + verdict
|
||||
|
||||
Key principle: **Playwright assertions are truth** — no AI hallucination. If the test passes, the bug is proven.
|
||||
|
||||
## Pipeline Scripts
|
||||
|
||||
All scripts live in `.claude/skills/comfy-qa/scripts/`:
|
||||
|
||||
| Script | Role |
|
||||
| ------------------------- | ----------------------------------------------------- |
|
||||
| `qa.ts` | CLI entry point (`pnpm qa`) |
|
||||
| `qa-agent.ts` | Research phase: Claude writes E2E tests via Agent SDK |
|
||||
| `qa-record.ts` | Orchestrator: 3-phase pipeline |
|
||||
| `qa-deploy-pages.sh` | Cloudflare Pages deploy + badge generation |
|
||||
| `qa-report-template.html` | Report site template |
|
||||
| `qa-video-review.ts` | Gemini video review |
|
||||
| `qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide |
|
||||
|
||||
## Triggering QA
|
||||
|
||||
### Via CLI (`pnpm qa`)
|
||||
|
||||
```bash
|
||||
pnpm qa 10253 # issue (auto-detect)
|
||||
pnpm qa 10270 # PR head
|
||||
pnpm qa 10270 -t base # PR base
|
||||
pnpm qa 10270 -t both # both
|
||||
pnpm qa --uncommitted # local changes
|
||||
```
|
||||
|
||||
### Via GitHub Labels
|
||||
|
||||
- **`qa-issue`** — Reproduce a bug from an issue
|
||||
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after)
|
||||
- **`qa-full`** — Full QA (3-OS matrix)
|
||||
|
||||
### Via Push to trigger branches
|
||||
|
||||
```bash
|
||||
git push origin sno-skills:sno-qa-10253 --force
|
||||
```
|
||||
|
||||
## Research Phase (`qa-agent.ts`)
|
||||
|
||||
Claude receives the issue/PR context + a11y tree snapshot + ComfyPage fixture API docs.
|
||||
|
||||
Tools:
|
||||
|
||||
- **`inspect(selector?)`** — Read a11y tree
|
||||
- **`readFixture(path)`** — Read fixture source code
|
||||
- **`readTest(path)`** — Read existing tests for patterns
|
||||
- **`writeTest(code)`** — Write a Playwright .spec.ts
|
||||
- **`runTest()`** — Execute and get pass/fail + errors
|
||||
- **`done(verdict, summary, evidence, testCode, videoScript?)`** — Finish
|
||||
|
||||
When `verdict=REPRODUCED`, Claude also provides a `videoScript` — a separate test file using demowright's `createVideoScript()` for professional narrated demo video with title cards, TTS segments, and outro.
|
||||
|
||||
## Video Recording (demowright)
|
||||
|
||||
Phase 2 uses the video script to record with:
|
||||
|
||||
- `showTitleCard()` / `hideTitleCard()` — covers setup/loading screen
|
||||
- `createVideoScript().title().segment().outro()` — structured narration
|
||||
- `pace()` — narration-then-action timing
|
||||
- TTS audio + subtitles + cursor overlay + key badges
|
||||
|
||||
## Report Site
|
||||
|
||||
Deployed to `https://sno-qa-{number}.comfy-qa.pages.dev/`
|
||||
|
||||
Features:
|
||||
|
||||
- Video player (1x default, adjustable speed)
|
||||
- Research log (verdict, tool calls, timing)
|
||||
- E2E test code + video script code
|
||||
- Verdict banner for NOT_REPRODUCIBLE/INCONCLUSIVE with failure reason
|
||||
- Copy badge button (markdown)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `GEMINI_API_KEY` — video review, TTS
|
||||
- `ANTHROPIC_API_KEY` — Claude Agent SDK (research phase)
|
||||
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — report deployment (CI only)
|
||||
- ComfyUI server running (auto-detected, or auto-started)
|
||||
|
||||
## CI Workflow (`.github/workflows/pr-qa.yaml`)
|
||||
|
||||
```
|
||||
resolve-matrix → analyze-pr ──┐
|
||||
├→ qa-before (main branch)
|
||||
├→ qa-after (PR branch)
|
||||
└→ report (video review, deploy, comment)
|
||||
```
|
||||
673
.claude/skills/comfy-qa/scripts/qa-agent.ts
Normal file
673
.claude/skills/comfy-qa/scripts/qa-agent.ts
Normal file
@@ -0,0 +1,673 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
|
||||
*
|
||||
* Instead of driving a browser interactively, Claude:
|
||||
* 1. Reads the issue + a11y snapshot of the UI
|
||||
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
|
||||
* 3. Runs the test → reads errors → rewrites → repeats until it works
|
||||
* 4. Outputs the passing test + verdict
|
||||
*
|
||||
* Tools:
|
||||
* - inspect(selector) — read a11y tree to understand UI state
|
||||
* - writeTest(code) — write a Playwright test file
|
||||
* - runTest() — execute the test and get results
|
||||
* - done(verdict, summary, testCode) — finish with the working test
|
||||
*/
|
||||
|
||||
import type { Page } from '@playwright/test'
|
||||
/* eslint-disable import-x/no-unresolved */
|
||||
// @ts-expect-error — claude-agent-sdk has no type declarations for vue-tsc
|
||||
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
|
||||
/* eslint-enable import-x/no-unresolved */
|
||||
import { z } from 'zod'
|
||||
import { mkdirSync, readFileSync, writeFileSync } from 'fs'
|
||||
import { execSync } from 'child_process'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface ResearchOptions {
|
||||
page: Page
|
||||
issueContext: string
|
||||
qaGuide: string
|
||||
outputDir: string
|
||||
serverUrl: string
|
||||
anthropicApiKey?: string
|
||||
maxTurns?: number
|
||||
timeBudgetMs?: number
|
||||
}
|
||||
|
||||
export type ReproMethod = 'e2e_test' | 'video' | 'both' | 'none'
|
||||
|
||||
export interface ResearchResult {
|
||||
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
|
||||
reproducedBy: ReproMethod
|
||||
summary: string
|
||||
evidence: string
|
||||
testCode: string
|
||||
videoScript?: string
|
||||
log: Array<{
|
||||
turn: number
|
||||
timestampMs: number
|
||||
toolName: string
|
||||
toolInput: unknown
|
||||
toolResult: string
|
||||
}>
|
||||
}
|
||||
|
||||
// ── Main research function ──
|
||||
|
||||
export async function runResearchPhase(
|
||||
opts: ResearchOptions
|
||||
): Promise<ResearchResult> {
|
||||
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
|
||||
opts
|
||||
const maxTurns = opts.maxTurns ?? 50
|
||||
|
||||
let agentDone = false
|
||||
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
|
||||
let finalReproducedBy: ReproMethod = 'none'
|
||||
let finalSummary = 'Agent did not complete'
|
||||
let finalEvidence = ''
|
||||
let finalTestCode = ''
|
||||
let finalVideoScript = ''
|
||||
let turnCount = 0
|
||||
let lastPassedTurn = -1
|
||||
const startTime = Date.now()
|
||||
const researchLog: ResearchResult['log'] = []
|
||||
|
||||
const testDir = `${outputDir}/research`
|
||||
mkdirSync(testDir, { recursive: true })
|
||||
const testPath = `${testDir}/reproduce.spec.ts`
|
||||
|
||||
// Get initial a11y snapshot for context
|
||||
let initialA11y = ''
|
||||
try {
|
||||
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
|
||||
initialA11y = initialA11y.slice(0, 3000)
|
||||
} catch {
|
||||
initialA11y = '(could not capture initial a11y snapshot)'
|
||||
}
|
||||
|
||||
// ── Tool: inspect ──
|
||||
const inspectTool = tool(
|
||||
'inspect',
|
||||
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
|
||||
{
|
||||
selector: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Optional filter — only show elements matching this name/role. Omit for full tree.'
|
||||
)
|
||||
},
|
||||
async (args: { selector?: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const ariaText = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 5000 })
|
||||
if (args.selector) {
|
||||
const lines = ariaText.split('\n')
|
||||
const matches = lines.filter((l: string) =>
|
||||
l.toLowerCase().includes(args.selector!.toLowerCase())
|
||||
)
|
||||
resultText =
|
||||
matches.length > 0
|
||||
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
|
||||
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
|
||||
} else {
|
||||
resultText = ariaText.slice(0, 3000)
|
||||
}
|
||||
} catch (e) {
|
||||
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'inspect',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: readFixture ──
|
||||
const readFixtureTool = tool(
|
||||
'readFixture',
|
||||
'Read a fixture or helper file from browser_tests/fixtures/ to understand the API. Use this to discover available methods on comfyPage helpers before writing your test.',
|
||||
{
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
'Relative path within browser_tests/fixtures/, e.g. "helpers/CanvasHelper.ts" or "components/Topbar.ts" or "ComfyPage.ts"'
|
||||
)
|
||||
},
|
||||
async (args: { path: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const fullPath = `${projectRoot}/browser_tests/fixtures/${args.path}`
|
||||
const content = readFileSync(fullPath, 'utf-8')
|
||||
resultText = content.slice(0, 4000)
|
||||
if (content.length > 4000) {
|
||||
resultText += `\n\n... (truncated, ${content.length} total chars)`
|
||||
}
|
||||
} catch (e) {
|
||||
resultText = `Could not read fixture: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'readFixture',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: readTest ──
|
||||
const readTestTool = tool(
|
||||
'readTest',
|
||||
'Read an existing E2E test file from browser_tests/tests/ to learn patterns and conventions used in this project.',
|
||||
{
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
'Relative path within browser_tests/tests/, e.g. "workflow.spec.ts" or "subgraph.spec.ts"'
|
||||
)
|
||||
},
|
||||
async (args: { path: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const fullPath = `${projectRoot}/browser_tests/tests/${args.path}`
|
||||
const content = readFileSync(fullPath, 'utf-8')
|
||||
resultText = content.slice(0, 4000)
|
||||
if (content.length > 4000) {
|
||||
resultText += `\n\n... (truncated, ${content.length} total chars)`
|
||||
}
|
||||
} catch (e) {
|
||||
// List available test files if the path doesn't exist
|
||||
try {
|
||||
const { readdirSync } = await import('fs')
|
||||
const files = readdirSync(`${projectRoot}/browser_tests/tests/`)
|
||||
.filter((f: string) => f.endsWith('.spec.ts'))
|
||||
.slice(0, 30)
|
||||
resultText = `File not found: ${args.path}\n\nAvailable test files:\n${files.join('\n')}`
|
||||
} catch {
|
||||
resultText = `Could not read test: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'readTest',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: writeTest ──
|
||||
const writeTestTool = tool(
|
||||
'writeTest',
|
||||
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
|
||||
{
|
||||
code: z
|
||||
.string()
|
||||
.describe('Complete Playwright test file content (.spec.ts)')
|
||||
},
|
||||
async (args: { code: string }) => {
|
||||
writeFileSync(testPath, args.code)
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'writeTest',
|
||||
toolInput: { path: testPath, codeLength: args.code.length },
|
||||
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
|
||||
})
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text' as const,
|
||||
text: `Test written to ${testPath}. Use runTest() to execute it.`
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: runTest ──
|
||||
// Place test in browser_tests/ so Playwright config finds fixtures
|
||||
const projectRoot = process.cwd()
|
||||
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
|
||||
|
||||
const runTestTool = tool(
|
||||
'runTest',
|
||||
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
|
||||
{},
|
||||
async () => {
|
||||
turnCount++
|
||||
// Copy the test to browser_tests/tests/ where Playwright expects it
|
||||
const { copyFileSync } = await import('fs')
|
||||
try {
|
||||
copyFileSync(testPath, browserTestPath)
|
||||
} catch {
|
||||
// directory may not exist
|
||||
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
|
||||
copyFileSync(testPath, browserTestPath)
|
||||
}
|
||||
|
||||
let resultText: string
|
||||
try {
|
||||
const output = execSync(
|
||||
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
|
||||
{
|
||||
timeout: 90000,
|
||||
encoding: 'utf-8',
|
||||
env: {
|
||||
...process.env,
|
||||
COMFYUI_BASE_URL: serverUrl
|
||||
}
|
||||
}
|
||||
)
|
||||
resultText = `TEST PASSED:\n${output.slice(-1500)}`
|
||||
} catch (e) {
|
||||
const err = e as { stdout?: string; stderr?: string; message?: string }
|
||||
const output = (err.stdout || '') + '\n' + (err.stderr || '')
|
||||
resultText = `TEST FAILED:\n${output.slice(-2000)}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'runTest',
|
||||
toolInput: { testPath },
|
||||
toolResult: resultText.slice(0, 1000)
|
||||
})
|
||||
|
||||
// Auto-save passing test code for fallback completion — but only if
|
||||
// the test contains a bug-specific assertion (not just a discovery/debug test)
|
||||
if (resultText.startsWith('TEST PASSED')) {
|
||||
try {
|
||||
const code = readFileSync(browserTestPath, 'utf-8')
|
||||
const hasBugAssertion =
|
||||
/expect\s*\(/.test(code) &&
|
||||
!/^\s*expect\([^)]+\)\.toBeDefined\(\)/m.test(code) &&
|
||||
!/^\s*expect\([^)]+\)\.toBeGreaterThan\(0\)/m.test(code) &&
|
||||
!/Inspect|Find|Debug|discover/i.test(
|
||||
code.match(/test\(['"`]([^'"`]+)/)?.[1] ?? ''
|
||||
)
|
||||
if (hasBugAssertion) {
|
||||
finalTestCode = code
|
||||
lastPassedTurn = turnCount
|
||||
}
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
resultText +=
|
||||
'\n\n⚠️ Test PASSED — call done() now with verdict REPRODUCED and the test code. Do NOT write more tests.'
|
||||
}
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: done ──
|
||||
const doneTool = tool(
|
||||
'done',
|
||||
'Finish research with verdict and the final test code.',
|
||||
{
|
||||
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
|
||||
reproducedBy: z
|
||||
.enum(['e2e_test', 'video', 'both', 'none'])
|
||||
.describe(
|
||||
'How the bug was proven: e2e_test = Playwright assertion passed, video = visual evidence only, both = both methods, none = not reproduced'
|
||||
),
|
||||
summary: z.string().describe('What you found and why'),
|
||||
evidence: z.string().describe('Test output that proves the verdict'),
|
||||
testCode: z
|
||||
.string()
|
||||
.describe(
|
||||
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
|
||||
),
|
||||
videoScript: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Demowright video script for Phase 2 demo recording. REQUIRED when verdict is REPRODUCED. A separate test file using createVideoScript with title, segments, and outro. Do NOT include demowright imports in testCode.'
|
||||
)
|
||||
},
|
||||
async (args: {
|
||||
verdict: ResearchResult['verdict']
|
||||
reproducedBy: ReproMethod
|
||||
summary: string
|
||||
evidence: string
|
||||
testCode: string
|
||||
videoScript?: string
|
||||
}) => {
|
||||
agentDone = true
|
||||
finalVerdict = args.verdict
|
||||
finalReproducedBy = args.reproducedBy
|
||||
finalSummary = args.summary
|
||||
finalEvidence = args.evidence
|
||||
finalTestCode = args.testCode
|
||||
finalVideoScript = args.videoScript ?? ''
|
||||
writeFileSync(testPath, args.testCode)
|
||||
if (args.videoScript) {
|
||||
writeFileSync(`${outputDir}/video-script.spec.ts`, args.videoScript)
|
||||
}
|
||||
return {
|
||||
content: [
|
||||
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
// ── MCP Server ──
|
||||
const server = createSdkMcpServer({
|
||||
name: 'qa-research',
|
||||
version: '1.0.0',
|
||||
tools: [
|
||||
inspectTool,
|
||||
readFixtureTool,
|
||||
readTestTool,
|
||||
writeTestTool,
|
||||
runTestTool,
|
||||
doneTool
|
||||
]
|
||||
})
|
||||
|
||||
// ── System prompt ──
|
||||
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
|
||||
|
||||
## Your tools
|
||||
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
|
||||
- readFixture(path) — Read fixture source code from browser_tests/fixtures/. Use to discover available methods. E.g. "helpers/CanvasHelper.ts", "components/Topbar.ts", "ComfyPage.ts"
|
||||
- readTest(path) — Read an existing test from browser_tests/tests/ to learn patterns. E.g. "workflow.spec.ts". Pass any name to list available files.
|
||||
- writeTest(code) — Write a Playwright test file (.spec.ts)
|
||||
- runTest() — Execute the test and get results (pass/fail + errors)
|
||||
- done(verdict, summary, evidence, testCode) — Finish with the final test
|
||||
|
||||
## Workflow
|
||||
1. Read the issue description carefully
|
||||
2. Use inspect() to understand the current UI state and discover element selectors
|
||||
3. If unsure about the fixture API, use readFixture() to read the relevant helper source code
|
||||
4. If unsure about test patterns, use readTest() to read an existing test for reference
|
||||
5. Write a Playwright test that:
|
||||
- Performs the exact reproduction steps from the issue
|
||||
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
|
||||
6. Run the test with runTest()
|
||||
7. If it fails: read the error, fix the test, run again (max 5 attempts)
|
||||
8. Call done() with the final verdict and test code
|
||||
|
||||
## Test writing guidelines
|
||||
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
|
||||
- Import expect: \`import { expect } from '@playwright/test'\`
|
||||
- The fixture provides \`comfyPage\` which has all the helpers listed below
|
||||
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
|
||||
- Keep tests focused and minimal — test ONLY the reported bug
|
||||
- Write ONE test, not multiple. Focus on the single clearest reproduction.
|
||||
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
|
||||
- Use \`comfyPage.nextFrame()\` after interactions that trigger UI updates
|
||||
- NEVER use \`page.waitForTimeout()\` — use Locator actions and retrying assertions instead
|
||||
- ALWAYS call done() when finished, even if the test passed — do not keep iterating after a passing test
|
||||
- Use \`expect.poll()\` for async assertions: \`await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(8)\`
|
||||
- CRITICAL: Your assertions must be SPECIFIC TO THE BUG. A test that asserts \`expect(count).toBeGreaterThan(0)\` proves nothing — it would pass even without the bug. Instead assert the exact broken state, e.g. \`expect(clonedWidgets).toHaveLength(0)\` (missing widgets) or \`expect(zIndex).toBeLessThan(parentZIndex)\` (wrong z-order). If a test passes trivially, it's a false positive.
|
||||
- NEVER write "debug", "discovery", or "inspect node types" tests. These waste turns and produce false REPRODUCED verdicts. If you need to discover node type names, use inspect() or readFixture() — not a passing test.
|
||||
- If you cannot write a bug-specific assertion, call done() with verdict NOT_REPRODUCIBLE and explain why.
|
||||
|
||||
## ComfyPage Fixture API Reference
|
||||
|
||||
### Core properties
|
||||
- \`comfyPage.page\` — raw Playwright Page
|
||||
- \`comfyPage.canvas\` — Locator for #graph-canvas
|
||||
- \`comfyPage.queueButton\` — "Queue Prompt" button
|
||||
- \`comfyPage.runButton\` — "Run" button (new UI)
|
||||
- \`comfyPage.confirmDialog\` — ConfirmDialog (has .confirm, .delete, .overwrite, .reject locators + .click(name) method)
|
||||
- \`comfyPage.nextFrame()\` — wait for next requestAnimationFrame
|
||||
- \`comfyPage.setup()\` — navigate + wait for app ready (called automatically by fixture)
|
||||
|
||||
### Menu (comfyPage.menu)
|
||||
- \`comfyPage.menu.topbar\` — Topbar helper:
|
||||
- \`.triggerTopbarCommand(['File', 'Save As'])\` — navigate menu hierarchy
|
||||
- \`.openTopbarMenu()\` / \`.closeTopbarMenu()\` — open/close hamburger
|
||||
- \`.openSubmenu('File')\` — hover to open submenu, returns submenu Locator
|
||||
- \`.getTabNames()\` — get all open workflow tab names
|
||||
- \`.getActiveTabName()\` — get active tab name
|
||||
- \`.getWorkflowTab(name)\` — get tab Locator
|
||||
- \`.closeWorkflowTab(name)\` — close a tab
|
||||
- \`.saveWorkflow(name)\` / \`.saveWorkflowAs(name)\` / \`.exportWorkflow(name)\`
|
||||
- \`.switchTheme('dark' | 'light')\`
|
||||
- \`comfyPage.menu.workflowsTab\` — WorkflowsSidebarTab:
|
||||
- \`.open()\` / \`.close()\` — toggle workflows sidebar
|
||||
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflow names
|
||||
- \`comfyPage.menu.nodeLibraryTab\` — NodeLibrarySidebarTab
|
||||
- \`comfyPage.menu.assetsTab\` — AssetsSidebarTab
|
||||
|
||||
### Canvas (comfyPage.canvasOps)
|
||||
- \`.click({x, y})\` — click at position on canvas
|
||||
- \`.rightClick(x, y)\` — right-click (opens context menu)
|
||||
- \`.doubleClick()\` — double-click canvas (opens node search)
|
||||
- \`.clickEmptySpace()\` — click known empty area
|
||||
- \`.dragAndDrop(source, target)\` — drag from source to target position
|
||||
- \`.pan(offset, safeSpot?)\` — pan canvas by offset
|
||||
- \`.zoom(deltaY, steps?)\` — zoom via scroll wheel
|
||||
- \`.resetView()\` — reset zoom/pan to default
|
||||
- \`.getScale()\` / \`.setScale(n)\` — get/set canvas zoom
|
||||
- \`.getNodeCenterByTitle(title)\` — get screen coords of node center
|
||||
- \`.disconnectEdge()\` / \`.connectEdge()\` — default graph edge operations
|
||||
|
||||
### Node Operations (comfyPage.nodeOps)
|
||||
- \`.getGraphNodesCount()\` — count all nodes
|
||||
- \`.getSelectedGraphNodesCount()\` — count selected nodes
|
||||
- \`.getNodes()\` — get all nodes
|
||||
- \`.getFirstNodeRef()\` — get NodeReference for first node
|
||||
- \`.getNodeRefById(id)\` — get NodeReference by ID
|
||||
- \`.getNodeRefsByType(type)\` — get all nodes of a type
|
||||
- \`.waitForGraphNodes(count)\` — wait until node count matches
|
||||
|
||||
### Settings (comfyPage.settings)
|
||||
- \`.setSetting(id, value)\` — change a ComfyUI setting
|
||||
- \`.getSetting(id)\` — read current setting value
|
||||
|
||||
### Keyboard (comfyPage.keyboard)
|
||||
- \`.undo()\` / \`.redo()\` — Ctrl+Z / Ctrl+Y
|
||||
- \`.bypass()\` — Ctrl+B
|
||||
- \`.selectAll()\` — Ctrl+A
|
||||
- \`.ctrlSend(key)\` — send Ctrl+key
|
||||
|
||||
### Workflow (comfyPage.workflow)
|
||||
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
|
||||
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
|
||||
- \`.deleteWorkflow(name)\`
|
||||
- \`.isCurrentWorkflowModified()\` — check dirty state
|
||||
|
||||
### Context Menu (comfyPage.contextMenu)
|
||||
- \`.openFor(locator)\` — right-click locator and wait for menu
|
||||
- \`.clickMenuItem(name)\` — click a menu item by name
|
||||
- \`.isVisible()\` — check if context menu is showing
|
||||
- \`.assertHasItems(items)\` — assert menu contains items
|
||||
|
||||
### Other helpers
|
||||
- \`comfyPage.settingDialog\` — SettingDialog component
|
||||
- \`comfyPage.searchBox\` / \`comfyPage.searchBoxV2\` — node search
|
||||
- \`comfyPage.toast\` — ToastHelper (\`.visibleToasts\`)
|
||||
- \`comfyPage.subgraph\` — SubgraphHelper
|
||||
- \`comfyPage.vueNodes\` — VueNodeHelpers
|
||||
- \`comfyPage.bottomPanel\` — BottomPanel
|
||||
- \`comfyPage.clipboard\` — ClipboardHelper
|
||||
- \`comfyPage.dragDrop\` — DragDropHelper
|
||||
|
||||
### Available fixture files (use readFixture to explore)
|
||||
- ComfyPage.ts — main fixture with all helpers
|
||||
- helpers/CanvasHelper.ts, NodeOperationsHelper.ts, WorkflowHelper.ts
|
||||
- helpers/KeyboardHelper.ts, SettingsHelper.ts, SubgraphHelper.ts
|
||||
- components/Topbar.ts, ContextMenu.ts, SettingDialog.ts, SidebarTab.ts
|
||||
|
||||
## Video Script (IMPORTANT — provide via done() tool)
|
||||
|
||||
When calling done(), provide a \`videoScript\` parameter with a SEPARATE test file that uses demowright's createVideoScript.
|
||||
Do NOT put demowright imports in testCode — they won't resolve in Phase 1.
|
||||
|
||||
The videoScript is a complete, standalone Playwright test file for Phase 2 demo recording:
|
||||
|
||||
\`\`\`typescript
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { createVideoScript, showTitleCard, hideTitleCard } from 'demowright/video-script'
|
||||
|
||||
test('Demo: Bug Title', async ({ comfyPage }) => {
|
||||
// Show title card IMMEDIATELY — covers the screen while setup runs behind it
|
||||
await showTitleCard(comfyPage.page, 'Bug Title Here', { subtitle: 'Issue #NNNN' })
|
||||
|
||||
// Setup runs while title card is visible
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.workflow.setupWorkflowsDirectory({})
|
||||
|
||||
// Remove early title card before script starts (script will show its own)
|
||||
await hideTitleCard(comfyPage.page)
|
||||
|
||||
const script = createVideoScript()
|
||||
.title('Bug Title Here', { subtitle: 'Issue #NNNN', durationMs: 4000 })
|
||||
.segment('Step 1: description of what we do', async (pace) => {
|
||||
await pace() // narration finishes FIRST
|
||||
await comfyPage.menu.topbar.saveWorkflow('name') // THEN action
|
||||
await comfyPage.page.waitForTimeout(2000) // pause for viewer
|
||||
})
|
||||
.segment('Bug evidence: what we see proves the bug', async (pace) => {
|
||||
await pace()
|
||||
await comfyPage.page.waitForTimeout(5000) // hold on evidence
|
||||
})
|
||||
.outro({ text: 'Bug Reproduced', subtitle: 'Summary' })
|
||||
|
||||
await script.render(comfyPage.page)
|
||||
})
|
||||
\`\`\`
|
||||
|
||||
Key API:
|
||||
- \`.title(text, {subtitle?, durationMs?})\` — title card (4s default)
|
||||
- \`.segment(narrationText, async (pace) => { await pace(); ...actions... })\` — TTS narrated step
|
||||
- \`.outro({text?, subtitle?, durationMs?})\` — ending card
|
||||
- \`pace()\` — wait for narration audio to finish
|
||||
|
||||
CRITICAL TIMING: Call \`await pace()\` FIRST in each segment, BEFORE the Playwright actions.
|
||||
This makes the narration play and finish, THEN the actions execute — so viewers hear what's about
|
||||
to happen before it happens. Pattern:
|
||||
|
||||
\`\`\`typescript
|
||||
.segment('Now we save the workflow as a new name', async (pace) => {
|
||||
await pace() // narration finishes first
|
||||
await comfyPage.menu.topbar.saveWorkflowAs('new-name') // then action happens
|
||||
await comfyPage.page.waitForTimeout(2000) // pause so viewer sees the result
|
||||
})
|
||||
\`\`\`
|
||||
|
||||
IMPORTANT RULES for videoScript:
|
||||
1. You MUST provide videoScript when verdict is REPRODUCED — every reproduced bug needs a narrated demo
|
||||
2. Call showTitleCard() BEFORE setup, run setup behind it, call hideTitleCard() before createVideoScript() — see example
|
||||
3. Call \`await pace()\` FIRST in each segment callback, BEFORE actions
|
||||
4. Add \`waitForTimeout(2000)\` after each action so viewers can see the result
|
||||
5. Final evidence segment: hold for 5+ seconds
|
||||
6. Reproduce the same steps as testCode but slower with clear narration
|
||||
|
||||
## Current UI state (accessibility tree)
|
||||
${initialA11y}
|
||||
|
||||
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
|
||||
## Issue to Reproduce
|
||||
${issueContext}`
|
||||
|
||||
// ── Run the agent ──
|
||||
console.warn('Starting research phase (Claude writes E2E tests)...')
|
||||
|
||||
try {
|
||||
for await (const message of query({
|
||||
prompt:
|
||||
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, readFixture() or readTest() if you need to understand the fixture API or see existing test patterns, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
|
||||
options: {
|
||||
model: 'claude-sonnet-4-6',
|
||||
systemPrompt,
|
||||
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
|
||||
maxTurns,
|
||||
mcpServers: { 'qa-research': server },
|
||||
allowedTools: [
|
||||
'mcp__qa-research__inspect',
|
||||
'mcp__qa-research__readFixture',
|
||||
'mcp__qa-research__readTest',
|
||||
'mcp__qa-research__writeTest',
|
||||
'mcp__qa-research__runTest',
|
||||
'mcp__qa-research__done'
|
||||
]
|
||||
}
|
||||
})) {
|
||||
if (message.type === 'assistant' && message.message?.content) {
|
||||
for (const block of message.message.content) {
|
||||
if ('text' in block && block.text) {
|
||||
console.warn(` Claude: ${block.text.slice(0, 200)}`)
|
||||
}
|
||||
if ('name' in block) {
|
||||
console.warn(
|
||||
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (agentDone) break
|
||||
}
|
||||
} catch (e) {
|
||||
const errMsg = e instanceof Error ? e.message : String(e)
|
||||
console.warn(`Research error: ${errMsg}`)
|
||||
|
||||
// Detect billing/auth errors and surface them clearly
|
||||
if (
|
||||
errMsg.includes('Credit balance is too low') ||
|
||||
errMsg.includes('insufficient_quota') ||
|
||||
errMsg.includes('rate_limit')
|
||||
) {
|
||||
finalSummary = `API error: ${errMsg.slice(0, 200)}`
|
||||
finalEvidence = 'Agent could not start due to API billing/auth issue'
|
||||
console.warn(
|
||||
'::error::Anthropic API credits exhausted — cannot run research phase'
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-complete: if a test passed but done() was never called, use the passing test
|
||||
if (!agentDone && lastPassedTurn >= 0 && finalTestCode) {
|
||||
console.warn(
|
||||
`Auto-completing: test passed at turn ${lastPassedTurn} but done() was not called`
|
||||
)
|
||||
finalVerdict = 'REPRODUCED'
|
||||
finalReproducedBy = 'e2e_test'
|
||||
finalSummary = `Test passed at turn ${lastPassedTurn} (auto-completed — agent did not call done())`
|
||||
finalEvidence = `Test passed with exit code 0`
|
||||
}
|
||||
|
||||
const result: ResearchResult = {
|
||||
verdict: finalVerdict,
|
||||
reproducedBy: finalReproducedBy,
|
||||
summary: finalSummary,
|
||||
evidence: finalEvidence,
|
||||
testCode: finalTestCode,
|
||||
videoScript: finalVideoScript || undefined,
|
||||
log: researchLog
|
||||
}
|
||||
|
||||
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
|
||||
console.warn(
|
||||
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
|
||||
)
|
||||
|
||||
return result
|
||||
}
|
||||
84
.claude/skills/comfy-qa/scripts/qa-analyze-pr.test.ts
Normal file
84
.claude/skills/comfy-qa/scripts/qa-analyze-pr.test.ts
Normal file
@@ -0,0 +1,84 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import { extractMediaUrls } from './qa-analyze-pr'
|
||||
|
||||
describe('extractMediaUrls', () => {
|
||||
it('extracts markdown image URLs', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
|
||||
})
|
||||
|
||||
it('extracts multiple markdown images', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Some text',
|
||||
''
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/before.png',
|
||||
'https://example.com/after.jpg'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts raw URLs with media extensions', () => {
|
||||
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
|
||||
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
|
||||
})
|
||||
|
||||
it('extracts GitHub user-attachments URLs', () => {
|
||||
const text =
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts private-user-images URLs', () => {
|
||||
const text =
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts URLs with query parameters', () => {
|
||||
const text = 'https://example.com/image.png?w=800&h=600'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/image.png?w=800&h=600'
|
||||
])
|
||||
})
|
||||
|
||||
it('deduplicates URLs', () => {
|
||||
const text = [
|
||||
'',
|
||||
'',
|
||||
'Also https://example.com/same.png'
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
|
||||
})
|
||||
|
||||
it('returns empty array for empty input', () => {
|
||||
expect(extractMediaUrls('')).toEqual([])
|
||||
})
|
||||
|
||||
it('returns empty array for text with no media URLs', () => {
|
||||
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
|
||||
})
|
||||
|
||||
it('handles mixed media types', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Video: https://example.com/demo.webm',
|
||||
''
|
||||
].join('\n')
|
||||
const urls = extractMediaUrls(text)
|
||||
expect(urls).toContain('https://example.com/screenshot.png')
|
||||
expect(urls).toContain('https://example.com/demo.webm')
|
||||
expect(urls).toContain('https://example.com/animation.gif')
|
||||
})
|
||||
|
||||
it('ignores non-http URLs in markdown', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual([])
|
||||
})
|
||||
})
|
||||
799
.claude/skills/comfy-qa/scripts/qa-analyze-pr.ts
Normal file
799
.claude/skills/comfy-qa/scripts/qa-analyze-pr.ts
Normal file
@@ -0,0 +1,799 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA PR Analysis Script
|
||||
*
|
||||
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
|
||||
* for before/after recording sessions. Fetches PR thread, extracts media,
|
||||
* and produces structured test plans.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-analyze-pr.ts \
|
||||
* --pr-number 10270 \
|
||||
* --repo owner/repo \
|
||||
* --output-dir qa-guides/ \
|
||||
* [--model gemini-3.1-pro-preview]
|
||||
*
|
||||
* Env: GEMINI_API_KEY (required)
|
||||
*/
|
||||
|
||||
import { execSync } from 'node:child_process'
|
||||
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface QaGuideStep {
|
||||
action: string
|
||||
description: string
|
||||
expected_before?: string
|
||||
expected_after?: string
|
||||
}
|
||||
|
||||
interface QaGuide {
|
||||
summary: string
|
||||
test_focus: string
|
||||
prerequisites: string[]
|
||||
steps: QaGuideStep[]
|
||||
visual_checks: string[]
|
||||
}
|
||||
|
||||
interface PrThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
issueComments: string[]
|
||||
reviewComments: string[]
|
||||
reviews: string[]
|
||||
diff: string
|
||||
}
|
||||
|
||||
type TargetType = 'pr' | 'issue'
|
||||
|
||||
interface Options {
|
||||
prNumber: string
|
||||
repo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
apiKey: string
|
||||
mediaBudgetBytes: number
|
||||
maxVideoBytes: number
|
||||
type: TargetType
|
||||
}
|
||||
|
||||
// ── CLI parsing ──
|
||||
|
||||
function parseArgs(): Options {
|
||||
const args = process.argv.slice(2)
|
||||
const opts: Partial<Options> = {
|
||||
model: 'gemini-3.1-pro-preview',
|
||||
apiKey: process.env.GEMINI_API_KEY || '',
|
||||
mediaBudgetBytes: 20 * 1024 * 1024,
|
||||
maxVideoBytes: 10 * 1024 * 1024,
|
||||
type: 'pr'
|
||||
}
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--pr-number':
|
||||
opts.prNumber = args[++i]
|
||||
break
|
||||
case '--repo':
|
||||
opts.repo = args[++i]
|
||||
break
|
||||
case '--output-dir':
|
||||
opts.outputDir = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--type':
|
||||
opts.type = args[++i] as TargetType
|
||||
break
|
||||
case '--help':
|
||||
console.warn(
|
||||
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
|
||||
)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
|
||||
console.error(
|
||||
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!opts.apiKey) {
|
||||
console.error('GEMINI_API_KEY environment variable is required')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts as Options
|
||||
}
|
||||
|
||||
// ── PR thread fetching ──
|
||||
|
||||
function ghExec(cmd: string): string {
|
||||
try {
|
||||
return execSync(cmd, {
|
||||
encoding: 'utf-8',
|
||||
timeout: 30_000,
|
||||
stdio: ['pipe', 'pipe', 'pipe']
|
||||
}).trim()
|
||||
} catch (err) {
|
||||
console.warn(`gh command failed: ${cmd}`)
|
||||
console.warn((err as Error).message)
|
||||
return ''
|
||||
}
|
||||
}
|
||||
|
||||
function fetchPrThread(prNumber: string, repo: string): PrThread {
|
||||
console.warn('Fetching PR thread...')
|
||||
|
||||
const prView = ghExec(
|
||||
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const prData = prView
|
||||
? JSON.parse(prView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const issueCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
|
||||
)
|
||||
const issueComments: string[] = issueCommentsRaw
|
||||
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
|
||||
)
|
||||
const reviewComments: string[] = reviewCommentsRaw
|
||||
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
|
||||
)
|
||||
const reviews: string[] = reviewsRaw
|
||||
? JSON.parse(reviewsRaw)
|
||||
.filter((r: { body: string }) => r.body)
|
||||
.map((r: { body: string }) => r.body)
|
||||
: []
|
||||
|
||||
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
|
||||
|
||||
console.warn(
|
||||
`PR #${prNumber}: "${prData.title}" | ` +
|
||||
`${issueComments.length} issue comments, ` +
|
||||
`${reviewComments.length} review comments, ` +
|
||||
`${reviews.length} reviews, ` +
|
||||
`diff: ${diff.length} chars`
|
||||
)
|
||||
|
||||
return {
|
||||
title: prData.title || '',
|
||||
body: prData.body || '',
|
||||
labels: (prData.labels || []).map((l: { name: string }) => l.name),
|
||||
issueComments,
|
||||
reviewComments,
|
||||
reviews,
|
||||
diff
|
||||
}
|
||||
}
|
||||
|
||||
interface IssueThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
comments: string[]
|
||||
}
|
||||
|
||||
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
|
||||
console.warn('Fetching issue thread...')
|
||||
|
||||
const issueView = ghExec(
|
||||
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const issueData = issueView
|
||||
? JSON.parse(issueView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const commentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
|
||||
)
|
||||
const comments: string[] = commentsRaw
|
||||
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
console.warn(
|
||||
`Issue #${issueNumber}: "${issueData.title}" | ` +
|
||||
`${comments.length} comments`
|
||||
)
|
||||
|
||||
return {
|
||||
title: issueData.title || '',
|
||||
body: issueData.body || '',
|
||||
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
|
||||
comments
|
||||
}
|
||||
}
|
||||
|
||||
// ── Media extraction ──
|
||||
|
||||
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
|
||||
|
||||
const MEDIA_URL_PATTERNS = [
|
||||
// Markdown images: 
|
||||
/!\[[^\]]*\]\(([^)]+)\)/g,
|
||||
// GitHub user-attachments
|
||||
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
|
||||
// Private user images
|
||||
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
|
||||
// Raw URLs with media extensions (standalone or in text)
|
||||
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
|
||||
]
|
||||
|
||||
export function extractMediaUrls(text: string): string[] {
|
||||
if (!text) return []
|
||||
|
||||
const urls = new Set<string>()
|
||||
|
||||
for (const pattern of MEDIA_URL_PATTERNS) {
|
||||
// Reset lastIndex for global patterns
|
||||
pattern.lastIndex = 0
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = pattern.exec(text)) !== null) {
|
||||
// For markdown images, the URL is in capture group 1
|
||||
const url = match[1] || match[0]
|
||||
// Clean trailing markdown/html artifacts
|
||||
const cleaned = url.replace(/[)>"'\s]+$/, '')
|
||||
if (cleaned.startsWith('http')) {
|
||||
urls.add(cleaned)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [...urls]
|
||||
}
|
||||
|
||||
// ── Media downloading ──
|
||||
|
||||
const ALLOWED_MEDIA_DOMAINS = [
|
||||
'github.com',
|
||||
'raw.githubusercontent.com',
|
||||
'user-images.githubusercontent.com',
|
||||
'private-user-images.githubusercontent.com',
|
||||
'objects.githubusercontent.com',
|
||||
'github.githubassets.com'
|
||||
]
|
||||
|
||||
function isAllowedMediaDomain(url: string): boolean {
|
||||
try {
|
||||
const hostname = new URL(url).hostname
|
||||
return ALLOWED_MEDIA_DOMAINS.some(
|
||||
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
|
||||
)
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadMedia(
|
||||
urls: string[],
|
||||
outputDir: string,
|
||||
budgetBytes: number,
|
||||
maxVideoBytes: number
|
||||
): Promise<Array<{ path: string; mimeType: string }>> {
|
||||
const downloaded: Array<{ path: string; mimeType: string }> = []
|
||||
let totalBytes = 0
|
||||
|
||||
const mediaDir = resolve(outputDir, 'media')
|
||||
mkdirSync(mediaDir, { recursive: true })
|
||||
|
||||
for (const url of urls) {
|
||||
if (totalBytes >= budgetBytes) {
|
||||
console.warn(
|
||||
`Media budget exhausted (${totalBytes} bytes), skipping rest`
|
||||
)
|
||||
break
|
||||
}
|
||||
|
||||
if (!isAllowedMediaDomain(url)) {
|
||||
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
headers: { Accept: 'image/*,video/*' },
|
||||
redirect: 'follow'
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
console.warn(`Failed to download ${url}: ${response.status}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const contentLength = response.headers.get('content-length')
|
||||
if (contentLength) {
|
||||
const declaredSize = Number.parseInt(contentLength, 10)
|
||||
if (declaredSize > budgetBytes - totalBytes) {
|
||||
console.warn(
|
||||
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
|
||||
)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
const contentType = response.headers.get('content-type') || ''
|
||||
const buffer = Buffer.from(await response.arrayBuffer())
|
||||
|
||||
// Skip oversized videos
|
||||
const isVideo =
|
||||
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
|
||||
if (isVideo && buffer.length > maxVideoBytes) {
|
||||
console.warn(
|
||||
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (totalBytes + buffer.length > budgetBytes) {
|
||||
console.warn(`Would exceed budget, skipping ${url}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const ext = guessExtension(url, contentType)
|
||||
const filename = `media-${downloaded.length}${ext}`
|
||||
const filepath = resolve(mediaDir, filename)
|
||||
writeFileSync(filepath, buffer)
|
||||
totalBytes += buffer.length
|
||||
|
||||
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
|
||||
|
||||
downloaded.push({ path: filepath, mimeType })
|
||||
console.warn(
|
||||
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
|
||||
)
|
||||
} catch (err) {
|
||||
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Downloaded ${downloaded.length}/${urls.length} media files ` +
|
||||
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
|
||||
)
|
||||
return downloaded
|
||||
}
|
||||
|
||||
function guessExtension(url: string, contentType: string): string {
|
||||
const urlMatch = url.match(MEDIA_EXTENSIONS)
|
||||
if (urlMatch) return urlMatch[0].toLowerCase()
|
||||
|
||||
const typeMap: Record<string, string> = {
|
||||
'image/png': '.png',
|
||||
'image/jpeg': '.jpg',
|
||||
'image/gif': '.gif',
|
||||
'image/webp': '.webp',
|
||||
'video/mp4': '.mp4',
|
||||
'video/webm': '.webm'
|
||||
}
|
||||
return typeMap[contentType.split(';')[0]] || '.bin'
|
||||
}
|
||||
|
||||
function guessMimeType(ext: string): string {
|
||||
const map: Record<string, string> = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime'
|
||||
}
|
||||
return map[ext] || 'application/octet-stream'
|
||||
}
|
||||
|
||||
// ── Gemini analysis ──
|
||||
|
||||
function buildIssueAnalysisPrompt(issue: IssueThread): string {
|
||||
const allText = [
|
||||
`# Issue: ${issue.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
issue.body,
|
||||
'',
|
||||
issue.comments.length > 0
|
||||
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
|
||||
: ''
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
|
||||
|
||||
The UI has:
|
||||
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
|
||||
- Nodes are boxes with input/output slots connected by wires
|
||||
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
|
||||
- Sidebars (Workflows, Node Library, Models)
|
||||
- A topbar with workflow tabs and Queue button
|
||||
- The default workflow loads with these nodes (approximate center coordinates):
|
||||
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
|
||||
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
|
||||
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
|
||||
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
|
||||
|
||||
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
|
||||
### Menu actions
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
|
||||
### Element actions (by visible text)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "rightClick" — right-clicks an element to open context menu (text required)
|
||||
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
|
||||
|
||||
### Canvas actions (by coordinates — viewport is 1280x720)
|
||||
- "clickCanvas" — click at coordinates (x, y required)
|
||||
- "rightClickCanvas" — right-click at coordinates (x, y required)
|
||||
- "doubleClick" — double-click at coordinates to open node search (x, y)
|
||||
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
|
||||
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
|
||||
|
||||
### Utility
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Common ComfyUI interactions
|
||||
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
|
||||
- Double-click empty canvas → opens node search dialog
|
||||
- Ctrl+C / Ctrl+V → copy/paste selected nodes
|
||||
- Delete key → remove selected node
|
||||
- Ctrl+G → group selected nodes
|
||||
- Drag from output slot to input slot → create connection
|
||||
- Click a node to select it, Shift+click for multi-select
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly one key: "reproduce", containing:
|
||||
{
|
||||
"summary": "One sentence: what bug this issue reports",
|
||||
"test_focus": "Specific behavior to reproduce",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "clickCanvas",
|
||||
"description": "Click on first node to select it",
|
||||
"expected_before": "What should happen if the bug is present"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual evidence of the bug to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
|
||||
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
|
||||
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
|
||||
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
|
||||
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
|
||||
- Do NOT include login steps.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
function buildAnalysisPrompt(thread: PrThread): string {
|
||||
const allText = [
|
||||
`# PR: ${thread.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
thread.body,
|
||||
'',
|
||||
thread.issueComments.length > 0
|
||||
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviewComments.length > 0
|
||||
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviews.length > 0
|
||||
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
'',
|
||||
'## Diff (truncated)',
|
||||
'```',
|
||||
thread.diff.slice(0, 8000),
|
||||
'```'
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
|
||||
|
||||
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly two keys: "before" and "after", each containing:
|
||||
{
|
||||
"summary": "One sentence: what this PR changes",
|
||||
"test_focus": "Specific behaviors to verify in this recording",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "openMenu",
|
||||
"description": "Open the main menu to check file options",
|
||||
"expected_before": "Old behavior description (before key only)",
|
||||
"expected_after": "New behavior description (after key only)"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual elements to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
|
||||
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
|
||||
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
|
||||
- Use information from PR description, screenshots, and comments to understand intended behavior.
|
||||
- Include at least one screenshot step in each guide.
|
||||
- Do NOT include login steps.
|
||||
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
async function analyzeWithGemini(
|
||||
thread: PrThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<{ before: QaGuide; after: QaGuide }> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildAnalysisPrompt(thread)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
// Add media as inline data
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
// Strip markdown fences if present
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
// Handle different response shapes from Gemini
|
||||
let before: QaGuide
|
||||
let after: QaGuide
|
||||
|
||||
if (Array.isArray(parsed) && parsed.length >= 2) {
|
||||
// Array format: [before, after]
|
||||
before = parsed[0]
|
||||
after = parsed[1]
|
||||
} else if (parsed.before && parsed.after) {
|
||||
// Object format: { before, after }
|
||||
before = parsed.before
|
||||
after = parsed.after
|
||||
} else {
|
||||
// Try nested wrapper keys
|
||||
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
|
||||
if (inner.before && inner.after) {
|
||||
before = inner.before
|
||||
after = inner.after
|
||||
} else {
|
||||
console.warn(
|
||||
'Full response:',
|
||||
JSON.stringify(parsed, null, 2).slice(0, 2000)
|
||||
)
|
||||
throw new Error(
|
||||
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
return { before, after }
|
||||
}
|
||||
|
||||
async function analyzeIssueWithGemini(
|
||||
issue: IssueThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<QaGuide> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildIssueAnalysisPrompt(issue)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
const guide: QaGuide =
|
||||
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
|
||||
return guide
|
||||
}
|
||||
|
||||
// ── Main ──
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
mkdirSync(opts.outputDir, { recursive: true })
|
||||
|
||||
if (opts.type === 'issue') {
|
||||
await analyzeIssue(opts)
|
||||
} else {
|
||||
await analyzePr(opts)
|
||||
}
|
||||
}
|
||||
|
||||
async function analyzeIssue(opts: Options) {
|
||||
const issue = fetchIssueThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [issue.body, ...issue.comments].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guide = await analyzeIssueWithGemini(
|
||||
issue,
|
||||
media,
|
||||
opts.model,
|
||||
opts.apiKey
|
||||
)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guide:`)
|
||||
console.warn(` Reproduce: ${beforePath}`)
|
||||
}
|
||||
|
||||
async function analyzePr(opts: Options) {
|
||||
const thread = fetchPrThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [
|
||||
thread.body,
|
||||
...thread.issueComments,
|
||||
...thread.reviewComments,
|
||||
...thread.reviews
|
||||
].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
|
||||
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guides:`)
|
||||
console.warn(` Before: ${beforePath}`)
|
||||
console.warn(` After: ${afterPath}`)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
main().catch((err) => {
|
||||
console.error('PR analysis failed:', err)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
413
.claude/skills/comfy-qa/scripts/qa-deploy-pages.sh
Normal file
413
.claude/skills/comfy-qa/scripts/qa-deploy-pages.sh
Normal file
@@ -0,0 +1,413 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deploy QA report to Cloudflare Pages.
|
||||
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
|
||||
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
|
||||
# Writes outputs to GITHUB_OUTPUT: badge_status, url
|
||||
set -euo pipefail
|
||||
|
||||
npm install -g wrangler@4.74.0 >/dev/null 2>&1
|
||||
|
||||
DEPLOY_DIR=$(mktemp -d)
|
||||
mkdir -p "$DEPLOY_DIR"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
|
||||
for prefix in qa qa-before; do
|
||||
VID="${DIR}/${prefix}-session.mp4"
|
||||
if [ -f "$VID" ]; then
|
||||
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
|
||||
cp "$VID" "$DEST"
|
||||
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
|
||||
fi
|
||||
done
|
||||
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
|
||||
for numbered in "$DIR"/qa-session-[0-9].mp4; do
|
||||
[ -f "$numbered" ] || continue
|
||||
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
|
||||
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
|
||||
cp "$numbered" "$DEST"
|
||||
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
|
||||
done
|
||||
# Generate GIF thumbnail from after video (or first pass)
|
||||
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
|
||||
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
|
||||
if [ -f "$THUMB_SRC" ]; then
|
||||
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
|
||||
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
|
||||
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|
||||
|| echo "GIF generation failed for ${os} (non-fatal)"
|
||||
fi
|
||||
done
|
||||
|
||||
# Build video cards and report sections
|
||||
CARDS=""
|
||||
# shellcheck disable=SC2034 # accessed via eval
|
||||
ICONS_Linux="🐧" ICONS_macOS="🍎" ICONS_Windows="🪟"
|
||||
CARD_COUNT=0
|
||||
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
eval "ICON=\$ICONS_${os}"
|
||||
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
|
||||
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
|
||||
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
|
||||
[ "$HAS_AFTER" = "0" ] && continue
|
||||
|
||||
# Collect all reports for this platform (single + multi-pass)
|
||||
REPORT_FILES=""
|
||||
REPORT_LINK=""
|
||||
REPORT_HTML=""
|
||||
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
|
||||
done
|
||||
|
||||
if [ -n "$REPORT_FILES" ]; then
|
||||
# Concatenate all reports into one combined report file
|
||||
COMBINED_MD=""
|
||||
for rpt in $REPORT_FILES; do
|
||||
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
|
||||
RPT_MD=$(sed 's/&/\&/g; s/</\</g; s/>/\>/g' "$rpt")
|
||||
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD} --- "
|
||||
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
|
||||
done
|
||||
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
|
||||
FIRST_BASENAME=$(basename "$FIRST_REPORT")
|
||||
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
|
||||
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
|
||||
fi
|
||||
|
||||
if [ "$HAS_BEFORE" = "1" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
|
||||
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
|
||||
else
|
||||
PASS_VIDEOS=""
|
||||
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
|
||||
[ -f "$pass_vid" ] || continue
|
||||
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
|
||||
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
|
||||
done
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
|
||||
fi
|
||||
CARD_COUNT=$((CARD_COUNT + 1))
|
||||
done
|
||||
|
||||
# Build commit info and target link for the report header
|
||||
COMMIT_HTML=""
|
||||
REPO_URL="https://github.com/${REPO}"
|
||||
if [ -n "${TARGET_NUM:-}" ]; then
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
|
||||
else
|
||||
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
|
||||
fi
|
||||
fi
|
||||
if [ -n "${BEFORE_SHA:-}" ]; then
|
||||
SHORT_BEFORE="${BEFORE_SHA:0:7}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
|
||||
fi
|
||||
if [ -n "${AFTER_SHA:-}" ]; then
|
||||
SHORT_AFTER="${AFTER_SHA:0:7}"
|
||||
AFTER_LABEL="PR"
|
||||
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
|
||||
fi
|
||||
if [ -n "${PIPELINE_SHA:-}" ]; then
|
||||
SHORT_PIPE="${PIPELINE_SHA:0:7}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
|
||||
fi
|
||||
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" · ${COMMIT_HTML}"
|
||||
|
||||
RUN_LINK=""
|
||||
if [ -n "${RUN_URL:-}" ]; then
|
||||
RUN_LINK=" · <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
|
||||
fi
|
||||
|
||||
# Timing info
|
||||
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
|
||||
TIMING_HTML=""
|
||||
if [ -n "${RUN_START_TIME:-}" ]; then
|
||||
TIMING_HTML=" · <span class=sha title='Pipeline timing'>${RUN_START_TIME} → ${DEPLOY_TIME}</span>"
|
||||
fi
|
||||
|
||||
# Generate index.html from template
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
|
||||
|
||||
# Write dynamic content to temp files for safe substitution
|
||||
# Cloudflare Pages _headers file — enable range requests for video seeking
|
||||
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
|
||||
/*.mp4
|
||||
Accept-Ranges: bytes
|
||||
Cache-Control: public, max-age=86400
|
||||
HEADERSEOF
|
||||
|
||||
# Build purpose description from pr-context.txt
|
||||
PURPOSE_HTML=""
|
||||
if [ -f pr-context.txt ]; then
|
||||
# Extract title line and first paragraph of description
|
||||
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt 2>/dev/null | sed 's/^Title: //' || true)
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
PURPOSE_LABEL="Issue #${TARGET_NUM}"
|
||||
PURPOSE_VERB="reports"
|
||||
else
|
||||
PURPOSE_LABEL="PR #${TARGET_NUM}"
|
||||
PURPOSE_VERB="aims to"
|
||||
fi
|
||||
# Get first ~300 chars of description body (after "Description:" line)
|
||||
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt 2>/dev/null | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&/g; s/</\</g; s/>/\>/g' | tr '\n' ' ' | head -c 400 || true)
|
||||
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt 2>/dev/null | sed 's/&/\&/g; s/</\</g; s/>/\>/g' | tr '\n' ' ' | head -c 400 || true)
|
||||
# Build requirements from QA guide JSON
|
||||
REQS_HTML=""
|
||||
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1 || true)
|
||||
if [ -f "$QA_GUIDE" ]; then
|
||||
PREREQS=$(python3 -c "
|
||||
import json, sys, html
|
||||
try:
|
||||
g = json.load(open(sys.argv[1]))
|
||||
prereqs = g.get('prerequisites', [])
|
||||
steps = g.get('steps', [])
|
||||
focus = g.get('test_focus', '')
|
||||
parts = []
|
||||
if focus:
|
||||
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
|
||||
if prereqs:
|
||||
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
|
||||
if steps:
|
||||
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
|
||||
if len(steps) > 6:
|
||||
parts[-1] += ' → ...'
|
||||
print('<br>'.join(parts))
|
||||
except: pass
|
||||
" "$QA_GUIDE" 2>/dev/null)
|
||||
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
|
||||
fi
|
||||
|
||||
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
|
||||
fi
|
||||
|
||||
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
|
||||
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
|
||||
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
|
||||
# Badge HTML with copy button (placeholder URL filled after deploy)
|
||||
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
|
||||
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
|
||||
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
|
||||
python3 -c "
|
||||
import sys, pathlib
|
||||
d = pathlib.Path(sys.argv[1])
|
||||
t = pathlib.Path(sys.argv[2]).read_text()
|
||||
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
|
||||
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
|
||||
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
|
||||
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
|
||||
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
|
||||
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
|
||||
sys.stdout.write(t)
|
||||
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
|
||||
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
|
||||
|
||||
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
|
||||
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
|
||||
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
|
||||
ERROREOF
|
||||
|
||||
# Copy research log to deploy dir if it exists
|
||||
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
|
||||
if [ -f "$rlog" ]; then
|
||||
cp "$rlog" "$DEPLOY_DIR/research-log.json"
|
||||
echo "Found research log: $rlog"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy generated test code to deploy dir
|
||||
for tfile in qa-artifacts/*/research/reproduce.spec.ts qa-artifacts/*/*/research/reproduce.spec.ts qa-artifacts/before/*/research/reproduce.spec.ts; do
|
||||
if [ -f "$tfile" ]; then
|
||||
cp "$tfile" "$DEPLOY_DIR/reproduce.spec.ts"
|
||||
echo "Found test code: $tfile"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy video script if available
|
||||
for vsfile in qa-artifacts/*/video-script.spec.ts qa-artifacts/*/*/video-script.spec.ts qa-artifacts/before/*/video-script.spec.ts; do
|
||||
if [ -f "$vsfile" ]; then
|
||||
cp "$vsfile" "$DEPLOY_DIR/video-script.spec.ts"
|
||||
echo "Found video script: $vsfile"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Generate badge SVGs into deploy dir
|
||||
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
|
||||
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
|
||||
|
||||
# Try research log first (ground truth from a11y assertions)
|
||||
RESEARCH_VERDICT=""
|
||||
REPRO_METHOD=""
|
||||
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
|
||||
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
|
||||
REPRO_METHOD=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('reproducedBy','none'))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
|
||||
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none} (by: ${REPRO_METHOD:-none})"
|
||||
if [ -n "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=1
|
||||
case "$RESEARCH_VERDICT" in
|
||||
REPRODUCED) REPRO_COUNT=1 ;;
|
||||
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
|
||||
INCONCLUSIVE) INCONC_COUNT=1 ;;
|
||||
esac
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check video review verdicts (always, not just as fallback)
|
||||
VIDEO_REPRODUCED=false
|
||||
if [ -d video-reviews ]; then
|
||||
for rpt in video-reviews/*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] || continue
|
||||
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
|
||||
|
||||
if [ -n "$VERDICT_JSON" ]; then
|
||||
echo "Video review verdict: $VERDICT_JSON ($(basename "$rpt"))"
|
||||
[ "$VERDICT_JSON" = "REPRODUCED" ] && VIDEO_REPRODUCED=true
|
||||
# Only count video as separate report if no research log
|
||||
if [ -z "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
|
||||
case "$VERDICT_JSON" in
|
||||
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
|
||||
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
|
||||
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
|
||||
esac
|
||||
fi
|
||||
elif [ -z "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
|
||||
# Fallback: grep Summary section (for older reports without ## Verdict)
|
||||
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
|
||||
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
|
||||
INCONC_COUNT=$((INCONC_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
|
||||
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
|
||||
REPRO_COUNT=$((REPRO_COUNT + 1))
|
||||
VIDEO_REPRODUCED=true
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Upgrade reproduction method to "both" when E2E and video agree
|
||||
if [ "$REPRO_METHOD" = "e2e_test" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
|
||||
REPRO_METHOD="both"
|
||||
echo "Upgraded reproducedBy to 'both' (E2E + video review agree)"
|
||||
elif [ -z "$RESEARCH_VERDICT" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
|
||||
REPRO_METHOD="video"
|
||||
fi
|
||||
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
|
||||
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
|
||||
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
|
||||
|
||||
# Warn on verdict mismatch between E2E and video review
|
||||
if [ -n "$RESEARCH_VERDICT" ]; then
|
||||
VIDEO_VERDICT=$(grep -oP '"verdict":\s*"[A-Z_]+' video-reviews/*-qa-video-report.md 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
|
||||
if [ -n "$VIDEO_VERDICT" ] && [ "$RESEARCH_VERDICT" != "$VIDEO_VERDICT" ]; then
|
||||
echo "⚠ Verdict mismatch: E2E=$RESEARCH_VERDICT vs Video=$VIDEO_VERDICT (E2E takes priority)"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Verdict: ${REPRO_COUNT}✓ ${NOT_REPRO_COUNT}✗ ${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
|
||||
|
||||
# Badge text:
|
||||
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
|
||||
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
|
||||
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
|
||||
if [ "$TOTAL_REPORTS" -le 1 ]; then
|
||||
# Single report — simple label
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
|
||||
elif [ "$FAIL_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
else
|
||||
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
|
||||
PARTS=""
|
||||
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}✓"
|
||||
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}✗"
|
||||
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}⚠"
|
||||
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
|
||||
# Color based on best outcome
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Badge label: #NUM QA0327 (with today's date)
|
||||
QA_DATE=$(date -u '+%m%d')
|
||||
BADGE_LABEL="QA${QA_DATE}"
|
||||
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
|
||||
|
||||
# For PRs, also extract fix quality from Overall Risk section
|
||||
FIX_RESULT="" FIX_COLOR="#4c1"
|
||||
if [ "$TARGET_TYPE" != "issue" ]; then
|
||||
# Try structured JSON risk first
|
||||
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
|
||||
if [ -n "$ALL_RISKS" ]; then
|
||||
# Use worst risk across all reports
|
||||
if echo "$ALL_RISKS" | grep -q 'high'; then
|
||||
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
|
||||
elif echo "$ALL_RISKS" | grep -q 'medium'; then
|
||||
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
|
||||
elif echo "$ALL_RISKS" | grep -q 'low'; then
|
||||
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
|
||||
fi
|
||||
else
|
||||
# Fallback: grep Overall Risk section
|
||||
RISK_TEXT=""
|
||||
if [ -d video-reviews ]; then
|
||||
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20 || true)
|
||||
fi
|
||||
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]' || true)
|
||||
if [ -n "$RISK_FIRST" ]; then
|
||||
case "$RISK_FIRST" in
|
||||
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
|
||||
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
|
||||
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
|
||||
esac
|
||||
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
|
||||
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# Always use vertical box badge
|
||||
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
|
||||
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
|
||||
"$FIX_RESULT" "$FIX_COLOR" "$REPRO_METHOD"
|
||||
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
|
||||
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Remove files exceeding Cloudflare Pages 25MB limit to prevent silent deploy failures
|
||||
MAX_SIZE=$((25 * 1024 * 1024))
|
||||
find "$DEPLOY_DIR" -type f -size +${MAX_SIZE}c | while read -r big_file; do
|
||||
SIZE_MB=$(( $(stat -c%s "$big_file") / 1024 / 1024 ))
|
||||
echo "Removing oversized file: $(basename "$big_file") (${SIZE_MB}MB > 25MB limit)"
|
||||
rm "$big_file"
|
||||
done
|
||||
|
||||
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
|
||||
|
||||
DEPLOY_OUTPUT=$(wrangler pages deploy "$DEPLOY_DIR" \
|
||||
--project-name="comfy-qa" \
|
||||
--branch="$BRANCH" 2>&1) || true
|
||||
echo "$DEPLOY_OUTPUT" | tail -5
|
||||
|
||||
URL=$(echo "$DEPLOY_OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1 || true)
|
||||
FALLBACK_URL="https://${BRANCH}.comfy-qa.pages.dev"
|
||||
|
||||
echo "url=${URL:-$FALLBACK_URL}" >> "$GITHUB_OUTPUT"
|
||||
echo "Deployed to: ${URL:-$FALLBACK_URL}"
|
||||
208
.claude/skills/comfy-qa/scripts/qa-generate-test.ts
Normal file
208
.claude/skills/comfy-qa/scripts/qa-generate-test.ts
Normal file
@@ -0,0 +1,208 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
|
||||
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-generate-test.ts \
|
||||
* --qa-report <path> QA video review report (markdown)
|
||||
* --pr-diff <path> PR diff file
|
||||
* --output <path> Output .spec.ts file path
|
||||
* --model <name> Gemini model (default: gemini-3-flash-preview)
|
||||
*/
|
||||
import { readFile, writeFile } from 'node:fs/promises'
|
||||
import { basename, resolve } from 'node:path'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
interface CliOptions {
|
||||
qaReport: string
|
||||
prDiff: string
|
||||
output: string
|
||||
model: string
|
||||
}
|
||||
|
||||
const DEFAULTS: CliOptions = {
|
||||
qaReport: '',
|
||||
prDiff: '',
|
||||
output: '',
|
||||
model: 'gemini-3-flash-preview'
|
||||
}
|
||||
|
||||
// ── Fixture API reference for the prompt ────────────────────────────
|
||||
const FIXTURE_API = `
|
||||
## ComfyUI Playwright Test Fixture API
|
||||
|
||||
Import pattern:
|
||||
\`\`\`typescript
|
||||
import { expect } from '@playwright/test'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
\`\`\`
|
||||
|
||||
### Available helpers on \`comfyPage\`:
|
||||
- \`comfyPage.page\` — raw Playwright Page
|
||||
- \`comfyPage.menu.topbar\` — Topbar helper:
|
||||
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
|
||||
- \`.getActiveTabName(): Promise<string>\` — get active tab name
|
||||
- \`.saveWorkflow(name)\` — Save via File > Save dialog
|
||||
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
|
||||
- \`.exportWorkflow(name)\` — Export via File > Export dialog
|
||||
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
|
||||
- \`.getWorkflowTab(name)\` — get a tab locator by name
|
||||
- \`.closeWorkflowTab(name)\` — close a tab
|
||||
- \`.openTopbarMenu()\` — open the hamburger menu
|
||||
- \`.openSubmenu(label)\` — hover to open a submenu
|
||||
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
|
||||
- \`.open()\` / \`.close()\` — toggle sidebar
|
||||
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
|
||||
- \`.getPersistedItem(name)\` — get a workflow item locator
|
||||
- \`comfyPage.workflow\` — WorkflowHelper:
|
||||
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
|
||||
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
|
||||
- \`.deleteWorkflow(name)\` — delete a workflow
|
||||
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
|
||||
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
|
||||
- \`comfyPage.settings.setSetting(key, value)\` — change settings
|
||||
- \`comfyPage.keyboard\` — KeyboardHelper:
|
||||
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
|
||||
- \`comfyPage.nodeOps\` — NodeOperationsHelper
|
||||
- \`comfyPage.canvas\` — CanvasHelper
|
||||
- \`comfyPage.contextMenu\` — ContextMenu
|
||||
- \`comfyPage.toast\` — ToastHelper
|
||||
- \`comfyPage.confirmDialog\` — confirmation dialog
|
||||
- \`comfyPage.nextFrame()\` — wait for Vue re-render
|
||||
|
||||
### Test patterns:
|
||||
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
|
||||
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
|
||||
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
|
||||
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
|
||||
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
|
||||
`
|
||||
|
||||
// ── Prompt builder ──────────────────────────────────────────────────
|
||||
function buildPrompt(qaReport: string, prDiff: string): string {
|
||||
return `You are a Playwright test generator for the ComfyUI frontend.
|
||||
|
||||
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
|
||||
described in the QA report below. The test must:
|
||||
|
||||
1. Use the ComfyUI Playwright fixture API (documented below)
|
||||
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
|
||||
3. NOT test code implementation details
|
||||
4. Be concise — only test the behavior that the PR changed
|
||||
5. Follow existing test conventions (see API reference)
|
||||
|
||||
${FIXTURE_API}
|
||||
|
||||
## QA Video Review Report
|
||||
${qaReport}
|
||||
|
||||
## PR Diff (for context on what changed)
|
||||
${prDiff.slice(0, 8000)}
|
||||
|
||||
## Output Requirements
|
||||
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
|
||||
- Start with imports, end with closing brace
|
||||
- Use descriptive test names that explain the expected behavior
|
||||
- Add screenshot assertions where visual verification matters
|
||||
- Keep it focused: 2-5 test cases covering the core behavioral change
|
||||
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
|
||||
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
|
||||
`
|
||||
}
|
||||
|
||||
// ── Gemini call ─────────────────────────────────────────────────────
|
||||
async function generateTest(
|
||||
qaReport: string,
|
||||
prDiff: string,
|
||||
model: string
|
||||
): Promise<string> {
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
|
||||
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const genModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildPrompt(qaReport, prDiff)
|
||||
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
|
||||
|
||||
const result = await genModel.generateContent({
|
||||
contents: [{ role: 'user', parts: [{ text: prompt }] }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192
|
||||
}
|
||||
})
|
||||
|
||||
const text = result.response.text()
|
||||
|
||||
// Strip markdown fences if model wraps output
|
||||
return text
|
||||
.replace(/^```(?:typescript|ts)?\n?/, '')
|
||||
.replace(/\n?```$/, '')
|
||||
.trim()
|
||||
}
|
||||
|
||||
// ── CLI ─────────────────────────────────────────────────────────────
|
||||
function parseArgs(): CliOptions {
|
||||
const args = process.argv.slice(2)
|
||||
const opts = { ...DEFAULTS }
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--qa-report':
|
||||
opts.qaReport = args[++i]
|
||||
break
|
||||
case '--pr-diff':
|
||||
opts.prDiff = args[++i]
|
||||
break
|
||||
case '--output':
|
||||
opts.output = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--help':
|
||||
console.warn(`Usage:
|
||||
pnpm exec tsx scripts/qa-generate-test.ts [options]
|
||||
|
||||
Options:
|
||||
--qa-report <path> QA video review report (markdown) [required]
|
||||
--pr-diff <path> PR diff file [required]
|
||||
--output <path> Output .spec.ts path [required]
|
||||
--model <name> Gemini model (default: gemini-3-flash-preview)`)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.qaReport || !opts.prDiff || !opts.output) {
|
||||
console.error('Missing required args. Run with --help for usage.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
|
||||
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
|
||||
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
|
||||
|
||||
console.warn(
|
||||
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
|
||||
)
|
||||
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
|
||||
|
||||
const testCode = await generateTest(qaReport, prDiff, opts.model)
|
||||
|
||||
const outputPath = resolve(opts.output)
|
||||
await writeFile(outputPath, testCode + '\n')
|
||||
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err)
|
||||
process.exit(1)
|
||||
})
|
||||
2203
.claude/skills/comfy-qa/scripts/qa-record.ts
Normal file
2203
.claude/skills/comfy-qa/scripts/qa-record.ts
Normal file
File diff suppressed because it is too large
Load Diff
164
.claude/skills/comfy-qa/scripts/qa-report-template.html
Normal file
164
.claude/skills/comfy-qa/scripts/qa-report-template.html
Normal file
@@ -0,0 +1,164 @@
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
|
||||
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<style>
|
||||
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
|
||||
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
|
||||
*{margin:0;padding:0;box-sizing:border-box}
|
||||
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
|
||||
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
|
||||
.container{max-width:1200px;margin:0 auto}
|
||||
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
|
||||
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
|
||||
.header-icon svg{color:var(--primary)}
|
||||
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
|
||||
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
|
||||
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
|
||||
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
|
||||
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
|
||||
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
|
||||
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
|
||||
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
|
||||
.icon{font-size:1.125rem}
|
||||
.links{display:flex;gap:.5rem}
|
||||
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
|
||||
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
|
||||
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
|
||||
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
|
||||
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
|
||||
.comp-panel{border-right:1px solid var(--border-faint)}
|
||||
.comp-panel:last-child{border-right:none}
|
||||
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
|
||||
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
|
||||
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
|
||||
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
|
||||
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
|
||||
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
|
||||
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
|
||||
.report summary:hover{color:var(--fg)}
|
||||
.report summary svg{flex-shrink:0;opacity:.5}
|
||||
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
|
||||
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
|
||||
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
|
||||
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
|
||||
.report-body p{margin:.4rem 0}
|
||||
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
|
||||
.report-body li{margin:.25rem 0}
|
||||
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
|
||||
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
|
||||
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
|
||||
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
|
||||
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
|
||||
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
|
||||
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
|
||||
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
|
||||
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
|
||||
.report-body strong{color:var(--fg)}
|
||||
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
|
||||
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
|
||||
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
|
||||
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
|
||||
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
|
||||
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
|
||||
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
|
||||
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
|
||||
.badge-img{height:20px;display:block}
|
||||
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
|
||||
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
|
||||
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
|
||||
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
|
||||
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
|
||||
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
|
||||
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
|
||||
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
|
||||
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
|
||||
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
|
||||
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
|
||||
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
|
||||
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
|
||||
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
|
||||
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
|
||||
.purpose strong{color:var(--fg);font-weight:600}
|
||||
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
|
||||
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
|
||||
</style></head><body><div class=container>
|
||||
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend · Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
|
||||
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
|
||||
<div id=research-section style="margin-top:2rem"></div>
|
||||
</div><script>
|
||||
// Load research-log.json and reproduce.spec.ts if available
|
||||
(async()=>{
|
||||
const sec=document.getElementById('research-section');
|
||||
try{
|
||||
const [logRes,testRes,vsRes]=await Promise.allSettled([fetch('research-log.json'),fetch('reproduce.spec.ts'),fetch('video-script.spec.ts')]);
|
||||
let html='';
|
||||
if(logRes.status==='fulfilled'&&logRes.value.ok){
|
||||
const log=await logRes.value.json();
|
||||
// Show verdict banner for non-reproduced results
|
||||
if(log.verdict&&log.verdict!=='REPRODUCED'){
|
||||
const colors={NOT_REPRODUCIBLE:{bg:'oklch(25% 0.08 25)',border:'oklch(40% 0.15 25)',icon:'✗'},INCONCLUSIVE:{bg:'oklch(25% 0.06 80)',border:'oklch(40% 0.12 80)',icon:'⚠'}};
|
||||
const c=colors[log.verdict]||colors.INCONCLUSIVE;
|
||||
html+=`<div style="margin-bottom:1.5rem;padding:1.25rem;background:${c.bg};border:1px solid ${c.border};border-radius:var(--r-lg)"><div style="font-size:1.25rem;font-weight:700;margin-bottom:.5rem">${c.icon} ${log.verdict.replace(/_/g,' ')}</div><div style="font-size:.9rem;line-height:1.6;opacity:.9">${(log.summary||'No details available.').replace(/</g,'<')}</div>${log.evidence?`<div style="margin-top:.75rem;padding:.75rem;background:oklch(0% 0 0/.2);border-radius:var(--r);font-family:var(--font-mono);font-size:.8rem;white-space:pre-wrap;max-height:200px;overflow:auto">${log.evidence.replace(/</g,'<')}</div>`:''}</div>`;
|
||||
}
|
||||
html+=`<details style="margin-bottom:1.5rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Research Log — ${log.verdict||'?'} (${(log.log||[]).length||'?'} tool calls, ${((log.elapsedMs||0)/1000).toFixed(1)}s)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${JSON.stringify(log,null,2)}</pre></div></details>`;
|
||||
}
|
||||
if(testRes.status==='fulfilled'&&testRes.value.ok){
|
||||
const code=await testRes.value.text();
|
||||
html+=`<details><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">E2E Test Code (reproduce.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${code.replace(/</g,'<').replace(/>/g,'>')}</pre></div></details>`;
|
||||
}
|
||||
if(vsRes.status==='fulfilled'&&vsRes.value.ok){
|
||||
const vsCode=await vsRes.value.text();
|
||||
html+=`<details style="margin-top:1rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Video Script (video-script.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${vsCode.replace(/</g,'<').replace(/>/g,'>')}</pre></div></details>`;
|
||||
}
|
||||
if(html)sec.innerHTML=html;
|
||||
}catch(e){console.warn('research load failed',e)}
|
||||
})();
|
||||
</script><script>
|
||||
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
|
||||
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
|
||||
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
|
||||
document.querySelectorAll('.video-wrap video').forEach(v=>{
|
||||
v.playbackRate=1;
|
||||
const c=document.createElement('div');c.className='vctrl';
|
||||
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
|
||||
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
|
||||
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
|
||||
btn('\u23EE',()=>{v.currentTime=0});
|
||||
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
|
||||
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
|
||||
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
|
||||
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
|
||||
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
|
||||
sep();
|
||||
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===1)b.classList.add('active');return b});
|
||||
sep();c.appendChild(time);
|
||||
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
|
||||
// Custom seekbar — works even without server range request support
|
||||
const seekWrap=document.createElement('div');seekWrap.className='vseek';
|
||||
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
|
||||
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
|
||||
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
|
||||
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
|
||||
let seeking=false;
|
||||
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
|
||||
seekBar.onchange=()=>{seeking=false};
|
||||
v.closest('.video-wrap').after(seekWrap);
|
||||
seekWrap.after(c);
|
||||
v.ontimeupdate=()=>{
|
||||
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
|
||||
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
|
||||
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
|
||||
};
|
||||
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
|
||||
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
|
||||
v.parentElement.addEventListener('keydown',e=>{
|
||||
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
|
||||
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
|
||||
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
|
||||
});
|
||||
v.parentElement.setAttribute('tabindex','0');
|
||||
});
|
||||
</script></body></html>
|
||||
253
.claude/skills/comfy-qa/scripts/qa-reproduce.ts
Normal file
253
.claude/skills/comfy-qa/scripts/qa-reproduce.ts
Normal file
@@ -0,0 +1,253 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA Reproduce Phase — Deterministic replay of research plan with narration
|
||||
*
|
||||
* Takes a reproduction plan from the research phase and replays it:
|
||||
* 1. Execute each action deterministically (no AI decisions)
|
||||
* 2. Capture a11y snapshot before/after each action
|
||||
* 3. Gemini describes what visually changed (narration for humans)
|
||||
* 4. Output: narration-log.json with full evidence chain
|
||||
*/
|
||||
|
||||
import type { Page } from '@playwright/test'
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
import { mkdirSync, writeFileSync } from 'fs'
|
||||
|
||||
import type { ActionResult } from './qa-record.js'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface ReproductionStep {
|
||||
action: Record<string, unknown> & { action: string }
|
||||
expectedAssertion: string
|
||||
}
|
||||
|
||||
interface NarrationEntry {
|
||||
step: number
|
||||
action: string
|
||||
params: Record<string, unknown>
|
||||
result: ActionResult
|
||||
a11yBefore: unknown
|
||||
a11yAfter: unknown
|
||||
assertionExpected: string
|
||||
assertionPassed: boolean
|
||||
assertionActual: string
|
||||
geminiNarration: string
|
||||
timestampMs: number
|
||||
}
|
||||
|
||||
export interface NarrationLog {
|
||||
entries: NarrationEntry[]
|
||||
allAssertionsPassed: boolean
|
||||
}
|
||||
|
||||
interface ReproduceOptions {
|
||||
page: Page
|
||||
plan: ReproductionStep[]
|
||||
geminiApiKey: string
|
||||
outputDir: string
|
||||
}
|
||||
|
||||
// ── A11y helpers ──
|
||||
|
||||
interface A11yNode {
|
||||
role: string
|
||||
name: string
|
||||
value?: string
|
||||
checked?: boolean
|
||||
disabled?: boolean
|
||||
expanded?: boolean
|
||||
children?: A11yNode[]
|
||||
}
|
||||
|
||||
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
|
||||
if (!node) return null
|
||||
const sel = selector.toLowerCase()
|
||||
if (
|
||||
node.name?.toLowerCase().includes(sel) ||
|
||||
node.role?.toLowerCase().includes(sel)
|
||||
) {
|
||||
return node
|
||||
}
|
||||
if (node.children) {
|
||||
for (const child of node.children) {
|
||||
const found = searchA11y(child, selector)
|
||||
if (found) return found
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function summarizeA11y(node: A11yNode | null): string {
|
||||
if (!node) return 'null'
|
||||
const parts = [`role=${node.role}`, `name="${node.name}"`]
|
||||
if (node.value !== undefined) parts.push(`value="${node.value}"`)
|
||||
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
|
||||
if (node.disabled) parts.push('disabled')
|
||||
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
|
||||
return `{${parts.join(', ')}}`
|
||||
}
|
||||
|
||||
// ── Subtitle overlay ──
|
||||
|
||||
async function showSubtitle(page: Page, text: string, step: number) {
|
||||
const encoded = encodeURIComponent(
|
||||
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
|
||||
)
|
||||
await page.addScriptTag({
|
||||
content: `(function(){
|
||||
var id='qa-subtitle';
|
||||
var el=document.getElementById(id);
|
||||
if(!el){
|
||||
el=document.createElement('div');
|
||||
el.id=id;
|
||||
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
|
||||
document.body.appendChild(el);
|
||||
}
|
||||
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
|
||||
})()`
|
||||
})
|
||||
}
|
||||
|
||||
// ── Gemini visual narration ──
|
||||
|
||||
async function geminiDescribe(
|
||||
page: Page,
|
||||
geminiApiKey: string,
|
||||
focus: string
|
||||
): Promise<string> {
|
||||
try {
|
||||
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
|
||||
const genAI = new GoogleGenerativeAI(geminiApiKey)
|
||||
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
|
||||
|
||||
const result = await model.generateContent([
|
||||
{
|
||||
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
|
||||
},
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: 'image/jpeg',
|
||||
data: screenshot.toString('base64')
|
||||
}
|
||||
}
|
||||
])
|
||||
return result.response.text().trim()
|
||||
} catch (e) {
|
||||
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
|
||||
}
|
||||
}
|
||||
|
||||
// ── Main reproduce function ──
|
||||
|
||||
export async function runReproducePhase(
|
||||
opts: ReproduceOptions
|
||||
): Promise<NarrationLog> {
|
||||
const { page, plan, geminiApiKey, outputDir } = opts
|
||||
const { executeAction } = await import('./qa-record.js')
|
||||
|
||||
const narrationDir = `${outputDir}/narration`
|
||||
mkdirSync(narrationDir, { recursive: true })
|
||||
|
||||
const entries: NarrationEntry[] = []
|
||||
const startMs = Date.now()
|
||||
|
||||
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
|
||||
|
||||
for (let i = 0; i < plan.length; i++) {
|
||||
const step = plan[i]
|
||||
const actionObj = step.action
|
||||
const elapsed = Date.now() - startMs
|
||||
|
||||
// Show subtitle
|
||||
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
|
||||
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
|
||||
|
||||
// Capture a11y BEFORE
|
||||
const a11yBefore = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 3000 })
|
||||
.catch(() => null)
|
||||
|
||||
// Execute action
|
||||
const result = await executeAction(
|
||||
page,
|
||||
actionObj as Parameters<typeof executeAction>[1],
|
||||
outputDir
|
||||
)
|
||||
await new Promise((r) => setTimeout(r, 500))
|
||||
|
||||
// Capture a11y AFTER
|
||||
const a11yAfter = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 3000 })
|
||||
.catch(() => null)
|
||||
|
||||
// Check assertion
|
||||
let assertionPassed = false
|
||||
let assertionActual = ''
|
||||
if (step.expectedAssertion) {
|
||||
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
|
||||
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
|
||||
const selectorName = parts[0]
|
||||
const expectedState = parts.slice(1).join(':').trim()
|
||||
|
||||
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
|
||||
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
|
||||
|
||||
if (expectedState === 'visible' || expectedState === 'exists') {
|
||||
assertionPassed = found !== null
|
||||
} else if (expectedState === 'hidden' || expectedState === 'gone') {
|
||||
assertionPassed = found === null
|
||||
} else {
|
||||
// Generic: check if the actual state contains the expected text
|
||||
assertionPassed = assertionActual
|
||||
.toLowerCase()
|
||||
.includes(expectedState.toLowerCase())
|
||||
}
|
||||
|
||||
console.warn(
|
||||
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
|
||||
)
|
||||
}
|
||||
|
||||
// Gemini narration (visual description for humans)
|
||||
const geminiNarration = await geminiDescribe(
|
||||
page,
|
||||
geminiApiKey,
|
||||
`What changed after ${actionObj.action}?`
|
||||
)
|
||||
|
||||
entries.push({
|
||||
step: i + 1,
|
||||
action: actionObj.action,
|
||||
params: actionObj,
|
||||
result,
|
||||
a11yBefore,
|
||||
a11yAfter,
|
||||
assertionExpected: step.expectedAssertion,
|
||||
assertionPassed,
|
||||
assertionActual,
|
||||
geminiNarration,
|
||||
timestampMs: elapsed
|
||||
})
|
||||
}
|
||||
|
||||
// Final screenshot
|
||||
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
|
||||
|
||||
const log: NarrationLog = {
|
||||
entries,
|
||||
allAssertionsPassed: entries.every((e) => e.assertionPassed)
|
||||
}
|
||||
|
||||
writeFileSync(
|
||||
`${narrationDir}/narration-log.json`,
|
||||
JSON.stringify(log, null, 2)
|
||||
)
|
||||
console.warn(
|
||||
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
|
||||
)
|
||||
|
||||
return log
|
||||
}
|
||||
150
.claude/skills/comfy-qa/scripts/qa-video-review.test.ts
Normal file
150
.claude/skills/comfy-qa/scripts/qa-video-review.test.ts
Normal file
@@ -0,0 +1,150 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import {
|
||||
extractPlatformFromArtifactDirName,
|
||||
pickLatestVideosByPlatform,
|
||||
selectVideoCandidateByFile
|
||||
} from './qa-video-review'
|
||||
|
||||
describe('extractPlatformFromArtifactDirName', () => {
|
||||
it('extracts and normalizes known qa artifact directory names', () => {
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
|
||||
).toBe('windows')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
|
||||
).toBe('macos')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
|
||||
).toBe('linux')
|
||||
})
|
||||
|
||||
it('falls back to slugifying unknown directory names', () => {
|
||||
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
|
||||
'custom-platform-run'
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('pickLatestVideosByPlatform', () => {
|
||||
it('keeps only the latest candidate per platform', () => {
|
||||
const selected = pickLatestVideosByPlatform([
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-old.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
}
|
||||
])
|
||||
|
||||
expect(selected).toEqual([
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('selectVideoCandidateByFile', () => {
|
||||
it('selects a single candidate by artifacts-relative path', () => {
|
||||
const selected = selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-Linux-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
|
||||
expect(selected).toEqual({
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
})
|
||||
})
|
||||
|
||||
it('throws when basename matches multiple videos', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('matched 2 videos')
|
||||
})
|
||||
|
||||
it('throws when there is no matching video', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-macOS-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('No video matched')
|
||||
})
|
||||
|
||||
it('throws when video file is missing', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: ' '
|
||||
}
|
||||
)
|
||||
).toThrow('--video-file is required')
|
||||
})
|
||||
})
|
||||
771
.claude/skills/comfy-qa/scripts/qa-video-review.ts
Normal file
771
.claude/skills/comfy-qa/scripts/qa-video-review.ts
Normal file
@@ -0,0 +1,771 @@
|
||||
#!/usr/bin/env tsx
|
||||
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
|
||||
import { basename, dirname, extname, relative, resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
import { globSync } from 'glob'
|
||||
|
||||
interface CliOptions {
|
||||
artifactsDir: string
|
||||
videoFile: string
|
||||
beforeVideo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
requestTimeoutMs: number
|
||||
dryRun: boolean
|
||||
prContext: string
|
||||
targetUrl: string
|
||||
passLabel: string
|
||||
}
|
||||
|
||||
interface VideoCandidate {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
mtimeMs: number
|
||||
}
|
||||
|
||||
const DEFAULT_OPTIONS: CliOptions = {
|
||||
artifactsDir: './tmp/qa-artifacts',
|
||||
videoFile: '',
|
||||
beforeVideo: '',
|
||||
outputDir: './tmp',
|
||||
model: 'gemini-3-flash-preview',
|
||||
requestTimeoutMs: 300_000,
|
||||
dryRun: false,
|
||||
prContext: '',
|
||||
targetUrl: '',
|
||||
passLabel: ''
|
||||
}
|
||||
|
||||
const USAGE = `Usage:
|
||||
pnpm exec tsx scripts/qa-video-review.ts [options]
|
||||
|
||||
Options:
|
||||
--artifacts-dir <path> Artifacts root directory
|
||||
(default: ./tmp/qa-artifacts)
|
||||
--video-file <name-or-path> Video file to analyze (required)
|
||||
(supports basename or relative/absolute path)
|
||||
--before-video <path> Before video (main branch) for comparison
|
||||
When provided, sends both videos to Gemini
|
||||
for comparative before/after analysis
|
||||
--output-dir <path> Output directory for markdown reports
|
||||
(default: ./tmp)
|
||||
--model <name> Gemini model
|
||||
(default: gemini-3-flash-preview)
|
||||
--request-timeout-ms <n> Request timeout in milliseconds
|
||||
(default: 300000)
|
||||
--pr-context <file> File with PR context (title, body, diff)
|
||||
for PR-aware review
|
||||
--target-url <url> Issue or PR URL to include in the report
|
||||
--pass-label <label> Label for multi-pass reports (e.g. pass1)
|
||||
Output becomes {platform}-{label}-qa-video-report.md
|
||||
--dry-run Discover videos and output targets only
|
||||
--help Show this help text
|
||||
|
||||
Environment:
|
||||
GEMINI_API_KEY Required unless --dry-run
|
||||
`
|
||||
|
||||
function parsePositiveInteger(rawValue: string, flagName: string): number {
|
||||
const parsedValue = Number.parseInt(rawValue, 10)
|
||||
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
|
||||
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
|
||||
}
|
||||
return parsedValue
|
||||
}
|
||||
|
||||
function parseCliOptions(args: string[]): CliOptions {
|
||||
const options: CliOptions = { ...DEFAULT_OPTIONS }
|
||||
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const argument = args[index]
|
||||
const nextValue = args[index + 1]
|
||||
const requireValue = (flagName: string): string => {
|
||||
if (!nextValue || nextValue.startsWith('--')) {
|
||||
throw new Error(`Missing value for ${flagName}`)
|
||||
}
|
||||
index += 1
|
||||
return nextValue
|
||||
}
|
||||
|
||||
if (argument === '--help') {
|
||||
process.stdout.write(USAGE)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
if (argument === '--artifacts-dir') {
|
||||
options.artifactsDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--video-file') {
|
||||
options.videoFile = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--output-dir') {
|
||||
options.outputDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--model') {
|
||||
options.model = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--request-timeout-ms') {
|
||||
options.requestTimeoutMs = parsePositiveInteger(
|
||||
requireValue(argument),
|
||||
argument
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--before-video') {
|
||||
options.beforeVideo = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pr-context') {
|
||||
options.prContext = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--target-url') {
|
||||
options.targetUrl = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pass-label') {
|
||||
options.passLabel = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--dry-run') {
|
||||
options.dryRun = true
|
||||
continue
|
||||
}
|
||||
|
||||
throw new Error(`Unknown argument: ${argument}`)
|
||||
}
|
||||
|
||||
return options
|
||||
}
|
||||
|
||||
function normalizePlatformName(value: string): string {
|
||||
const slug = value
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
|
||||
return slug.length > 0 ? slug : 'unknown-platform'
|
||||
}
|
||||
|
||||
export function extractPlatformFromArtifactDirName(dirName: string): string {
|
||||
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
|
||||
return normalizePlatformName(matchedValue ?? dirName)
|
||||
}
|
||||
|
||||
function extractPlatformFromVideoPath(videoPath: string): string {
|
||||
const artifactDirName = basename(dirname(videoPath))
|
||||
return extractPlatformFromArtifactDirName(artifactDirName)
|
||||
}
|
||||
|
||||
export function pickLatestVideosByPlatform(
|
||||
candidates: VideoCandidate[]
|
||||
): VideoCandidate[] {
|
||||
const latestByPlatform = new Map<string, VideoCandidate>()
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const current = latestByPlatform.get(candidate.platformName)
|
||||
if (!current || candidate.mtimeMs > current.mtimeMs) {
|
||||
latestByPlatform.set(candidate.platformName, candidate)
|
||||
}
|
||||
}
|
||||
|
||||
return [...latestByPlatform.values()].sort((a, b) =>
|
||||
a.platformName.localeCompare(b.platformName)
|
||||
)
|
||||
}
|
||||
|
||||
function toProjectRelativePath(targetPath: string): string {
|
||||
const relativePath = relative(process.cwd(), targetPath)
|
||||
if (relativePath.startsWith('.')) {
|
||||
return relativePath
|
||||
}
|
||||
return `./${relativePath}`
|
||||
}
|
||||
|
||||
function errorToString(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
|
||||
function normalizePathForMatch(value: string): string {
|
||||
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
|
||||
}
|
||||
|
||||
export function selectVideoCandidateByFile(
|
||||
candidates: VideoCandidate[],
|
||||
options: { artifactsDir: string; videoFile: string }
|
||||
): VideoCandidate {
|
||||
const requestedValue = options.videoFile.trim()
|
||||
if (requestedValue.length === 0) {
|
||||
throw new Error('--video-file is required')
|
||||
}
|
||||
|
||||
const artifactsRoot = resolve(options.artifactsDir)
|
||||
const requestedAbsolutePath = resolve(requestedValue)
|
||||
const requestedPathKey = normalizePathForMatch(requestedValue)
|
||||
|
||||
const matches = candidates.filter((candidate) => {
|
||||
const candidateAbsolutePath = resolve(candidate.videoPath)
|
||||
if (candidateAbsolutePath === requestedAbsolutePath) {
|
||||
return true
|
||||
}
|
||||
|
||||
const candidateBaseName = basename(candidate.videoPath)
|
||||
if (candidateBaseName === requestedValue) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToCwd = normalizePathForMatch(
|
||||
relative(process.cwd(), candidateAbsolutePath)
|
||||
)
|
||||
if (relativeToCwd === requestedPathKey) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToArtifacts = normalizePathForMatch(
|
||||
relative(artifactsRoot, candidateAbsolutePath)
|
||||
)
|
||||
return relativeToArtifacts === requestedPathKey
|
||||
})
|
||||
|
||||
if (matches.length === 1) {
|
||||
return matches[0]
|
||||
}
|
||||
|
||||
if (matches.length === 0) {
|
||||
const availableVideos = candidates.map((candidate) =>
|
||||
toProjectRelativePath(candidate.videoPath)
|
||||
)
|
||||
throw new Error(
|
||||
[
|
||||
`No video matched --video-file "${options.videoFile}".`,
|
||||
'Available videos:',
|
||||
...availableVideos.map((videoPath) => `- ${videoPath}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
[
|
||||
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
|
||||
'Please pass a more specific path.',
|
||||
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
async function collectVideoCandidates(
|
||||
artifactsDir: string
|
||||
): Promise<VideoCandidate[]> {
|
||||
const absoluteArtifactsDir = resolve(artifactsDir)
|
||||
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
|
||||
cwd: absoluteArtifactsDir,
|
||||
absolute: true,
|
||||
nodir: true
|
||||
}).sort()
|
||||
|
||||
const candidates = await Promise.all(
|
||||
videoPaths.map(async (videoPath) => {
|
||||
const videoStat = await stat(videoPath)
|
||||
return {
|
||||
platformName: extractPlatformFromVideoPath(videoPath),
|
||||
videoPath,
|
||||
mtimeMs: videoStat.mtimeMs
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
return candidates
|
||||
}
|
||||
|
||||
function getMimeType(filePath: string): string {
|
||||
const ext = extname(filePath).toLowerCase()
|
||||
const mimeMap: Record<string, string> = {
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime',
|
||||
'.avi': 'video/x-msvideo',
|
||||
'.mkv': 'video/x-matroska',
|
||||
'.m4v': 'video/mp4'
|
||||
}
|
||||
return mimeMap[ext] || 'video/mp4'
|
||||
}
|
||||
|
||||
function buildReviewPrompt(options: {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
prContext: string
|
||||
isComparative: boolean
|
||||
}): string {
|
||||
const { platformName, videoPath, prContext, isComparative } = options
|
||||
|
||||
if (isComparative) {
|
||||
return buildComparativePrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
return buildSingleVideoPrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
function buildComparativePrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
|
||||
'',
|
||||
'You are given TWO videos:',
|
||||
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
|
||||
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
|
||||
'',
|
||||
'Both videos show the same test steps executed on different code versions.',
|
||||
''
|
||||
]
|
||||
|
||||
if (prContext) {
|
||||
lines.push('## PR Context', prContext, '')
|
||||
}
|
||||
|
||||
lines.push(
|
||||
'## Your Task',
|
||||
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'',
|
||||
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
|
||||
' Describe what you observe — this establishes the baseline.',
|
||||
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
|
||||
'3. **Comparison**: What specifically changed between before and after?',
|
||||
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
|
||||
' that were NOT present in the BEFORE video?',
|
||||
'',
|
||||
'Note: Brief black frames during page transitions are NORMAL.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible differences. Avoid speculation.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
|
||||
'',
|
||||
'## Behavior Changes',
|
||||
'Summarize ALL behavioral differences as a markdown TABLE:',
|
||||
'| Behavior | Before (main) | After (PR) | Verdict |',
|
||||
'',
|
||||
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
|
||||
'- **Before (main)**: how it works/looks in the BEFORE video',
|
||||
'- **After (PR)**: how it works/looks in the AFTER video',
|
||||
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
|
||||
'',
|
||||
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
|
||||
'that were tested, so reviewers can confirm nothing was missed.',
|
||||
'',
|
||||
'## Timeline Comparison',
|
||||
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
|
||||
'| Time | Type | Severity | Before (main) | After (PR) |',
|
||||
'',
|
||||
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
|
||||
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
|
||||
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
|
||||
'- **Before (main)**: what is observed in the BEFORE video at that time',
|
||||
'- **After (PR)**: what is observed in the AFTER video at that time',
|
||||
'',
|
||||
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
|
||||
'omit that row. Focus on meaningful differences, not narrating every frame.',
|
||||
'',
|
||||
'## Confirmed Issues',
|
||||
'For each issue, use this exact format:',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'',
|
||||
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
|
||||
'',
|
||||
'**Evidence:** [What you observed at the given timestamp in which video]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk',
|
||||
'(Assess whether the PR achieves its goal based on the before/after comparison)',
|
||||
'',
|
||||
'## Verdict',
|
||||
'End your report with this EXACT JSON block (no markdown fence):',
|
||||
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
|
||||
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
|
||||
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
|
||||
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
function buildSingleVideoPrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer reviewing a UI test session recording.',
|
||||
'',
|
||||
'## ANTI-HALLUCINATION RULES (READ FIRST)',
|
||||
'- Describe ONLY what you can directly observe in the video frames',
|
||||
'- NEVER infer or assume what "must have happened" between frames',
|
||||
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
|
||||
'- Your job is to be a CAMERA — report facts, not interpretations',
|
||||
''
|
||||
]
|
||||
|
||||
const isIssueContext =
|
||||
prContext &&
|
||||
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
|
||||
prContext
|
||||
)
|
||||
|
||||
if (prContext) {
|
||||
lines.push(
|
||||
'## Phase 1: Blind Observation (describe what you SEE and HEAR)',
|
||||
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
|
||||
'- What elements does the user click/hover/type?',
|
||||
'- What dialogs/menus open and close?',
|
||||
'- What keyboard indicators appear? (look for subtitle overlays)',
|
||||
'- What is the BEFORE state and AFTER state of each action?',
|
||||
'- **Audio**: Does the video have a TTS narration audio track? If yes, transcribe what the voice says. This narration describes the bug being reproduced.',
|
||||
'',
|
||||
'## Phase 2: Compare against expected behavior',
|
||||
'Now compare your observations against the context below.',
|
||||
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
|
||||
''
|
||||
)
|
||||
|
||||
if (isIssueContext) {
|
||||
lines.push(
|
||||
'## Issue Context',
|
||||
prContext,
|
||||
'',
|
||||
'## Comparison Questions',
|
||||
'1. Did the video perform the reproduction steps described in the issue?',
|
||||
'2. Did your Phase 1 observations show the reported bug behavior?',
|
||||
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
|
||||
''
|
||||
)
|
||||
} else {
|
||||
lines.push(
|
||||
'## PR Context',
|
||||
prContext,
|
||||
'',
|
||||
'## Comparison Questions',
|
||||
'1. Did the video test the specific behavior the PR changes?',
|
||||
'2. Did your Phase 1 observations show the expected before/after difference?',
|
||||
'3. If the test was incomplete or inconclusive, say so honestly.',
|
||||
''
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
lines.push(
|
||||
`Review this QA session video for platform "${platformName}".`,
|
||||
`Source video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'The video shows the full test session — analyze it chronologically.',
|
||||
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
|
||||
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible problems and avoid speculation.',
|
||||
'If confidence is low, mark it explicitly.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
isIssueContext
|
||||
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
|
||||
: prContext
|
||||
? '(Explain what the PR intended and whether the video confirms it works)'
|
||||
: '',
|
||||
'## Confirmed Issues',
|
||||
'For each confirmed issue, use this exact format (one block per issue):',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`HIGH` `01:03` `Confidence: High`',
|
||||
'',
|
||||
'[Description of the issue — what went wrong and what was expected]',
|
||||
'',
|
||||
'**Evidence:** [What you observed in the video at the given timestamp]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'Do NOT use a table for issues — use the block format above.',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk',
|
||||
'',
|
||||
'## Narration',
|
||||
'If the video contains a TTS audio narration track, transcribe it here.',
|
||||
'If there is no audio or the video is silent, write "No narration detected."',
|
||||
'',
|
||||
'## Verdict',
|
||||
'End your report with this EXACT JSON block (no markdown fence):',
|
||||
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low", "narrationDetected": true | false}',
|
||||
'- REPRODUCED: the bug/behavior is clearly visible in the video',
|
||||
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
|
||||
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient',
|
||||
'- narrationDetected: true if you heard TTS voice narration in the video, false if silent'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
|
||||
|
||||
async function readVideoFile(videoPath: string): Promise<Buffer> {
|
||||
const fileStat = await stat(videoPath)
|
||||
if (fileStat.size > MAX_VIDEO_BYTES) {
|
||||
throw new Error(
|
||||
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
|
||||
)
|
||||
}
|
||||
return readFile(videoPath)
|
||||
}
|
||||
|
||||
async function requestGeminiReview(options: {
|
||||
apiKey: string
|
||||
model: string
|
||||
platformName: string
|
||||
videoPath: string
|
||||
beforeVideoPath: string
|
||||
timeoutMs: number
|
||||
prContext: string
|
||||
}): Promise<string> {
|
||||
const genAI = new GoogleGenerativeAI(options.apiKey)
|
||||
const model = genAI.getGenerativeModel({ model: options.model })
|
||||
|
||||
const isComparative = options.beforeVideoPath.length > 0
|
||||
const prompt = buildReviewPrompt({
|
||||
platformName: options.platformName,
|
||||
videoPath: options.videoPath,
|
||||
prContext: options.prContext,
|
||||
isComparative
|
||||
})
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
if (isComparative) {
|
||||
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
|
||||
parts.push(
|
||||
{ text: 'Video 1 — BEFORE (main branch):' },
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.beforeVideoPath),
|
||||
data: beforeBuffer.toString('base64')
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
const afterBuffer = await readVideoFile(options.videoPath)
|
||||
if (isComparative) {
|
||||
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
|
||||
}
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.videoPath),
|
||||
data: afterBuffer.toString('base64')
|
||||
}
|
||||
})
|
||||
|
||||
const result = await model.generateContent(parts, {
|
||||
timeout: options.timeoutMs
|
||||
})
|
||||
const response = result.response
|
||||
const text = response.text()
|
||||
|
||||
if (!text || text.trim().length === 0) {
|
||||
throw new Error('Gemini API returned no output text')
|
||||
}
|
||||
|
||||
return text.trim()
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
function buildReportMarkdown(input: {
|
||||
platformName: string
|
||||
model: string
|
||||
videoPath: string
|
||||
videoSizeBytes: number
|
||||
beforeVideoPath?: string
|
||||
beforeVideoSizeBytes?: number
|
||||
reviewText: string
|
||||
targetUrl?: string
|
||||
}): string {
|
||||
const headerLines = [
|
||||
`# ${input.platformName} QA Video Report`,
|
||||
'',
|
||||
`- Generated at: ${new Date().toISOString()}`,
|
||||
`- Model: \`${input.model}\``
|
||||
]
|
||||
|
||||
if (input.targetUrl) {
|
||||
headerLines.push(`- Target: ${input.targetUrl}`)
|
||||
}
|
||||
|
||||
if (input.beforeVideoPath) {
|
||||
headerLines.push(
|
||||
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
|
||||
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
|
||||
'- Mode: **Comparative (before/after)**'
|
||||
)
|
||||
} else {
|
||||
headerLines.push(
|
||||
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
|
||||
`- Video size: ${formatBytes(input.videoSizeBytes)}`
|
||||
)
|
||||
}
|
||||
|
||||
headerLines.push('', '## AI Review', '')
|
||||
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
|
||||
}
|
||||
|
||||
async function reviewVideo(
|
||||
video: VideoCandidate,
|
||||
options: CliOptions,
|
||||
apiKey: string
|
||||
): Promise<void> {
|
||||
let prContext = ''
|
||||
if (options.prContext) {
|
||||
try {
|
||||
prContext = await readFile(options.prContext, 'utf-8')
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
|
||||
)
|
||||
} catch {
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const beforeVideoPath = options.beforeVideo
|
||||
? resolve(options.beforeVideo)
|
||||
: ''
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
|
||||
)
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
|
||||
)
|
||||
|
||||
const reviewText = await requestGeminiReview({
|
||||
apiKey,
|
||||
model: options.model,
|
||||
platformName: video.platformName,
|
||||
videoPath: video.videoPath,
|
||||
beforeVideoPath,
|
||||
timeoutMs: options.requestTimeoutMs,
|
||||
prContext
|
||||
})
|
||||
|
||||
const videoStat = await stat(video.videoPath)
|
||||
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
|
||||
const outputPath = resolve(
|
||||
options.outputDir,
|
||||
`${video.platformName}${passSegment}-qa-video-report.md`
|
||||
)
|
||||
|
||||
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
|
||||
platformName: video.platformName,
|
||||
model: options.model,
|
||||
videoPath: video.videoPath,
|
||||
videoSizeBytes: videoStat.size,
|
||||
reviewText,
|
||||
targetUrl: options.targetUrl || undefined
|
||||
}
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
reportInput.beforeVideoPath = beforeVideoPath
|
||||
reportInput.beforeVideoSizeBytes = beforeStat.size
|
||||
}
|
||||
|
||||
const reportMarkdown = buildReportMarkdown(reportInput)
|
||||
|
||||
await mkdir(dirname(outputPath), { recursive: true })
|
||||
await writeFile(outputPath, reportMarkdown, 'utf-8')
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
|
||||
)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const options = parseCliOptions(process.argv.slice(2))
|
||||
const candidates = await collectVideoCandidates(options.artifactsDir)
|
||||
|
||||
if (candidates.length === 0) {
|
||||
process.stdout.write(
|
||||
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
const selectedVideo = selectVideoCandidateByFile(candidates, {
|
||||
artifactsDir: options.artifactsDir,
|
||||
videoFile: options.videoFile
|
||||
})
|
||||
|
||||
process.stdout.write(
|
||||
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
|
||||
)
|
||||
|
||||
if (options.dryRun) {
|
||||
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
|
||||
return
|
||||
}
|
||||
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) {
|
||||
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
|
||||
}
|
||||
|
||||
await reviewVideo(selectedVideo, options, apiKey)
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
void main().catch((error: unknown) => {
|
||||
const message = errorToString(error)
|
||||
process.stderr.write(`qa-video-review failed: ${message}\n`)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
513
.claude/skills/comfy-qa/scripts/qa.ts
Normal file
513
.claude/skills/comfy-qa/scripts/qa.ts
Normal file
@@ -0,0 +1,513 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA CLI — simplified entry point for local & CI QA runs
|
||||
*
|
||||
* Usage:
|
||||
* pnpm qa 10253 # auto-detects issue vs PR
|
||||
* pnpm qa https://github.com/.../pull/10270
|
||||
* pnpm qa 10270 -t base # test PR base (reproduce bug)
|
||||
* pnpm qa 10270 -t both # test base + head
|
||||
* pnpm qa --uncommitted # test local uncommitted changes
|
||||
*
|
||||
* Automatically loads .env.local / .env for GEMINI_API_KEY, ANTHROPIC_API_KEY.
|
||||
* Results are written to .comfy-qa/<number>/ by default.
|
||||
*/
|
||||
|
||||
import { parseArgs } from 'node:util'
|
||||
import { config } from 'dotenv'
|
||||
import { existsSync, mkdirSync, writeFileSync } from 'fs'
|
||||
import { dirname, resolve } from 'path'
|
||||
import { execSync, spawn, spawnSync } from 'child_process'
|
||||
import { fileURLToPath } from 'url'
|
||||
|
||||
// ── Constants ──
|
||||
|
||||
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
|
||||
const RECORD_SCRIPT = resolve(SCRIPT_DIR, 'qa-record.ts')
|
||||
const DEFAULT_REPO = 'Comfy-Org/ComfyUI_frontend'
|
||||
const VALID_TARGETS = ['head', 'base', 'both'] as const
|
||||
const CLOUD_FALLBACK_URL = 'https://testcloud.comfy.org/'
|
||||
type PrTarget = (typeof VALID_TARGETS)[number]
|
||||
type TargetType = 'issue' | 'pr'
|
||||
|
||||
// ── Load .env.local / .env ──
|
||||
|
||||
for (const f of ['.env.local', '.env']) {
|
||||
if (existsSync(f)) {
|
||||
config({ path: f })
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// ── Parse CLI ──
|
||||
|
||||
const { values, positionals } = tryParseArgs()
|
||||
|
||||
if (values.help) {
|
||||
printUsage()
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const serverUrl =
|
||||
values.url || process.env.DEV_SERVER_COMFYUI_URL || 'http://127.0.0.1:8188'
|
||||
|
||||
const prTarget = values.target as PrTarget
|
||||
if (!VALID_TARGETS.includes(prTarget)) {
|
||||
console.error(
|
||||
`Invalid --target "${prTarget}". Must be one of: ${VALID_TARGETS.join(', ')}`
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Ensure server is reachable (may fall back to cloud) ──
|
||||
|
||||
const resolvedServerUrl = await ensureServer(serverUrl)
|
||||
|
||||
// ── Dispatch by mode ──
|
||||
|
||||
if (values.uncommitted) {
|
||||
runUncommitted()
|
||||
} else {
|
||||
const input = positionals[0]
|
||||
if (!input) {
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
runTarget(input)
|
||||
}
|
||||
|
||||
// ── Mode: uncommitted changes ──
|
||||
|
||||
function runUncommitted(): never {
|
||||
const diff = shell('git diff && git diff --staged')
|
||||
if (!diff.trim()) {
|
||||
console.error('No uncommitted changes found')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const outputDir = resolveOutputDir('.comfy-qa/local')
|
||||
const diffFile = writeTmpFile(outputDir, 'uncommitted.diff', diff)
|
||||
|
||||
logHeader({ label: 'uncommitted changes', outputDir })
|
||||
const code = runQaRecord('after', diffFile, outputDir)
|
||||
exit(code, outputDir)
|
||||
}
|
||||
|
||||
// ── Mode: issue or PR by number/URL ──
|
||||
|
||||
function runTarget(input: string): never {
|
||||
const { targetType, number, repo } = resolveTarget(input)
|
||||
const outputDir = resolveOutputDir(`.comfy-qa/${number}`)
|
||||
|
||||
logHeader({
|
||||
label: `${targetType} #${number} (${repo})`,
|
||||
outputDir,
|
||||
extra: targetType === 'pr' ? `Target: ${prTarget}` : undefined
|
||||
})
|
||||
|
||||
const diffFile =
|
||||
targetType === 'issue'
|
||||
? fetchIssue(number, repo, outputDir)
|
||||
: fetchPR(number, repo, outputDir)
|
||||
|
||||
let exitCode: number
|
||||
if (targetType === 'issue') {
|
||||
exitCode = runQaRecord('reproduce', diffFile, outputDir)
|
||||
} else if (prTarget === 'both') {
|
||||
exitCode = runPrBoth(diffFile, outputDir)
|
||||
} else if (prTarget === 'base') {
|
||||
exitCode = runQaRecord('before', diffFile, outputDir)
|
||||
} else {
|
||||
exitCode = runQaRecord('after', diffFile, outputDir)
|
||||
}
|
||||
|
||||
exit(exitCode, outputDir)
|
||||
}
|
||||
|
||||
// ── PR both phases ──
|
||||
|
||||
function runPrBoth(diffFile: string, outputDir: string): number {
|
||||
console.warn('\n=== Phase 1: Reproduce bug on base ===')
|
||||
const baseDir = resolve(outputDir, 'base')
|
||||
mkdirSync(baseDir, { recursive: true })
|
||||
const baseCode = runQaRecord('before', diffFile, baseDir)
|
||||
if (baseCode !== 0) {
|
||||
console.warn('Base phase failed, continuing to head...')
|
||||
}
|
||||
|
||||
console.warn('\n=== Phase 2: Demonstrate fix on head ===')
|
||||
const headDir = resolve(outputDir, 'head')
|
||||
mkdirSync(headDir, { recursive: true })
|
||||
return runQaRecord('after', diffFile, headDir)
|
||||
}
|
||||
|
||||
// ── Target resolution ──
|
||||
|
||||
function resolveTarget(input: string): {
|
||||
targetType: TargetType
|
||||
number: string
|
||||
repo: string
|
||||
} {
|
||||
const urlMatch = input.match(
|
||||
/github\.com\/([^/]+\/[^/]+)\/(issues|pull)\/(\d+)/
|
||||
)
|
||||
|
||||
if (urlMatch) {
|
||||
return {
|
||||
repo: urlMatch[1],
|
||||
targetType: urlMatch[2] === 'pull' ? 'pr' : 'issue',
|
||||
number: urlMatch[3]
|
||||
}
|
||||
}
|
||||
|
||||
if (/^\d+$/.test(input)) {
|
||||
return {
|
||||
repo: DEFAULT_REPO,
|
||||
targetType: detectType(input, DEFAULT_REPO),
|
||||
number: input
|
||||
}
|
||||
}
|
||||
|
||||
console.error(`Cannot parse target: ${input}`)
|
||||
console.error('Expected a GitHub URL or issue/PR number')
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
function detectType(number: string, repo: string): TargetType {
|
||||
try {
|
||||
const result = execSync(
|
||||
`gh api repos/${repo}/issues/${number} --jq 'has("pull_request")'`,
|
||||
{ encoding: 'utf-8', timeout: 15000, stdio: ['pipe', 'pipe', 'pipe'] }
|
||||
)
|
||||
return result.trim() === 'true' ? 'pr' : 'issue'
|
||||
} catch {
|
||||
return 'issue'
|
||||
}
|
||||
}
|
||||
|
||||
// ── Data fetching ──
|
||||
|
||||
function fetchIssue(number: string, repo: string, outputDir: string): string {
|
||||
console.warn(`Fetching issue #${number}...`)
|
||||
const body = shell(
|
||||
`gh issue view ${number} --repo ${repo} --json title,body,labels --jq '"Title: " + .title + "\\n\\nLabels: " + ([.labels[].name] | join(", ")) + "\\n\\n" + .body'`
|
||||
)
|
||||
return writeTmpFile(outputDir, `issue-${number}.txt`, body)
|
||||
}
|
||||
|
||||
function fetchPR(number: string, repo: string, outputDir: string): string {
|
||||
console.warn(`Fetching PR #${number}...`)
|
||||
const prJson = shell(
|
||||
`gh pr view ${number} --repo ${repo} --json title,body,baseRefName,headRefName,baseRefOid,headRefOid`
|
||||
)
|
||||
const pr = JSON.parse(prJson) as {
|
||||
title: string
|
||||
body: string
|
||||
baseRefName: string
|
||||
headRefName: string
|
||||
baseRefOid: string
|
||||
headRefOid: string
|
||||
}
|
||||
|
||||
console.warn(` Base: ${pr.baseRefName} (${pr.baseRefOid.slice(0, 8)})`)
|
||||
console.warn(` Head: ${pr.headRefName} (${pr.headRefOid.slice(0, 8)})`)
|
||||
|
||||
let diff = ''
|
||||
try {
|
||||
diff = shell(`gh pr diff ${number} --repo ${repo}`)
|
||||
} catch {
|
||||
console.warn('Could not fetch PR diff')
|
||||
}
|
||||
|
||||
writeTmpFile(
|
||||
outputDir,
|
||||
'refs.json',
|
||||
JSON.stringify(
|
||||
{
|
||||
base: { ref: pr.baseRefName, sha: pr.baseRefOid },
|
||||
head: { ref: pr.headRefName, sha: pr.headRefOid }
|
||||
},
|
||||
null,
|
||||
2
|
||||
)
|
||||
)
|
||||
|
||||
return writeTmpFile(
|
||||
outputDir,
|
||||
`pr-${number}.txt`,
|
||||
`Title: ${pr.title}\n\n${pr.body}\n\n--- DIFF ---\n\n${diff}`
|
||||
)
|
||||
}
|
||||
|
||||
// ── QA record runner ──
|
||||
|
||||
function runQaRecord(
|
||||
mode: string,
|
||||
diffFile: string,
|
||||
outputDir: string
|
||||
): number {
|
||||
console.warn(`\nStarting QA ${mode} mode...\n`)
|
||||
const r = spawnSync(
|
||||
'pnpm',
|
||||
[
|
||||
'exec',
|
||||
'tsx',
|
||||
RECORD_SCRIPT,
|
||||
'--mode',
|
||||
mode,
|
||||
'--diff',
|
||||
diffFile,
|
||||
'--output-dir',
|
||||
outputDir,
|
||||
'--url',
|
||||
resolvedServerUrl
|
||||
],
|
||||
{ stdio: 'inherit', env: process.env }
|
||||
)
|
||||
return r.status ?? 1
|
||||
}
|
||||
|
||||
// ── Server management ──
|
||||
|
||||
async function ensureServer(url: string): Promise<string> {
|
||||
if (await isReachable(url)) {
|
||||
console.warn(`Server OK: ${url}`)
|
||||
return url
|
||||
}
|
||||
|
||||
console.warn(`Server not reachable at ${url}, attempting auto-start...`)
|
||||
const port = new URL(url).port || '8188'
|
||||
|
||||
// Strategy 1: comfy-cli (pip install comfy-cli)
|
||||
try {
|
||||
execSync('which comfy', { stdio: 'pipe' })
|
||||
console.warn('Starting ComfyUI via comfy-cli...')
|
||||
const proc = spawn(
|
||||
'comfy',
|
||||
['launch', '--background', '--', '--cpu', '--port', port],
|
||||
{
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
}
|
||||
)
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
} catch {
|
||||
// comfy-cli not available
|
||||
}
|
||||
|
||||
// Strategy 2: python main.py from TEST_COMFYUI_DIR or .comfy-qa/ComfyUI
|
||||
const comfyDir = findComfyUIDir()
|
||||
if (comfyDir) {
|
||||
console.warn(`Starting ComfyUI from ${comfyDir}...`)
|
||||
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
|
||||
cwd: comfyDir,
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
})
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
}
|
||||
|
||||
// Strategy 3: clone ComfyUI and start
|
||||
const cloneDir = resolve('.comfy-qa/ComfyUI')
|
||||
if (!existsSync(resolve(cloneDir, 'main.py'))) {
|
||||
console.warn('No ComfyUI installation found, cloning...')
|
||||
try {
|
||||
execSync(
|
||||
`git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git "${cloneDir}"`,
|
||||
{ stdio: 'inherit', timeout: 120000 }
|
||||
)
|
||||
console.warn('Installing ComfyUI dependencies...')
|
||||
execSync('pip install -r requirements.txt', {
|
||||
cwd: cloneDir,
|
||||
stdio: 'inherit',
|
||||
timeout: 300000
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Clone/install failed: ${err instanceof Error ? err.message : err}`
|
||||
)
|
||||
}
|
||||
}
|
||||
if (existsSync(resolve(cloneDir, 'main.py'))) {
|
||||
console.warn(`Starting ComfyUI from ${cloneDir}...`)
|
||||
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
|
||||
cwd: cloneDir,
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
})
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
}
|
||||
|
||||
// Strategy 4: fallback to testcloud
|
||||
console.warn(`Local server failed. Falling back to ${CLOUD_FALLBACK_URL}`)
|
||||
if (await isReachable(CLOUD_FALLBACK_URL)) {
|
||||
console.warn(`Cloud server OK: ${CLOUD_FALLBACK_URL}`)
|
||||
return CLOUD_FALLBACK_URL
|
||||
}
|
||||
|
||||
console.error(`
|
||||
No ComfyUI server available. Tried:
|
||||
1. ${url} (not reachable)
|
||||
2. comfy-cli (not installed)
|
||||
3. Local ComfyUI installation (not found)
|
||||
4. ${CLOUD_FALLBACK_URL} (not reachable)
|
||||
|
||||
Install: pip install comfy-cli && comfy install && comfy launch --cpu
|
||||
`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
function findComfyUIDir(): string | undefined {
|
||||
const candidates = [
|
||||
process.env.TEST_COMFYUI_DIR,
|
||||
resolve('.comfy-qa/ComfyUI'),
|
||||
'/home/ComfyUI'
|
||||
]
|
||||
return candidates.find((d) => d && existsSync(resolve(d, 'main.py')))
|
||||
}
|
||||
|
||||
async function isReachable(url: string): Promise<boolean> {
|
||||
try {
|
||||
const controller = new AbortController()
|
||||
const timeout = setTimeout(() => controller.abort(), 5000)
|
||||
const res = await fetch(url, { signal: controller.signal })
|
||||
clearTimeout(timeout)
|
||||
return res.ok || res.status === 200 || res.status === 304
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForServer(url: string, timeoutMs: number): Promise<void> {
|
||||
const start = Date.now()
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (await isReachable(url)) {
|
||||
console.warn('Server is ready')
|
||||
return
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 2000))
|
||||
}
|
||||
console.error(`Server did not start within ${timeoutMs / 1000}s`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Utilities ──
|
||||
|
||||
function shell(cmd: string): string {
|
||||
return execSync(cmd, { encoding: 'utf-8', timeout: 30000 })
|
||||
}
|
||||
|
||||
function writeTmpFile(
|
||||
outputDir: string,
|
||||
filename: string,
|
||||
content: string
|
||||
): string {
|
||||
const tmpDir = resolve(outputDir, '.tmp')
|
||||
mkdirSync(tmpDir, { recursive: true })
|
||||
const filePath = resolve(tmpDir, filename)
|
||||
writeFileSync(filePath, content)
|
||||
return filePath
|
||||
}
|
||||
|
||||
function resolveOutputDir(defaultPath: string): string {
|
||||
const dir = values.output ? resolve(values.output) : resolve(defaultPath)
|
||||
mkdirSync(dir, { recursive: true })
|
||||
return dir
|
||||
}
|
||||
|
||||
function logHeader(opts: { label: string; outputDir: string; extra?: string }) {
|
||||
console.warn(`QA target: ${opts.label}`)
|
||||
console.warn(`Output: ${opts.outputDir}`)
|
||||
console.warn(`Server: ${resolvedServerUrl}`)
|
||||
if (values.ref) console.warn(`Ref: ${values.ref}`)
|
||||
if (opts.extra) console.warn(opts.extra)
|
||||
}
|
||||
|
||||
function exit(code: number, outputDir: string): never {
|
||||
console.warn('\n=== QA Complete ===')
|
||||
console.warn(`Results: ${outputDir}`)
|
||||
try {
|
||||
console.warn(shell(`ls -la "${outputDir}"`))
|
||||
} catch {
|
||||
// not critical
|
||||
}
|
||||
process.exit(code)
|
||||
}
|
||||
|
||||
function tryParseArgs() {
|
||||
try {
|
||||
const parsed = parseArgs({
|
||||
args: process.argv.slice(2),
|
||||
options: {
|
||||
target: { type: 'string', short: 't', default: 'head' },
|
||||
uncommitted: { type: 'boolean', default: false },
|
||||
url: { type: 'string', default: '' },
|
||||
ref: { type: 'string', default: '' },
|
||||
output: { type: 'string', short: 'o', default: '' },
|
||||
help: { type: 'boolean', short: 'h', default: false }
|
||||
},
|
||||
allowPositionals: true,
|
||||
strict: true
|
||||
})
|
||||
return {
|
||||
values: parsed.values as {
|
||||
target: string
|
||||
uncommitted: boolean
|
||||
url: string
|
||||
ref: string
|
||||
output: string
|
||||
help: boolean
|
||||
},
|
||||
positionals: parsed.positionals
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`Error: ${err instanceof Error ? err.message : err}\n`)
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
function printUsage() {
|
||||
console.warn(`
|
||||
QA CLI — Reproduce issues & test PRs for ComfyUI frontend
|
||||
|
||||
Usage:
|
||||
pnpm qa <number|url> [options]
|
||||
pnpm qa --uncommitted
|
||||
|
||||
Targets:
|
||||
10253 Number (auto-detects issue vs PR via gh CLI)
|
||||
https://github.com/Comfy-Org/ComfyUI_frontend/issues/10253
|
||||
https://github.com/Comfy-Org/ComfyUI_frontend/pull/10270
|
||||
|
||||
Options:
|
||||
-t, --target <head|base|both>
|
||||
For PRs: which ref to test (default: head)
|
||||
head — test the fix (PR head)
|
||||
base — reproduce the bug (PR base)
|
||||
both — base then head
|
||||
--uncommitted Test local uncommitted changes
|
||||
--url <url> ComfyUI server URL (default: from .env or http://127.0.0.1:8188)
|
||||
--ref <ref> Git ref to test against
|
||||
-o, --output <dir> Override output directory (default: .comfy-qa/<number>)
|
||||
-h, --help Show this help
|
||||
|
||||
Environment (auto-loaded from .env.local or .env):
|
||||
GEMINI_API_KEY Required — used for PR analysis, video review, TTS
|
||||
ANTHROPIC_API_KEY Optional locally — Claude Agent SDK auto-detects Claude Code session
|
||||
|
||||
Examples:
|
||||
pnpm qa 10253 # reproduce an issue
|
||||
pnpm qa 10270 # test PR head (the fix)
|
||||
pnpm qa 10270 -t base # reproduce bug on PR base
|
||||
pnpm qa 10270 -t both # test base + head
|
||||
pnpm qa --uncommitted # test local changes
|
||||
`)
|
||||
}
|
||||
361
.claude/skills/ticket-intake/SKILL.md
Normal file
361
.claude/skills/ticket-intake/SKILL.md
Normal file
@@ -0,0 +1,361 @@
|
||||
---
|
||||
name: ticket-intake
|
||||
description: 'Parse ticket URL (Notion or GitHub), extract all data, initialize pipeline run. Use when starting work on a new ticket or when asked to pick up a ticket.'
|
||||
---
|
||||
|
||||
# Ticket Intake
|
||||
|
||||
Parses a ticket URL from supported sources (Notion or GitHub), extracts all relevant information, and creates a ticket in the pipeline API.
|
||||
|
||||
> **🚨 CRITICAL REQUIREMENT**: This skill MUST register the ticket in the Pipeline API and update the source (Notion/GitHub). If these steps are skipped, the entire pipeline breaks. See [Mandatory API Calls](#mandatory-api-calls-execute-all-three) below.
|
||||
|
||||
## Supported Sources
|
||||
|
||||
| Source | URL Pattern | Provider File |
|
||||
| ------ | --------------------------------------------------- | --------------------- |
|
||||
| Notion | `https://notion.so/...` `https://www.notion.so/...` | `providers/notion.md` |
|
||||
| GitHub | `https://github.com/{owner}/{repo}/issues/{n}` | `providers/github.md` |
|
||||
|
||||
## Quick Start
|
||||
|
||||
When given a ticket URL:
|
||||
|
||||
1. **Detect source type** from URL pattern
|
||||
2. **Load provider-specific logic** from `providers/` directory
|
||||
3. Fetch ticket content via appropriate API
|
||||
4. Extract and normalize properties to common schema
|
||||
5. **Register ticket in pipeline API** ← MANDATORY
|
||||
6. **Update source** (Notion status / GitHub comment) ← MANDATORY
|
||||
7. **Run verification script** to confirm API registration
|
||||
8. Output summary and handoff to `research-orchestrator`
|
||||
|
||||
## Configuration
|
||||
|
||||
Uses the **production API** by default. No configuration needed for read operations.
|
||||
|
||||
**Defaults (no setup required):**
|
||||
|
||||
- API URL: `https://api-gateway-856475788601.us-central1.run.app`
|
||||
- Read-only endpoints at `/public/*` require no authentication
|
||||
|
||||
**For write operations** (transitions, creating tickets), set:
|
||||
|
||||
```bash
|
||||
export PIPELINE_API_KEY="..." # Get from GCP Secret Manager or ask admin
|
||||
```
|
||||
|
||||
**Optional (for local working artifacts):**
|
||||
|
||||
```bash
|
||||
PIPELINE_DIR="${PIPELINE_DIR:-$HOME/repos/ticket-to-pr-pipeline}"
|
||||
```
|
||||
|
||||
## Mandatory API Calls (Execute ALL Three)
|
||||
|
||||
**⚠️ These three API calls are the ENTIRE POINT of this skill. Without them, the ticket is invisible to the pipeline, downstream skills will fail, and Notion status won't update.**
|
||||
|
||||
**You MUST make these HTTP requests.** Use `curl` from bash — do not just read this as documentation.
|
||||
|
||||
### Call 1: Create Ticket
|
||||
|
||||
```bash
|
||||
API_URL="${PIPELINE_API_URL:-https://api-gateway-856475788601.us-central1.run.app}"
|
||||
API_KEY="${PIPELINE_API_KEY}"
|
||||
|
||||
curl -s -X POST "${API_URL}/v1/tickets" \
|
||||
-H "Authorization: Bearer ${API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
|
||||
-d '{
|
||||
"notion_page_id": "NOTION_PAGE_UUID_HERE",
|
||||
"title": "TICKET_TITLE_HERE",
|
||||
"source": "notion",
|
||||
"metadata": {
|
||||
"description": "DESCRIPTION_HERE",
|
||||
"priority": "High",
|
||||
"labels": [],
|
||||
"acceptanceCriteria": []
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Save the returned `id` — you need it for the next two calls.
|
||||
|
||||
### Call 2: Transition to RESEARCH
|
||||
|
||||
```bash
|
||||
TICKET_ID="id-from-step-1"
|
||||
|
||||
curl -s -X POST "${API_URL}/v1/tickets/${TICKET_ID}/transition" \
|
||||
-H "Authorization: Bearer ${API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
|
||||
-d '{
|
||||
"to_state": "RESEARCH",
|
||||
"reason": "Intake complete, starting research"
|
||||
}'
|
||||
```
|
||||
|
||||
### Call 3: Queue Source Update
|
||||
|
||||
```bash
|
||||
curl -s -X POST "${API_URL}/v1/sync/queue" \
|
||||
-H "Authorization: Bearer ${API_KEY}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
|
||||
-d '{
|
||||
"ticket_id": "TICKET_ID_HERE",
|
||||
"action": "update_status",
|
||||
"payload": { "status": "In Progress" },
|
||||
"priority": "normal"
|
||||
}'
|
||||
```
|
||||
|
||||
> **Note:** The action MUST be `"update_status"` (not `"UPDATE_NOTION_STATUS"`). Valid actions: `update_status`, `update_pr_url`, `mark_done`.
|
||||
|
||||
### TypeScript Equivalent (if using pipeline client)
|
||||
|
||||
```typescript
|
||||
import { PipelineClient } from '@pipeline/client'
|
||||
|
||||
const client = new PipelineClient({
|
||||
apiUrl:
|
||||
process.env.PIPELINE_API_URL ||
|
||||
'https://api-gateway-856475788601.us-central1.run.app',
|
||||
agentId: process.env.AGENT_ID!
|
||||
})
|
||||
|
||||
const ticket = await client.createTicket({
|
||||
notion_page_id: pageId,
|
||||
title: ticketTitle,
|
||||
source: 'notion',
|
||||
metadata: { description, priority, labels, acceptanceCriteria }
|
||||
})
|
||||
|
||||
await client.transitionState(
|
||||
ticket.id,
|
||||
'RESEARCH',
|
||||
'Intake complete, starting research'
|
||||
)
|
||||
|
||||
await client.queueSync(ticket.id, 'update_status', { status: 'In Progress' })
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Detect Source Type
|
||||
|
||||
Parse the URL to determine source:
|
||||
|
||||
```javascript
|
||||
if (url.includes('notion.so')) {
|
||||
source = 'notion'
|
||||
// Load providers/notion.md
|
||||
} else if (url.match(/github\.com\/[^\/]+\/[^\/]+\/issues\/\d+/)) {
|
||||
source = 'github'
|
||||
// Load providers/github.md
|
||||
} else {
|
||||
// Error: Unsupported URL format
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Load Provider and Fetch Data
|
||||
|
||||
Read the appropriate provider file for source-specific instructions:
|
||||
|
||||
- **Notion**: `providers/notion.md` - Uses Notion MCP, handles Slack links
|
||||
- **GitHub**: `providers/github.md` - Uses `gh` CLI, handles Dosu comments
|
||||
|
||||
Follow the provider's instructions for:
|
||||
|
||||
- Fetching content
|
||||
- Extracting properties
|
||||
- **Updating the source** (Notion status → "In Progress", Assignee → pipeline owner)
|
||||
|
||||
### Step 3: Normalize to Common Schema
|
||||
|
||||
All providers must extract normalized ticket data following `schema.md`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "abc12345",
|
||||
"url": "https://...",
|
||||
"source": "notion | github",
|
||||
"title": "Ticket title",
|
||||
"description": "Full description",
|
||||
"status": "Not Started",
|
||||
"assignee": "username",
|
||||
"priority": "High",
|
||||
"area": "UI",
|
||||
"labels": ["bug", "frontend"],
|
||||
"acceptanceCriteria": ["Criterion 1", "Criterion 2"],
|
||||
"fetchedAt": "2024-01-15T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Register Ticket in Pipeline API (MANDATORY — DO NOT SKIP)
|
||||
|
||||
**Execute all three API calls from [Mandatory API Calls](#mandatory-api-calls-execute-all-three) above.**
|
||||
|
||||
This is not optional. This is not documentation. You MUST make these HTTP requests right now.
|
||||
|
||||
1. `createTicket()` → save the returned ticket ID
|
||||
2. `transitionState(id, 'RESEARCH')` → confirm state changed
|
||||
3. `queueSync(id, 'update_status', { status: 'In Progress' })` → confirm queued
|
||||
|
||||
**If any call fails**, retry once. If it still fails, report the error prominently — do NOT silently continue.
|
||||
|
||||
### Step 5: Run Verification Script
|
||||
|
||||
After making the API calls, run the verification script to confirm everything worked:
|
||||
|
||||
```bash
|
||||
bash scripts/verify-intake.sh TICKET_ID_OR_NOTION_PAGE_ID
|
||||
```
|
||||
|
||||
**If the script is not available locally**, verify manually via the public API:
|
||||
|
||||
```bash
|
||||
curl -s "${API_URL}/public/tickets/${TICKET_ID}" | jq '{id, state, title, notion_page_id}'
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "...",
|
||||
"state": "RESEARCH",
|
||||
"title": "...",
|
||||
"notion_page_id": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**If `state` is not `RESEARCH`, go back to Step 4 and complete the missing calls.**
|
||||
|
||||
### Step 6: Output Summary and Handoff
|
||||
|
||||
Print a clear summary:
|
||||
|
||||
```markdown
|
||||
## Ticket Intake Complete
|
||||
|
||||
**Source:** Notion | GitHub
|
||||
**Title:** [Ticket title]
|
||||
**ID:** abc12345
|
||||
**Status:** In Progress (queued)
|
||||
**Priority:** High
|
||||
**Area:** UI
|
||||
|
||||
### Description
|
||||
|
||||
[Brief description or first 200 chars]
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- [ ] Criterion 1
|
||||
- [ ] Criterion 2
|
||||
|
||||
### Links
|
||||
|
||||
- **Ticket:** [Original URL]
|
||||
- **Slack:** [Slack thread content fetched via slackdump] (Notion only)
|
||||
|
||||
### Pipeline
|
||||
|
||||
- **API Ticket ID:** abc12345
|
||||
- **State:** RESEARCH
|
||||
- **Verified:** ✅ (via verify-intake.sh or public API)
|
||||
```
|
||||
|
||||
**After printing the summary, immediately handoff** to continue the pipeline. Use the `handoff` tool with all necessary context (ticket ID, source, title, description, slack context if any):
|
||||
|
||||
> **Handoff goal:** "Continue pipeline for ticket {ID} ({title}). Ticket is in RESEARCH state. Load skill: `research-orchestrator` to begin research phase. Ticket data: source={source}, notion_page_id={pageId}, priority={priority}. {slack context summary if available}"
|
||||
|
||||
**Do NOT wait for human approval to proceed.** The intake phase is complete — handoff immediately.
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Unsupported URL
|
||||
|
||||
```
|
||||
❌ Unsupported ticket URL format.
|
||||
|
||||
Supported formats:
|
||||
- Notion: https://notion.so/... or https://www.notion.so/...
|
||||
- GitHub: https://github.com/{owner}/{repo}/issues/{number}
|
||||
|
||||
Received: [provided URL]
|
||||
```
|
||||
|
||||
### Provider-Specific Errors
|
||||
|
||||
See individual provider files for source-specific error handling:
|
||||
|
||||
- `providers/notion.md` - Authentication, page not found
|
||||
- `providers/github.md` - Auth, rate limits, issue not found
|
||||
|
||||
### Missing Properties
|
||||
|
||||
Continue with available data and note what's missing:
|
||||
|
||||
```
|
||||
⚠️ Some properties unavailable:
|
||||
- Priority: not found (using default: Medium)
|
||||
- Area: not found
|
||||
|
||||
Proceeding with available data...
|
||||
```
|
||||
|
||||
### API Call Failures
|
||||
|
||||
```
|
||||
❌ Pipeline API call failed: {method} {endpoint}
|
||||
Status: {status}
|
||||
Error: {message}
|
||||
|
||||
Retrying once...
|
||||
|
||||
❌ Retry also failed. INTAKE IS INCOMPLETE.
|
||||
The ticket was NOT registered in the pipeline.
|
||||
Downstream skills will not work until this is fixed.
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- This skill focuses ONLY on intake — it does not do research
|
||||
- Slack thread content is fetched automatically via the `slackdump` skill — no manual copy-paste needed
|
||||
- ALL API calls (createTicket, transitionState, queueSync) are MANDATORY — never skip them
|
||||
- The `queueSync` action must be `"update_status"`, NOT `"UPDATE_NOTION_STATUS"`
|
||||
- Pipeline state is tracked via the API, not local files
|
||||
- Working artifacts (research-report.md, plan.md) can be saved locally to `$PIPELINE_DIR/runs/{ticket-id}/`
|
||||
- The `source` field in the ticket determines which research strategies to use
|
||||
|
||||
## API Client Reference
|
||||
|
||||
### Available Methods
|
||||
|
||||
| Method | Description |
|
||||
| ----------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `createTicket({ notion_page_id, title, source, metadata })` | Create a new ticket in the API |
|
||||
| `getTicket(id)` | Retrieve a ticket by ID |
|
||||
| `findByNotionId(notionPageId)` | Look up a ticket by its Notion page ID |
|
||||
| `listTickets({ state, agent_id, limit, offset })` | List tickets with optional filters |
|
||||
| `transitionState(id, state, reason)` | Move ticket to a new state (e.g., `'RESEARCH'`) |
|
||||
| `setPRCreated(id, prUrl)` | Mark ticket as having a PR created |
|
||||
| `queueSync(id, action, payload)` | Queue a sync action (`update_status`, `update_pr_url`, `mark_done`) |
|
||||
| `registerBranch(id, branch, repo)` | Register working branch for automatic PR detection |
|
||||
|
||||
### Error Handling
|
||||
|
||||
```typescript
|
||||
import { PipelineClient, PipelineAPIError } from '@pipeline/client';
|
||||
|
||||
try {
|
||||
await client.createTicket({ ... });
|
||||
} catch (error) {
|
||||
if (error instanceof PipelineAPIError) {
|
||||
console.error(`API Error ${error.status}: ${error.message}`);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
```
|
||||
194
.claude/skills/ticket-intake/providers/github.md
Normal file
194
.claude/skills/ticket-intake/providers/github.md
Normal file
@@ -0,0 +1,194 @@
|
||||
# GitHub Provider - Ticket Intake
|
||||
|
||||
Provider-specific logic for ingesting tickets from GitHub Issues.
|
||||
|
||||
## URL Pattern
|
||||
|
||||
```
|
||||
https://github.com/{owner}/{repo}/issues/{number}
|
||||
https://www.github.com/{owner}/{repo}/issues/{number}
|
||||
```
|
||||
|
||||
Extract: `owner`, `repo`, `issue_number` from URL.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
- Access to the repository
|
||||
|
||||
## Fetch Issue Content
|
||||
|
||||
Use `gh` CLI to fetch issue details:
|
||||
|
||||
```bash
|
||||
# Get issue details in JSON
|
||||
gh issue view {number} --repo {owner}/{repo} --json title,body,state,labels,assignees,milestone,author,createdAt,comments,linkedPRs
|
||||
|
||||
# Get comments separately if needed
|
||||
gh issue view {number} --repo {owner}/{repo} --comments
|
||||
```
|
||||
|
||||
## Extract Ticket Data
|
||||
|
||||
Map GitHub issue fields to normalized ticket data (stored via API):
|
||||
|
||||
| GitHub Field | ticket.json Field | Notes |
|
||||
| ------------ | ----------------- | -------------------------- |
|
||||
| title | title | Direct mapping |
|
||||
| body | description | Issue body/description |
|
||||
| state | status | Map: open → "Not Started" |
|
||||
| labels | labels | Array of label names |
|
||||
| assignees | assignee | First assignee login |
|
||||
| author | author | Issue author login |
|
||||
| milestone | milestone | Milestone title if present |
|
||||
| comments | comments | Array of comment objects |
|
||||
| linkedPRs | linkedPRs | PRs linked to this issue |
|
||||
|
||||
### Priority Mapping
|
||||
|
||||
Infer priority from labels:
|
||||
|
||||
- `priority:critical`, `P0` → "Critical"
|
||||
- `priority:high`, `P1` → "High"
|
||||
- `priority:medium`, `P2` → "Medium"
|
||||
- `priority:low`, `P3` → "Low"
|
||||
- No priority label → "Medium" (default)
|
||||
|
||||
### Area Mapping
|
||||
|
||||
Infer area from labels:
|
||||
|
||||
- `area:ui`, `frontend`, `component:*` → "UI"
|
||||
- `area:api`, `backend` → "API"
|
||||
- `area:docs`, `documentation` → "Docs"
|
||||
- `bug`, `fix` → "Bug"
|
||||
- `enhancement`, `feature` → "Feature"
|
||||
|
||||
## Update Source
|
||||
|
||||
**For GitHub issues, update is optional but recommended.**
|
||||
|
||||
Add a comment to indicate work has started:
|
||||
|
||||
```bash
|
||||
gh issue comment {number} --repo {owner}/{repo} --body "🤖 Pipeline started processing this issue."
|
||||
```
|
||||
|
||||
Optionally assign to self:
|
||||
|
||||
```bash
|
||||
gh issue edit {number} --repo {owner}/{repo} --add-assignee @me
|
||||
```
|
||||
|
||||
Log any updates via the Pipeline API:
|
||||
|
||||
```typescript
|
||||
await client.updateTicket(ticketId, {
|
||||
metadata: {
|
||||
...ticket.metadata,
|
||||
githubWrites: [
|
||||
...(ticket.metadata?.githubWrites || []),
|
||||
{
|
||||
action: 'comment',
|
||||
issueNumber: 123,
|
||||
at: new Date().toISOString(),
|
||||
skill: 'ticket-intake',
|
||||
success: true
|
||||
}
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
## GitHub-Specific Ticket Fields
|
||||
|
||||
Store via API using `client.createTicket()`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "github",
|
||||
"githubOwner": "Comfy-Org",
|
||||
"githubRepo": "ComfyUI_frontend",
|
||||
"githubIssueNumber": 123,
|
||||
"githubIssueUrl": "https://github.com/Comfy-Org/ComfyUI_frontend/issues/123",
|
||||
"labels": ["bug", "area:ui", "priority:high"],
|
||||
"linkedPRs": [456, 789],
|
||||
"dosuComment": "..." // Extracted Dosu bot analysis if present
|
||||
}
|
||||
```
|
||||
|
||||
## Dosu Bot Detection
|
||||
|
||||
Many repositories use Dosu bot for automated issue analysis. Check comments for Dosu:
|
||||
|
||||
```bash
|
||||
gh issue view {number} --repo {owner}/{repo} --comments | grep -A 100 "dosu"
|
||||
```
|
||||
|
||||
Look for comments from:
|
||||
|
||||
- `dosu[bot]`
|
||||
- `dosu-bot`
|
||||
|
||||
Extract Dosu analysis which typically includes:
|
||||
|
||||
- Root cause analysis
|
||||
- Suggested files to modify
|
||||
- Related issues/PRs
|
||||
- Potential solutions
|
||||
|
||||
Store in ticket data via API:
|
||||
|
||||
```json
|
||||
{
|
||||
"dosuComment": {
|
||||
"found": true,
|
||||
"analysis": "...",
|
||||
"suggestedFiles": ["src/file1.ts", "src/file2.ts"],
|
||||
"relatedIssues": [100, 101]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Extract Linked Issues/PRs
|
||||
|
||||
Parse issue body and comments for references:
|
||||
|
||||
- `#123` → Issue or PR reference
|
||||
- `fixes #123`, `closes #123` → Linked issue
|
||||
- `https://github.com/.../issues/123` → Full URL reference
|
||||
|
||||
Store in ticket data via API for research phase:
|
||||
|
||||
```json
|
||||
{
|
||||
"referencedIssues": [100, 101, 102],
|
||||
"referencedPRs": [200, 201]
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Authentication Error
|
||||
|
||||
```
|
||||
⚠️ GitHub CLI not authenticated.
|
||||
Run: gh auth login
|
||||
```
|
||||
|
||||
### Issue Not Found
|
||||
|
||||
```
|
||||
❌ GitHub issue not found or inaccessible.
|
||||
- Check the URL is correct
|
||||
- Ensure you have access to this repository
|
||||
- Run: gh auth status
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
```
|
||||
⚠️ GitHub API rate limited.
|
||||
Wait a few minutes and try again.
|
||||
Check status: gh api rate_limit
|
||||
```
|
||||
202
.claude/skills/ticket-intake/providers/notion.md
Normal file
202
.claude/skills/ticket-intake/providers/notion.md
Normal file
@@ -0,0 +1,202 @@
|
||||
# Notion Provider - Ticket Intake
|
||||
|
||||
Provider-specific logic for ingesting tickets from Notion.
|
||||
|
||||
## URL Pattern
|
||||
|
||||
```
|
||||
https://www.notion.so/workspace/Page-Title-abc123def456...
|
||||
https://notion.so/Page-Title-abc123def456...
|
||||
https://www.notion.so/abc123def456...
|
||||
```
|
||||
|
||||
Page ID is the 32-character hex string (with or without hyphens).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Notion MCP connected and authenticated
|
||||
- If not setup: `claude mcp add --transport http notion https://mcp.notion.com/mcp`
|
||||
- Authenticate via `/mcp` command if prompted
|
||||
|
||||
## Fetch Ticket Content
|
||||
|
||||
Use `Notion:notion-fetch` with the page URL or ID:
|
||||
|
||||
```
|
||||
Fetch the full page content including all properties
|
||||
```
|
||||
|
||||
## Extract Ticket Data
|
||||
|
||||
Extract these properties (names may vary):
|
||||
|
||||
| Property | Expected Name | Type |
|
||||
| ------------- | ------------------------- | ------------ |
|
||||
| Title | Name / Title | Title |
|
||||
| Status | Status | Select |
|
||||
| Assignee | Assignee / Assigned To | Person |
|
||||
| Description | - | Page content |
|
||||
| Slack Link | Slack Link / Slack Thread | URL |
|
||||
| GitHub PR | GitHub PR / PR Link | URL |
|
||||
| Priority | Priority | Select |
|
||||
| Area | Area / Category | Select |
|
||||
| Related Tasks | Related Tasks | Relation |
|
||||
|
||||
**If properties are missing**: Note what's unavailable and continue with available data.
|
||||
|
||||
## Update Source (REQUIRED)
|
||||
|
||||
**⚠️ DO NOT SKIP THIS STEP. This is a required action, not optional.**
|
||||
|
||||
**⚠️ Notion Write Safety rules apply (see `$PIPELINE_DIR/docs/notion-write-safety.md` for full reference):**
|
||||
|
||||
- **Whitelist**: Only `Status`, `GitHub PR`, and `Assignee` fields may be written
|
||||
- **Valid transitions**: Not Started → In Progress, In Progress → In Review, In Review → Done
|
||||
- **Logging**: Every write attempt MUST be logged with timestamp, field, value, previous value, skill name, and success status
|
||||
|
||||
Use `Notion:notion-update-page` to update the ticket:
|
||||
|
||||
1. **Status**: Set to "In Progress" (only valid from "Not Started")
|
||||
2. **Assignee**: Assign to pipeline owner (Notion ID: `175d872b-594c-81d4-ba5a-0002911c5966`)
|
||||
|
||||
```json
|
||||
{
|
||||
"page_id": "{page_id_from_ticket}",
|
||||
"command": "update_properties",
|
||||
"properties": {
|
||||
"Status": "In Progress",
|
||||
"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**After the update succeeds**, log the write via the Pipeline API:
|
||||
|
||||
```typescript
|
||||
await client.updateTicket(ticketId, {
|
||||
metadata: {
|
||||
...ticket.metadata,
|
||||
notionWrites: [
|
||||
...(ticket.metadata?.notionWrites || []),
|
||||
{
|
||||
field: 'Status',
|
||||
value: 'In Progress',
|
||||
previousValue: 'Not Started',
|
||||
at: new Date().toISOString(),
|
||||
skill: 'ticket-intake',
|
||||
success: true
|
||||
}
|
||||
]
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
If update fails, log with `success: false` and continue.
|
||||
|
||||
## Notion-Specific Ticket Fields
|
||||
|
||||
Store via API using `client.createTicket()`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "notion",
|
||||
"notionPageId": "abc123def456...",
|
||||
"slackLink": "https://slack.com/...",
|
||||
"relatedTasks": ["page-id-1", "page-id-2"]
|
||||
}
|
||||
```
|
||||
|
||||
## Slack Thread Handling
|
||||
|
||||
If a Slack link exists, use the `slackdump` skill to fetch the thread content programmatically.
|
||||
|
||||
### Slack URL Conversion
|
||||
|
||||
Notion stores Slack links in `slackMessage://` format:
|
||||
|
||||
```
|
||||
slackMessage://comfy-organization.slack.com/CHANNEL_ID/THREAD_TS/MESSAGE_TS
|
||||
```
|
||||
|
||||
Convert to browser-clickable format:
|
||||
|
||||
```
|
||||
https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS_NO_DOT
|
||||
```
|
||||
|
||||
**Example:**
|
||||
|
||||
- Input: `slackMessage://comfy-organization.slack.com/C075ANWQ8KS/1766022478.450909/1764772881.854829`
|
||||
- Output: `https://comfy-organization.slack.com/archives/C075ANWQ8KS/p1764772881854829`
|
||||
|
||||
(Remove the dot from the last timestamp and prefix with `p`)
|
||||
|
||||
### Fetching Thread Content
|
||||
|
||||
Load the `slackdump` skill and use the **export-thread** workflow:
|
||||
|
||||
```bash
|
||||
# Export thread by URL
|
||||
slackdump dump "https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS"
|
||||
|
||||
# Or by colon notation (channel_id:thread_ts)
|
||||
slackdump dump CHANNEL_ID:THREAD_TS
|
||||
```
|
||||
|
||||
Save the thread content to `$RUN_DIR/slack-context.md` and include it in the ticket metadata.
|
||||
|
||||
> **No manual action required.** The slackdump CLI handles authentication via stored credentials at `~/.cache/slackdump/comfy-organization.bin`.
|
||||
|
||||
## Database Reference: Comfy Tasks
|
||||
|
||||
The "Comfy Tasks" database has these properties (verify via `notion-search`):
|
||||
|
||||
- **Status values**: Not Started, In Progress, In Review, Done
|
||||
- **Team assignment**: "Frontend Team" for unassigned tickets
|
||||
- **Filtering note**: Team filtering in Notion may have quirks - handle gracefully
|
||||
|
||||
### Pipeline Owner Details
|
||||
|
||||
When assigning tickets, use these identifiers:
|
||||
|
||||
| Platform | Identifier |
|
||||
| --------------- | -------------------------------------- |
|
||||
| Notion User ID | `175d872b-594c-81d4-ba5a-0002911c5966` |
|
||||
| Notion Name | Christian Byrne |
|
||||
| Notion Email | cbyrne@comfy.org |
|
||||
| Slack User ID | U087MJCDHHC |
|
||||
| GitHub Username | christian-byrne |
|
||||
|
||||
**To update Assignee**, use the Notion User ID (not name):
|
||||
|
||||
```
|
||||
properties: {"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"}
|
||||
```
|
||||
|
||||
### Finding Active Tickets
|
||||
|
||||
To list your active tickets:
|
||||
|
||||
```
|
||||
Use Notion:notion-search for "Comfy Tasks"
|
||||
Filter by Assignee = current user OR Team = "Frontend Team"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Authentication Error
|
||||
|
||||
```
|
||||
⚠️ Notion authentication required.
|
||||
Run: claude mcp add --transport http notion https://mcp.notion.com/mcp
|
||||
Then authenticate via /mcp command.
|
||||
```
|
||||
|
||||
### Page Not Found
|
||||
|
||||
```
|
||||
❌ Notion page not found or inaccessible.
|
||||
- Check the URL is correct
|
||||
- Ensure you have access to this page
|
||||
- Try re-authenticating via /mcp
|
||||
```
|
||||
81
.claude/skills/ticket-intake/schema.md
Normal file
81
.claude/skills/ticket-intake/schema.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Ticket Schema
|
||||
|
||||
Common schema for normalized ticket data across all sources. This data is stored and retrieved via the Pipeline API, not local files.
|
||||
|
||||
## Ticket Data Schema
|
||||
|
||||
```json
|
||||
{
|
||||
// Required fields (all sources)
|
||||
"id": "string", // Unique identifier (short form)
|
||||
"url": "string", // Original URL
|
||||
"source": "notion | github", // Source type
|
||||
"title": "string", // Ticket title
|
||||
"description": "string", // Full description/body
|
||||
"fetchedAt": "ISO8601", // When ticket was fetched
|
||||
|
||||
// Common optional fields
|
||||
"status": "string", // Current status
|
||||
"assignee": "string", // Assigned user
|
||||
"priority": "string", // Priority level
|
||||
"area": "string", // Category/area
|
||||
"labels": ["string"], // Tags/labels
|
||||
"acceptanceCriteria": ["string"] // List of AC items
|
||||
|
||||
// Source-specific fields (see providers)
|
||||
// Notion: notionPageId, slackLink, relatedTasks, notionWrites
|
||||
// GitHub: githubOwner, githubRepo, githubIssueNumber, linkedPRs, dosuComment, referencedIssues
|
||||
}
|
||||
```
|
||||
|
||||
## Ticket State Schema (via API)
|
||||
|
||||
State is managed via the Pipeline API using `client.transitionState()`:
|
||||
|
||||
```json
|
||||
{
|
||||
"ticketId": "string",
|
||||
"state": "intake | research | planning | implementation | pr_created | done | failed",
|
||||
"stateChangedAt": "ISO8601",
|
||||
|
||||
// Timestamps tracked by API
|
||||
"createdAt": "ISO8601",
|
||||
"updatedAt": "ISO8601"
|
||||
}
|
||||
```
|
||||
|
||||
## Priority Normalization
|
||||
|
||||
All sources should normalize to these values:
|
||||
|
||||
| Normalized | Description |
|
||||
| ---------- | ------------------------- |
|
||||
| Critical | Production down, security |
|
||||
| High | Blocking work, urgent |
|
||||
| Medium | Normal priority (default) |
|
||||
| Low | Nice to have, backlog |
|
||||
|
||||
## Status Normalization
|
||||
|
||||
Pipeline tracks these statuses internally:
|
||||
|
||||
| Status | Description |
|
||||
| -------------- | ---------------------------- |
|
||||
| research | Gathering context |
|
||||
| planning | Creating implementation plan |
|
||||
| implementation | Writing code |
|
||||
| review | Code review in progress |
|
||||
| qa | Quality assurance |
|
||||
| done | PR merged or completed |
|
||||
|
||||
## ID Generation
|
||||
|
||||
IDs are generated by the API when creating tickets. For reference:
|
||||
|
||||
- **Notion**: First 8 characters of page ID
|
||||
- **GitHub**: `gh-{owner}-{repo}-{issue_number}` (sanitized)
|
||||
|
||||
Examples:
|
||||
|
||||
- Notion: `abc12345`
|
||||
- GitHub: `gh-comfy-org-frontend-123`
|
||||
@@ -46,3 +46,11 @@ ALGOLIA_API_KEY=684d998c36b67a9a9fce8fc2d8860579
|
||||
# SENTRY_ORG=comfy-org
|
||||
# SENTRY_PROJECT=cloud-frontend-staging
|
||||
# SENTRY_PROJECT_PROD= # prod project slug for sourcemap uploads
|
||||
|
||||
# ── QA Skill (scripts/qa-record.ts) ──
|
||||
# Required for automated bug reproduction via `pnpm exec tsx scripts/qa-record.ts`
|
||||
# GEMINI_API_KEY is required — used for PR analysis, video review, and TTS narration
|
||||
GEMINI_API_KEY=
|
||||
# ANTHROPIC_API_KEY is optional locally — Claude Agent SDK auto-detects Claude Code session
|
||||
# Required in CI (set as GitHub Actions secret)
|
||||
# ANTHROPIC_API_KEY=
|
||||
|
||||
31
.github/actions/lint-format-verify/action.yml
vendored
31
.github/actions/lint-format-verify/action.yml
vendored
@@ -1,31 +0,0 @@
|
||||
name: 'Lint and format verify'
|
||||
description: >
|
||||
Runs the lint/format/knip verification suite plus a conditional
|
||||
browser-tests typecheck. Shared by ci-lint-format.yaml (PR) and
|
||||
ci-lint-format-queue.yaml (merge queue) so both paths run the exact
|
||||
same checks. The caller is responsible for checkout and frontend setup
|
||||
before invoking this action.
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Detect browser_tests changes
|
||||
id: changed-paths
|
||||
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
|
||||
with:
|
||||
filters: |
|
||||
browser_tests:
|
||||
- 'browser_tests/**'
|
||||
|
||||
- name: Verify lint and format
|
||||
shell: bash
|
||||
run: |
|
||||
pnpm lint
|
||||
pnpm stylelint
|
||||
pnpm format:check
|
||||
pnpm knip
|
||||
|
||||
- name: Typecheck browser tests
|
||||
if: steps.changed-paths.outputs.browser_tests == 'true'
|
||||
shell: bash
|
||||
run: pnpm typecheck:browser
|
||||
29
.github/workflows/ci-lint-format-queue.yaml
vendored
29
.github/workflows/ci-lint-format-queue.yaml
vendored
@@ -1,29 +0,0 @@
|
||||
# Description: Lint and format verification for GitHub merge queue runs.
|
||||
# Paired with ci-lint-format.yaml — workflow name and job name must match
|
||||
# so branch protection resolves a single required check in both the
|
||||
# pull_request and merge_group contexts. This file runs verify-only steps
|
||||
# with a read-only token; auto-fix and PR comments live in the PR workflow.
|
||||
name: 'CI: Lint Format'
|
||||
|
||||
on:
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
lint-and-format:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout merge group ref
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Setup frontend
|
||||
uses: ./.github/actions/setup-frontend
|
||||
|
||||
- name: Verify lint and format
|
||||
uses: ./.github/actions/lint-format-verify
|
||||
25
.github/workflows/ci-lint-format.yaml
vendored
25
.github/workflows/ci-lint-format.yaml
vendored
@@ -1,7 +1,4 @@
|
||||
# Description: Linting and code formatting validation for pull requests.
|
||||
# Paired with ci-lint-format-queue.yaml - workflow name and job name must
|
||||
# match so branch protection resolves a single required check in both the
|
||||
# pull_request and merge_group contexts.
|
||||
# Description: Linting and code formatting validation for pull requests
|
||||
name: 'CI: Lint Format'
|
||||
|
||||
on:
|
||||
@@ -29,6 +26,14 @@ jobs:
|
||||
- name: Setup frontend
|
||||
uses: ./.github/actions/setup-frontend
|
||||
|
||||
- name: Detect browser_tests changes
|
||||
id: changed-paths
|
||||
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
|
||||
with:
|
||||
filters: |
|
||||
browser_tests:
|
||||
- 'browser_tests/**'
|
||||
|
||||
- name: Run ESLint with auto-fix
|
||||
run: pnpm lint:fix
|
||||
|
||||
@@ -72,8 +77,16 @@ jobs:
|
||||
echo "See CONTRIBUTING.md for more details."
|
||||
exit 1
|
||||
|
||||
- name: Verify lint and format
|
||||
uses: ./.github/actions/lint-format-verify
|
||||
- name: Final validation
|
||||
run: |
|
||||
pnpm lint
|
||||
pnpm stylelint
|
||||
pnpm format:check
|
||||
pnpm knip
|
||||
|
||||
- name: Typecheck browser tests
|
||||
if: steps.changed-paths.outputs.browser_tests == 'true'
|
||||
run: pnpm typecheck:browser
|
||||
|
||||
- name: Comment on PR about auto-fix
|
||||
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name == github.repository
|
||||
|
||||
1
.github/workflows/ci-tests-e2e.yaml
vendored
1
.github/workflows/ci-tests-e2e.yaml
vendored
@@ -8,7 +8,6 @@ on:
|
||||
pull_request:
|
||||
branches-ignore: [wip/*, draft/*, temp/*]
|
||||
paths-ignore: ['**/*.md']
|
||||
merge_group:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
|
||||
1
.github/workflows/ci-tests-unit.yaml
vendored
1
.github/workflows/ci-tests-unit.yaml
vendored
@@ -8,7 +8,6 @@ on:
|
||||
pull_request:
|
||||
branches-ignore: [wip/*, draft/*, temp/*]
|
||||
paths-ignore: ['**/*.md']
|
||||
merge_group:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
|
||||
1094
.github/workflows/pr-qa.yaml
vendored
Normal file
1094
.github/workflows/pr-qa.yaml
vendored
Normal file
File diff suppressed because it is too large
Load Diff
3
.gitignore
vendored
3
.gitignore
vendored
@@ -67,6 +67,9 @@ dist.zip
|
||||
/temp/
|
||||
/tmp/
|
||||
|
||||
# QA local output
|
||||
/.comfy-qa/
|
||||
|
||||
# Generated JSON Schemas
|
||||
/schemas/
|
||||
|
||||
|
||||
@@ -84,7 +84,6 @@
|
||||
"typescript/no-unsafe-declaration-merging": "off",
|
||||
"typescript/no-unused-vars": "off",
|
||||
"unicorn/no-empty-file": "off",
|
||||
"vitest/require-mock-type-parameters": "off",
|
||||
"unicorn/no-new-array": "off",
|
||||
"unicorn/no-single-promise-in-promise-methods": "off",
|
||||
"unicorn/no-useless-fallback-in-spread": "off",
|
||||
@@ -117,60 +116,13 @@
|
||||
},
|
||||
{
|
||||
"files": ["browser_tests/**/*.ts"],
|
||||
"jsPlugins": ["eslint-plugin-playwright"],
|
||||
"rules": {
|
||||
"typescript/no-explicit-any": "error",
|
||||
"no-async-promise-executor": "error",
|
||||
"no-control-regex": "error",
|
||||
"no-useless-rename": "error",
|
||||
"no-unused-private-class-members": "error",
|
||||
"unicorn/no-empty-file": "error",
|
||||
"playwright/consistent-spacing-between-blocks": "error",
|
||||
"playwright/expect-expect": [
|
||||
"error",
|
||||
{
|
||||
"assertFunctionNames": [
|
||||
"recordMeasurement",
|
||||
"logMeasurement",
|
||||
"builderSaveAs"
|
||||
],
|
||||
"assertFunctionPatterns": [
|
||||
"^expect",
|
||||
"^assert",
|
||||
"^verify",
|
||||
"^searchAndExpect",
|
||||
"waitForOpen",
|
||||
"waitForClosed",
|
||||
"waitForRequest"
|
||||
]
|
||||
}
|
||||
],
|
||||
"playwright/max-nested-describe": "error",
|
||||
"playwright/no-duplicate-hooks": "error",
|
||||
"playwright/no-element-handle": "error",
|
||||
"playwright/no-eval": "error",
|
||||
"playwright/no-focused-test": "error",
|
||||
"playwright/no-force-option": "error",
|
||||
"playwright/no-networkidle": "error",
|
||||
"playwright/no-page-pause": "error",
|
||||
"playwright/no-skipped-test": "error",
|
||||
"playwright/no-unsafe-references": "error",
|
||||
"playwright/no-unused-locators": "error",
|
||||
"playwright/no-useless-await": "error",
|
||||
"playwright/no-useless-not": "error",
|
||||
"playwright/no-wait-for-navigation": "error",
|
||||
"playwright/no-wait-for-selector": "error",
|
||||
"playwright/no-wait-for-timeout": "error",
|
||||
"playwright/prefer-hooks-on-top": "error",
|
||||
"playwright/prefer-locator": "error",
|
||||
"playwright/prefer-to-have-count": "error",
|
||||
"playwright/prefer-to-have-length": "error",
|
||||
"playwright/prefer-web-first-assertions": "error",
|
||||
"playwright/prefer-native-locators": "error",
|
||||
"playwright/require-to-pass-timeout": "error",
|
||||
"playwright/valid-expect": "error",
|
||||
"playwright/valid-expect-in-promise": "error",
|
||||
"playwright/valid-title": "error"
|
||||
"unicorn/no-empty-file": "error"
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
@@ -318,9 +318,6 @@ When referencing Comfy-Org repos:
|
||||
- Find existing `!important` classes that are interfering with the styling and propose corrections of those instead.
|
||||
- NEVER use arbitrary percentage values like `w-[80%]` when a Tailwind fraction utility exists
|
||||
- Use `w-4/5` instead of `w-[80%]`, `w-1/2` instead of `w-[50%]`, etc.
|
||||
- NEVER use font-size classes (`text-xs`, `text-sm`, etc.) to size `icon-[...]` (iconify) icons
|
||||
- Iconify icons size via `width`/`height: 1.2em`, so font-size produces unpredictable results
|
||||
- Use `size-*` classes for explicit sizing, or set font-size on the **parent** container and let `1.2em` scale naturally
|
||||
|
||||
## Agent-only rules
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ const features = computed(() => [
|
||||
<div class="mx-auto max-w-3xl px-6 text-center">
|
||||
<!-- Badge -->
|
||||
<span
|
||||
class="inline-block rounded-full bg-brand-yellow/10 px-4 py-1.5 text-xs tracking-widest text-brand-yellow uppercase"
|
||||
class="inline-block rounded-full bg-brand-yellow/10 px-4 py-1.5 text-xs uppercase tracking-widest text-brand-yellow"
|
||||
>
|
||||
{{ t('academy.badge', locale) }}
|
||||
</span>
|
||||
|
||||
@@ -40,7 +40,7 @@ const steps = computed(() => [
|
||||
<!-- Connecting line between steps (desktop only) -->
|
||||
<div
|
||||
v-if="index < steps.length - 1"
|
||||
class="absolute top-8 right-0 hidden w-full translate-x-1/2 border-t border-brand-yellow/20 md:block"
|
||||
class="absolute right-0 top-8 hidden w-full translate-x-1/2 border-t border-brand-yellow/20 md:block"
|
||||
/>
|
||||
|
||||
<div class="relative">
|
||||
|
||||
@@ -31,11 +31,11 @@ const ctaButtons = computed(() => [
|
||||
<div class="flex w-full items-center justify-center md:w-[55%]">
|
||||
<div class="relative -ml-12 -rotate-15 md:-ml-24" aria-hidden="true">
|
||||
<div
|
||||
class="size-64 rounded-full border-40 border-brand-yellow md:h-112 md:w-md md:border-64 lg:h-144 lg:w-xl lg:border-80"
|
||||
class="h-64 w-64 rounded-full border-[40px] border-brand-yellow md:h-[28rem] md:w-[28rem] md:border-[64px] lg:h-[36rem] lg:w-[36rem] lg:border-[80px]"
|
||||
>
|
||||
<!-- Gap on the right side to form "C" shape -->
|
||||
<div
|
||||
class="absolute top-1/2 right-0 h-32 w-24 translate-x-1/2 -translate-y-1/2 bg-black md:h-48 md:w-36 lg:h-64 lg:w-48"
|
||||
class="absolute right-0 top-1/2 h-32 w-24 -translate-y-1/2 translate-x-1/2 bg-black md:h-48 md:w-36 lg:h-64 lg:w-48"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
@@ -44,7 +44,7 @@ const ctaButtons = computed(() => [
|
||||
<!-- Right: Text content -->
|
||||
<div class="flex w-full flex-col items-start md:w-[45%]">
|
||||
<h1
|
||||
class="text-5xl/tight font-bold tracking-tight text-white md:text-6xl lg:text-7xl"
|
||||
class="text-5xl font-bold leading-tight tracking-tight text-white md:text-6xl lg:text-7xl"
|
||||
>
|
||||
{{ t('hero.headline', locale) }}
|
||||
</h1>
|
||||
|
||||
@@ -17,7 +17,7 @@ const { locale = 'en' } = defineProps<{ locale?: Locale }>()
|
||||
{{ t('manifesto.heading', locale) }}
|
||||
</h2>
|
||||
|
||||
<p class="mx-auto mt-6 max-w-2xl text-lg/relaxed text-smoke-700">
|
||||
<p class="mx-auto mt-6 max-w-2xl text-lg leading-relaxed text-smoke-700">
|
||||
{{ t('manifesto.body', locale) }}
|
||||
</p>
|
||||
|
||||
|
||||
@@ -33,11 +33,11 @@ const features = computed(() => [
|
||||
<div class="flex flex-col items-center gap-4">
|
||||
<!-- Play button triangle -->
|
||||
<div
|
||||
class="flex size-16 items-center justify-center rounded-full border-2 border-white/20"
|
||||
class="flex h-16 w-16 items-center justify-center rounded-full border-2 border-white/20"
|
||||
aria-hidden="true"
|
||||
>
|
||||
<div
|
||||
class="ml-1 size-0 border-y-8 border-l-14 border-y-transparent border-l-white"
|
||||
class="ml-1 h-0 w-0 border-y-8 border-l-[14px] border-y-transparent border-l-white"
|
||||
/>
|
||||
</div>
|
||||
<p class="text-sm text-smoke-700">
|
||||
@@ -54,7 +54,7 @@ const features = computed(() => [
|
||||
class="flex items-center gap-2"
|
||||
>
|
||||
<span
|
||||
class="size-2 rounded-full bg-brand-yellow"
|
||||
class="h-2 w-2 rounded-full bg-brand-yellow"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<span class="text-sm text-smoke-700">{{ feature }}</span>
|
||||
|
||||
@@ -32,7 +32,7 @@ const metrics = computed(() => [
|
||||
<div class="mx-auto max-w-7xl px-6">
|
||||
<!-- Heading -->
|
||||
<p
|
||||
class="text-center text-xs font-medium tracking-widest text-smoke-700 uppercase"
|
||||
class="text-center text-xs font-medium uppercase tracking-widest text-smoke-700"
|
||||
>
|
||||
{{ t('social.heading', locale) }}
|
||||
</p>
|
||||
|
||||
@@ -90,7 +90,7 @@ const filteredTestimonials = computed(() => {
|
||||
:key="testimonial.name"
|
||||
class="rounded-xl border border-white/10 bg-charcoal-600 p-6"
|
||||
>
|
||||
<blockquote class="text-base text-white italic">
|
||||
<blockquote class="text-base italic text-white">
|
||||
“{{ testimonial.quote }}”
|
||||
</blockquote>
|
||||
|
||||
|
||||
@@ -24,12 +24,12 @@ const activeCategory = ref(0)
|
||||
<!-- Left placeholder image (desktop only) -->
|
||||
<div class="hidden flex-1 lg:block">
|
||||
<div
|
||||
class="aspect-2/3 rounded-full border border-white/10 bg-charcoal-600"
|
||||
class="aspect-[2/3] rounded-full border border-white/10 bg-charcoal-600"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<!-- Center content -->
|
||||
<div class="flex flex-col items-center text-center lg:flex-2">
|
||||
<div class="flex flex-col items-center text-center lg:flex-[2]">
|
||||
<h2 class="text-3xl font-bold text-white">
|
||||
{{ t('useCase.heading', locale) }}
|
||||
</h2>
|
||||
@@ -70,7 +70,7 @@ const activeCategory = ref(0)
|
||||
<!-- Right placeholder image (desktop only) -->
|
||||
<div class="hidden flex-1 lg:block">
|
||||
<div
|
||||
class="aspect-2/3 rounded-3xl border border-white/10 bg-charcoal-600"
|
||||
class="aspect-[2/3] rounded-3xl border border-white/10 bg-charcoal-600"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -53,7 +53,7 @@ const pillars = computed(() => [
|
||||
class="rounded-xl border border-white/10 bg-charcoal-600 p-6 transition-colors hover:border-brand-yellow"
|
||||
>
|
||||
<div
|
||||
class="flex size-12 items-center justify-center rounded-full bg-brand-yellow text-xl"
|
||||
class="flex h-12 w-12 items-center justify-center rounded-full bg-brand-yellow text-xl"
|
||||
>
|
||||
{{ pillar.icon }}
|
||||
</div>
|
||||
|
||||
@@ -40,39 +40,6 @@ browser_tests/
|
||||
- **`fixtures/helpers/`** — Focused helper classes. Domain-specific actions that coordinate multiple page objects (e.g. canvas operations, workflow loading).
|
||||
- **`fixtures/utils/`** — Pure utility functions. No `Page` dependency; stateless helpers that can be used anywhere.
|
||||
|
||||
## Page Object Locator Style
|
||||
|
||||
Define UI element locators as `public readonly` properties assigned in the constructor — not as getter methods. Getters that simply return a locator add unnecessary indirection and hide the object shape from IDE auto-complete.
|
||||
|
||||
```typescript
|
||||
// ✅ Correct — public readonly, assigned in constructor
|
||||
export class MyDialog extends BaseDialog {
|
||||
public readonly submitButton: Locator
|
||||
public readonly cancelButton: Locator
|
||||
|
||||
constructor(page: Page) {
|
||||
super(page)
|
||||
this.submitButton = this.root.getByRole('button', { name: 'Submit' })
|
||||
this.cancelButton = this.root.getByRole('button', { name: 'Cancel' })
|
||||
}
|
||||
}
|
||||
|
||||
// ❌ Avoid — getter-based locators
|
||||
export class MyDialog extends BaseDialog {
|
||||
get submitButton() {
|
||||
return this.root.getByRole('button', { name: 'Submit' })
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Keep as getters only when:**
|
||||
|
||||
- Lazy initialization is needed (`this._tab ??= new Tab(this.page)`)
|
||||
- The value is computed from runtime state (e.g. `get id() { return this.userIds[index] }`)
|
||||
- It's a private convenience accessor (e.g. `private get page() { return this.comfyPage.page }`)
|
||||
|
||||
When a class has cached locator properties, prefer reusing them in methods rather than rebuilding locators from scratch.
|
||||
|
||||
## Polling Assertions
|
||||
|
||||
Prefer `expect.poll()` over `expect(async () => { ... }).toPass()` when the block contains a single async call with a single assertion. `expect.poll()` is more readable and gives better error messages (shows actual vs expected on failure).
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
{
|
||||
"last_node_id": 1,
|
||||
"last_link_id": 0,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 1,
|
||||
"type": "ImageCropV2",
|
||||
"pos": [50, 50],
|
||||
"size": [400, 500],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "image",
|
||||
"type": "IMAGE",
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ImageCropV2"
|
||||
},
|
||||
"widgets_values": [
|
||||
{
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"width": 512,
|
||||
"height": 512
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"links": [],
|
||||
"groups": [],
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"offset": [0, 0],
|
||||
"scale": 1
|
||||
}
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
{
|
||||
"last_node_id": 3,
|
||||
"last_link_id": 2,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 1,
|
||||
"type": "LoadImage",
|
||||
"pos": [50, 50],
|
||||
"size": [315, 314],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [1]
|
||||
},
|
||||
{
|
||||
"name": "MASK",
|
||||
"type": "MASK",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "LoadImage"
|
||||
},
|
||||
"widgets_values": ["example.png", "image"]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "ImageCropV2",
|
||||
"pos": [450, 50],
|
||||
"size": [400, 500],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "image",
|
||||
"type": "IMAGE",
|
||||
"link": 1
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [2]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "ImageCropV2"
|
||||
},
|
||||
"widgets_values": [
|
||||
{
|
||||
"x": 10,
|
||||
"y": 10,
|
||||
"width": 100,
|
||||
"height": 100
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "PreviewImage",
|
||||
"pos": [900, 50],
|
||||
"size": [315, 270],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "images",
|
||||
"type": "IMAGE",
|
||||
"link": 2
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "PreviewImage"
|
||||
},
|
||||
"widgets_values": []
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[1, 1, 0, 2, 0, "IMAGE"],
|
||||
[2, 2, 0, 3, 0, "IMAGE"]
|
||||
],
|
||||
"groups": [],
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"offset": [0, 0],
|
||||
"scale": 1
|
||||
}
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
@@ -26,10 +26,11 @@ export class ComfyMouse implements Omit<Mouse, 'move'> {
|
||||
static defaultSteps = 5
|
||||
static defaultOptions: DragOptions = { steps: ComfyMouse.defaultSteps }
|
||||
|
||||
readonly mouse: Mouse
|
||||
constructor(readonly comfyPage: ComfyPage) {}
|
||||
|
||||
constructor(readonly comfyPage: ComfyPage) {
|
||||
this.mouse = comfyPage.page.mouse
|
||||
/** The normal Playwright {@link Mouse} property from {@link ComfyPage.page}. */
|
||||
get mouse() {
|
||||
return this.comfyPage.page.mouse
|
||||
}
|
||||
|
||||
async nextFrame() {
|
||||
|
||||
@@ -73,13 +73,15 @@ class ComfyMenu {
|
||||
public readonly sideToolbar: Locator
|
||||
public readonly propertiesPanel: ComfyPropertiesPanel
|
||||
public readonly modeToggleButton: Locator
|
||||
public readonly buttons: Locator
|
||||
|
||||
constructor(public readonly page: Page) {
|
||||
this.sideToolbar = page.getByTestId(TestIds.sidebar.toolbar)
|
||||
this.modeToggleButton = page.getByTestId(TestIds.sidebar.modeToggle)
|
||||
this.propertiesPanel = new ComfyPropertiesPanel(page)
|
||||
this.buttons = this.sideToolbar.locator('.side-bar-button')
|
||||
}
|
||||
|
||||
get buttons() {
|
||||
return this.sideToolbar.locator('.side-bar-button')
|
||||
}
|
||||
|
||||
get modelLibraryTab() {
|
||||
@@ -181,7 +183,6 @@ export class ComfyPage {
|
||||
public readonly assetApi: AssetHelper
|
||||
public readonly modelLibrary: ModelLibraryHelper
|
||||
public readonly cloudAuth: CloudAuthHelper
|
||||
public readonly visibleToasts: Locator
|
||||
|
||||
/** Worker index to test user ID */
|
||||
public readonly userIds: string[] = []
|
||||
@@ -224,7 +225,6 @@ export class ComfyPage {
|
||||
this.workflow = new WorkflowHelper(this)
|
||||
this.contextMenu = new ContextMenu(page)
|
||||
this.toast = new ToastHelper(page)
|
||||
this.visibleToasts = this.toast.visibleToasts
|
||||
this.dragDrop = new DragDropHelper(page)
|
||||
this.featureFlags = new FeatureFlagHelper(page)
|
||||
this.command = new CommandHelper(page)
|
||||
@@ -237,6 +237,10 @@ export class ComfyPage {
|
||||
this.cloudAuth = new CloudAuthHelper(page)
|
||||
}
|
||||
|
||||
get visibleToasts() {
|
||||
return this.toast.visibleToasts
|
||||
}
|
||||
|
||||
async setupUser(username: string) {
|
||||
const res = await this.request.get(`${this.url}/api/users`)
|
||||
if (res.status() !== 200)
|
||||
@@ -321,7 +325,7 @@ export class ComfyPage {
|
||||
// window.app.extensionManager => GraphView ready
|
||||
window.app && window.app.extensionManager
|
||||
)
|
||||
await this.page.locator('.p-blockui-mask').waitFor({ state: 'hidden' })
|
||||
await this.page.waitForSelector('.p-blockui-mask', { state: 'hidden' })
|
||||
await this.nextFrame()
|
||||
}
|
||||
|
||||
@@ -371,7 +375,7 @@ export class ComfyPage {
|
||||
}
|
||||
|
||||
async closeMenu() {
|
||||
await this.page.locator('button.comfy-close-menu-btn').click()
|
||||
await this.page.click('button.comfy-close-menu-btn')
|
||||
await this.nextFrame()
|
||||
}
|
||||
|
||||
|
||||
@@ -1,22 +1,30 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
import type { Page } from '@playwright/test'
|
||||
import { test as base } from '@playwright/test'
|
||||
|
||||
export class UserSelectPage {
|
||||
public readonly selectionUrl: string
|
||||
public readonly container: Locator
|
||||
public readonly newUserInput: Locator
|
||||
public readonly existingUserSelect: Locator
|
||||
public readonly nextButton: Locator
|
||||
|
||||
constructor(
|
||||
public readonly url: string,
|
||||
public readonly page: Page
|
||||
) {
|
||||
this.selectionUrl = url + '/user-select'
|
||||
this.container = page.locator('#comfy-user-selection')
|
||||
this.newUserInput = this.container.locator('#new-user-input')
|
||||
this.existingUserSelect = this.container.locator('#existing-user-select')
|
||||
this.nextButton = this.container.getByText('Next')
|
||||
) {}
|
||||
|
||||
get selectionUrl() {
|
||||
return this.url + '/user-select'
|
||||
}
|
||||
|
||||
get container() {
|
||||
return this.page.locator('#comfy-user-selection')
|
||||
}
|
||||
|
||||
get newUserInput() {
|
||||
return this.container.locator('#new-user-input')
|
||||
}
|
||||
|
||||
get existingUserSelect() {
|
||||
return this.container.locator('#existing-user-select')
|
||||
}
|
||||
|
||||
get nextButton() {
|
||||
return this.container.getByText('Next')
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -7,20 +7,13 @@ import { TestIds } from '@e2e/fixtures/selectors'
|
||||
import { VueNodeFixture } from '@e2e/fixtures/utils/vueNodeFixtures'
|
||||
|
||||
export class VueNodeHelpers {
|
||||
constructor(private page: Page) {}
|
||||
|
||||
/**
|
||||
* Get locator for all Vue node components in the DOM
|
||||
*/
|
||||
public readonly nodes: Locator
|
||||
/**
|
||||
* Get locator for selected Vue node components (using visual selection indicators)
|
||||
*/
|
||||
public readonly selectedNodes: Locator
|
||||
|
||||
constructor(private page: Page) {
|
||||
this.nodes = page.locator('[data-node-id]')
|
||||
this.selectedNodes = page.locator(
|
||||
'[data-node-id].outline-node-component-outline'
|
||||
)
|
||||
get nodes(): Locator {
|
||||
return this.page.locator('[data-node-id]')
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -30,6 +23,13 @@ export class VueNodeHelpers {
|
||||
return this.page.locator(`[data-node-id="${nodeId}"]`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get locator for selected Vue node components (using visual selection indicators)
|
||||
*/
|
||||
get selectedNodes(): Locator {
|
||||
return this.page.locator('[data-node-id].outline-node-component-outline')
|
||||
}
|
||||
|
||||
/**
|
||||
* Get locator for Vue nodes by the node's title (displayed name in the header).
|
||||
* Matches against the actual title element, not the full node body.
|
||||
@@ -37,7 +37,7 @@ export class VueNodeHelpers {
|
||||
*/
|
||||
getNodeByTitle(title: string): Locator {
|
||||
return this.page.locator('[data-node-id]').filter({
|
||||
has: this.page.getByTestId('node-title').filter({ hasText: title })
|
||||
has: this.page.locator('[data-testid="node-title"]', { hasText: title })
|
||||
})
|
||||
}
|
||||
|
||||
@@ -146,7 +146,7 @@ export class VueNodeHelpers {
|
||||
expectedCount
|
||||
)
|
||||
} else {
|
||||
await this.page.locator('[data-node-id]').first().waitFor()
|
||||
await this.page.waitForSelector('[data-node-id]')
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -3,11 +3,13 @@ import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
export class ComfyNodeSearchFilterSelectionPanel {
|
||||
readonly root: Locator
|
||||
readonly header: Locator
|
||||
|
||||
constructor(public readonly page: Page) {
|
||||
this.root = page.getByRole('dialog')
|
||||
this.header = this.root
|
||||
}
|
||||
|
||||
get header() {
|
||||
return this.root
|
||||
.locator('div')
|
||||
.filter({ hasText: 'Add node filter condition' })
|
||||
}
|
||||
@@ -39,8 +41,6 @@ export class ComfyNodeSearchFilterSelectionPanel {
|
||||
export class ComfyNodeSearchBox {
|
||||
public readonly input: Locator
|
||||
public readonly dropdown: Locator
|
||||
public readonly filterButton: Locator
|
||||
public readonly filterChips: Locator
|
||||
public readonly filterSelectionPanel: ComfyNodeSearchFilterSelectionPanel
|
||||
|
||||
constructor(public readonly page: Page) {
|
||||
@@ -50,15 +50,13 @@ export class ComfyNodeSearchBox {
|
||||
this.dropdown = page.locator(
|
||||
'.comfy-vue-node-search-container .p-autocomplete-list'
|
||||
)
|
||||
this.filterButton = page.locator(
|
||||
'.comfy-vue-node-search-container .filter-button'
|
||||
)
|
||||
this.filterChips = page.locator(
|
||||
'.comfy-vue-node-search-container .p-autocomplete-chip-item'
|
||||
)
|
||||
this.filterSelectionPanel = new ComfyNodeSearchFilterSelectionPanel(page)
|
||||
}
|
||||
|
||||
get filterButton() {
|
||||
return this.page.locator('.comfy-vue-node-search-container .filter-button')
|
||||
}
|
||||
|
||||
async fillAndSelectFirstNode(
|
||||
nodeName: string,
|
||||
options?: { suggestionIndex?: number; exact?: boolean }
|
||||
@@ -80,6 +78,12 @@ export class ComfyNodeSearchBox {
|
||||
await this.filterSelectionPanel.addFilter(filterValue, filterType)
|
||||
}
|
||||
|
||||
get filterChips() {
|
||||
return this.page.locator(
|
||||
'.comfy-vue-node-search-container .p-autocomplete-chip-item'
|
||||
)
|
||||
}
|
||||
|
||||
async removeFilter(index: number) {
|
||||
await this.filterChips.nth(index).locator('.p-chip-remove-icon').click()
|
||||
}
|
||||
|
||||
@@ -2,14 +2,18 @@ import { expect } from '@playwright/test'
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
export class ContextMenu {
|
||||
public readonly primeVueMenu: Locator
|
||||
public readonly litegraphMenu: Locator
|
||||
public readonly menuItems: Locator
|
||||
constructor(public readonly page: Page) {}
|
||||
|
||||
constructor(public readonly page: Page) {
|
||||
this.primeVueMenu = page.locator('.p-contextmenu, .p-menu')
|
||||
this.litegraphMenu = page.locator('.litemenu')
|
||||
this.menuItems = page.locator('.p-menuitem, .litemenu-entry')
|
||||
get primeVueMenu() {
|
||||
return this.page.locator('.p-contextmenu, .p-menu')
|
||||
}
|
||||
|
||||
get litegraphMenu() {
|
||||
return this.page.locator('.litemenu')
|
||||
}
|
||||
|
||||
get menuItems() {
|
||||
return this.page.locator('.p-menuitem, .litemenu-entry')
|
||||
}
|
||||
|
||||
async clickMenuItem(name: string): Promise<void> {
|
||||
|
||||
@@ -1,97 +0,0 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
import { TestIds } from '../selectors'
|
||||
|
||||
const ids = TestIds.outputHistory
|
||||
|
||||
export class OutputHistoryComponent {
|
||||
constructor(private readonly page: Page) {}
|
||||
|
||||
get outputs(): Locator {
|
||||
return this.page.getByTestId(ids.outputs)
|
||||
}
|
||||
|
||||
get welcome(): Locator {
|
||||
return this.page.getByTestId(ids.welcome)
|
||||
}
|
||||
|
||||
get outputInfo(): Locator {
|
||||
return this.page.getByTestId(ids.outputInfo)
|
||||
}
|
||||
|
||||
get activeQueue(): Locator {
|
||||
return this.page.getByTestId(ids.activeQueue)
|
||||
}
|
||||
|
||||
get queueBadge(): Locator {
|
||||
return this.page.getByTestId(ids.queueBadge)
|
||||
}
|
||||
|
||||
get inProgressItems(): Locator {
|
||||
return this.page.getByTestId(ids.inProgressItem)
|
||||
}
|
||||
|
||||
get historyItems(): Locator {
|
||||
return this.page.getByTestId(ids.historyItem)
|
||||
}
|
||||
|
||||
get skeletons(): Locator {
|
||||
return this.page.getByTestId(ids.skeleton)
|
||||
}
|
||||
|
||||
get latentPreviews(): Locator {
|
||||
return this.page.getByTestId(ids.latentPreview)
|
||||
}
|
||||
|
||||
get imageOutputs(): Locator {
|
||||
return this.page.getByTestId(ids.imageOutput)
|
||||
}
|
||||
|
||||
get videoOutputs(): Locator {
|
||||
return this.page.getByTestId(ids.videoOutput)
|
||||
}
|
||||
|
||||
/** The currently selected (checked) in-progress item. */
|
||||
get selectedInProgressItem(): Locator {
|
||||
return this.page.locator(
|
||||
`[data-testid="${ids.inProgressItem}"][data-state="checked"]`
|
||||
)
|
||||
}
|
||||
|
||||
/** The currently selected (checked) history item. */
|
||||
get selectedHistoryItem(): Locator {
|
||||
return this.page.locator(
|
||||
`[data-testid="${ids.historyItem}"][data-state="checked"]`
|
||||
)
|
||||
}
|
||||
|
||||
/** The header-level progress bar. */
|
||||
get headerProgressBar(): Locator {
|
||||
return this.page.getByTestId(ids.headerProgressBar)
|
||||
}
|
||||
|
||||
/** The in-progress item's progress bar (inside the thumbnail). */
|
||||
get itemProgressBar(): Locator {
|
||||
return this.inProgressItems.first().getByTestId(ids.itemProgressBar)
|
||||
}
|
||||
|
||||
/** Overall progress in the header bar. */
|
||||
get headerOverallProgress(): Locator {
|
||||
return this.headerProgressBar.getByTestId(ids.progressOverall)
|
||||
}
|
||||
|
||||
/** Node progress in the header bar. */
|
||||
get headerNodeProgress(): Locator {
|
||||
return this.headerProgressBar.getByTestId(ids.progressNode)
|
||||
}
|
||||
|
||||
/** Overall progress in the in-progress item bar. */
|
||||
get itemOverallProgress(): Locator {
|
||||
return this.itemProgressBar.getByTestId(ids.progressOverall)
|
||||
}
|
||||
|
||||
/** Node progress in the in-progress item bar. */
|
||||
get itemNodeProgress(): Locator {
|
||||
return this.itemProgressBar.getByTestId(ids.progressNode)
|
||||
}
|
||||
}
|
||||
@@ -1,22 +1,15 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
import { BaseDialog } from '@e2e/fixtures/components/BaseDialog'
|
||||
|
||||
export class SettingDialog extends BaseDialog {
|
||||
public readonly searchBox: Locator
|
||||
public readonly categories: Locator
|
||||
public readonly contentArea: Locator
|
||||
|
||||
constructor(
|
||||
page: Page,
|
||||
public readonly comfyPage: ComfyPage
|
||||
) {
|
||||
super(page, TestIds.dialogs.settings)
|
||||
this.searchBox = this.root.getByPlaceholder(/Search/)
|
||||
this.categories = this.root.locator('nav').getByRole('button')
|
||||
this.contentArea = this.root.getByRole('main')
|
||||
}
|
||||
|
||||
async open() {
|
||||
@@ -43,15 +36,27 @@ export class SettingDialog extends BaseDialog {
|
||||
await settingInputDiv.locator('input').click()
|
||||
}
|
||||
|
||||
get searchBox() {
|
||||
return this.root.getByPlaceholder(/Search/)
|
||||
}
|
||||
|
||||
get categories() {
|
||||
return this.root.locator('nav').getByRole('button')
|
||||
}
|
||||
|
||||
category(name: string) {
|
||||
return this.root.locator('nav').getByRole('button', { name })
|
||||
}
|
||||
|
||||
get contentArea() {
|
||||
return this.root.getByRole('main')
|
||||
}
|
||||
|
||||
async goToAboutPanel() {
|
||||
const aboutButton = this.root.locator('nav').getByRole('button', {
|
||||
name: 'About'
|
||||
})
|
||||
await aboutButton.click()
|
||||
await this.page.locator('.about-container').waitFor()
|
||||
await this.page.waitForSelector('.about-container')
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,16 +5,18 @@ import type { WorkspaceStore } from '@e2e/types/globals'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
class SidebarTab {
|
||||
public readonly tabButton: Locator
|
||||
public readonly selectedTabButton: Locator
|
||||
|
||||
constructor(
|
||||
public readonly page: Page,
|
||||
public readonly tabId: string
|
||||
) {
|
||||
this.tabButton = page.locator(`.${tabId}-tab-button`)
|
||||
this.selectedTabButton = page.locator(
|
||||
`.${tabId}-tab-button.side-bar-button-selected`
|
||||
) {}
|
||||
|
||||
get tabButton() {
|
||||
return this.page.locator(`.${this.tabId}-tab-button`)
|
||||
}
|
||||
|
||||
get selectedTabButton() {
|
||||
return this.page.locator(
|
||||
`.${this.tabId}-tab-button.side-bar-button-selected`
|
||||
)
|
||||
}
|
||||
|
||||
@@ -33,19 +35,28 @@ class SidebarTab {
|
||||
}
|
||||
|
||||
export class NodeLibrarySidebarTab extends SidebarTab {
|
||||
public readonly nodeLibrarySearchBoxInput: Locator
|
||||
public readonly nodeLibraryTree: Locator
|
||||
public readonly nodePreview: Locator
|
||||
public readonly tabContainer: Locator
|
||||
public readonly newFolderButton: Locator
|
||||
|
||||
constructor(public override readonly page: Page) {
|
||||
super(page, 'node-library')
|
||||
this.nodeLibrarySearchBoxInput = page.getByPlaceholder('Search Nodes...')
|
||||
this.nodeLibraryTree = page.getByTestId(TestIds.sidebar.nodeLibrary)
|
||||
this.nodePreview = page.locator('.node-lib-node-preview')
|
||||
this.tabContainer = page.locator('.sidebar-content-container')
|
||||
this.newFolderButton = this.tabContainer.locator('.new-folder-button')
|
||||
}
|
||||
|
||||
get nodeLibrarySearchBoxInput() {
|
||||
return this.page.getByPlaceholder('Search Nodes...')
|
||||
}
|
||||
|
||||
get nodeLibraryTree() {
|
||||
return this.page.getByTestId(TestIds.sidebar.nodeLibrary)
|
||||
}
|
||||
|
||||
get nodePreview() {
|
||||
return this.page.locator('.node-lib-node-preview')
|
||||
}
|
||||
|
||||
get tabContainer() {
|
||||
return this.page.locator('.sidebar-content-container')
|
||||
}
|
||||
|
||||
get newFolderButton() {
|
||||
return this.tabContainer.locator('.new-folder-button')
|
||||
}
|
||||
|
||||
override async open() {
|
||||
@@ -90,25 +101,34 @@ export class NodeLibrarySidebarTab extends SidebarTab {
|
||||
}
|
||||
|
||||
export class NodeLibrarySidebarTabV2 extends SidebarTab {
|
||||
public readonly searchInput: Locator
|
||||
public readonly sidebarContent: Locator
|
||||
public readonly allTab: Locator
|
||||
public readonly blueprintsTab: Locator
|
||||
public readonly sortButton: Locator
|
||||
|
||||
constructor(public override readonly page: Page) {
|
||||
super(page, 'node-library')
|
||||
this.searchInput = page.getByPlaceholder('Search...')
|
||||
this.sidebarContent = page.locator('.sidebar-content-container')
|
||||
this.allTab = this.getTab('All')
|
||||
this.blueprintsTab = this.getTab('Blueprints')
|
||||
this.sortButton = this.sidebarContent.getByRole('button', { name: 'Sort' })
|
||||
}
|
||||
|
||||
get searchInput() {
|
||||
return this.page.getByPlaceholder('Search...')
|
||||
}
|
||||
|
||||
get sidebarContent() {
|
||||
return this.page.locator('.sidebar-content-container')
|
||||
}
|
||||
|
||||
getTab(name: string) {
|
||||
return this.sidebarContent.getByRole('tab', { name, exact: true })
|
||||
}
|
||||
|
||||
get allTab() {
|
||||
return this.getTab('All')
|
||||
}
|
||||
|
||||
get blueprintsTab() {
|
||||
return this.getTab('Blueprints')
|
||||
}
|
||||
|
||||
get sortButton() {
|
||||
return this.sidebarContent.getByRole('button', { name: 'Sort' })
|
||||
}
|
||||
|
||||
getFolder(folderName: string) {
|
||||
return this.sidebarContent
|
||||
.getByRole('treeitem', { name: folderName })
|
||||
@@ -134,15 +154,12 @@ export class NodeLibrarySidebarTabV2 extends SidebarTab {
|
||||
}
|
||||
|
||||
export class WorkflowsSidebarTab extends SidebarTab {
|
||||
public readonly root: Locator
|
||||
public readonly activeWorkflowLabel: Locator
|
||||
|
||||
constructor(public override readonly page: Page) {
|
||||
super(page, 'workflows')
|
||||
this.root = page.getByTestId(TestIds.sidebar.workflows)
|
||||
this.activeWorkflowLabel = this.root.locator(
|
||||
'.comfyui-workflows-open .p-tree-node-selected .node-label'
|
||||
)
|
||||
}
|
||||
|
||||
get root() {
|
||||
return this.page.getByTestId(TestIds.sidebar.workflows)
|
||||
}
|
||||
|
||||
async getOpenedWorkflowNames() {
|
||||
@@ -151,6 +168,12 @@ export class WorkflowsSidebarTab extends SidebarTab {
|
||||
.allInnerTexts()
|
||||
}
|
||||
|
||||
get activeWorkflowLabel(): Locator {
|
||||
return this.root.locator(
|
||||
'.comfyui-workflows-open .p-tree-node-selected .node-label'
|
||||
)
|
||||
}
|
||||
|
||||
async getActiveWorkflowName() {
|
||||
return await this.activeWorkflowLabel.innerText()
|
||||
}
|
||||
@@ -205,27 +228,36 @@ export class WorkflowsSidebarTab extends SidebarTab {
|
||||
}
|
||||
|
||||
export class ModelLibrarySidebarTab extends SidebarTab {
|
||||
public readonly searchInput: Locator
|
||||
public readonly modelTree: Locator
|
||||
public readonly refreshButton: Locator
|
||||
public readonly loadAllFoldersButton: Locator
|
||||
public readonly folderNodes: Locator
|
||||
public readonly leafNodes: Locator
|
||||
public readonly modelPreview: Locator
|
||||
|
||||
constructor(public override readonly page: Page) {
|
||||
super(page, 'model-library')
|
||||
this.searchInput = page.getByPlaceholder('Search Models...')
|
||||
this.modelTree = page.locator('.model-lib-tree-explorer')
|
||||
this.refreshButton = page.getByRole('button', { name: 'Refresh' })
|
||||
this.loadAllFoldersButton = page.getByRole('button', {
|
||||
name: 'Load All Folders'
|
||||
})
|
||||
this.folderNodes = this.modelTree.locator(
|
||||
'.p-tree-node:not(.p-tree-node-leaf)'
|
||||
)
|
||||
this.leafNodes = this.modelTree.locator('.p-tree-node-leaf')
|
||||
this.modelPreview = page.locator('.model-lib-model-preview')
|
||||
}
|
||||
|
||||
get searchInput() {
|
||||
return this.page.getByPlaceholder('Search Models...')
|
||||
}
|
||||
|
||||
get modelTree() {
|
||||
return this.page.locator('.model-lib-tree-explorer')
|
||||
}
|
||||
|
||||
get refreshButton() {
|
||||
return this.page.getByRole('button', { name: 'Refresh' })
|
||||
}
|
||||
|
||||
get loadAllFoldersButton() {
|
||||
return this.page.getByRole('button', { name: 'Load All Folders' })
|
||||
}
|
||||
|
||||
get folderNodes() {
|
||||
return this.modelTree.locator('.p-tree-node:not(.p-tree-node-leaf)')
|
||||
}
|
||||
|
||||
get leafNodes() {
|
||||
return this.modelTree.locator('.p-tree-node-leaf')
|
||||
}
|
||||
|
||||
get modelPreview() {
|
||||
return this.page.locator('.model-lib-model-preview')
|
||||
}
|
||||
|
||||
override async open() {
|
||||
@@ -249,97 +281,137 @@ export class ModelLibrarySidebarTab extends SidebarTab {
|
||||
}
|
||||
|
||||
export class AssetsSidebarTab extends SidebarTab {
|
||||
// --- Tab navigation ---
|
||||
public readonly generatedTab: Locator
|
||||
public readonly importedTab: Locator
|
||||
|
||||
// --- Empty state ---
|
||||
public readonly emptyStateMessage: Locator
|
||||
|
||||
// --- Search & filter ---
|
||||
public readonly searchInput: Locator
|
||||
public readonly settingsButton: Locator
|
||||
|
||||
// --- View mode ---
|
||||
public readonly listViewOption: Locator
|
||||
public readonly gridViewOption: Locator
|
||||
|
||||
// --- Sort options (cloud-only, shown inside settings popover) ---
|
||||
public readonly sortNewestFirst: Locator
|
||||
public readonly sortOldestFirst: Locator
|
||||
|
||||
// --- Asset cards ---
|
||||
public readonly assetCards: Locator
|
||||
public readonly selectedCards: Locator
|
||||
|
||||
// --- List view items ---
|
||||
public readonly listViewItems: Locator
|
||||
|
||||
// --- Selection footer ---
|
||||
public readonly selectionFooter: Locator
|
||||
public readonly selectionCountButton: Locator
|
||||
public readonly deselectAllButton: Locator
|
||||
public readonly deleteSelectedButton: Locator
|
||||
public readonly downloadSelectedButton: Locator
|
||||
|
||||
// --- Folder view ---
|
||||
public readonly backToAssetsButton: Locator
|
||||
|
||||
// --- Loading ---
|
||||
public readonly skeletonLoaders: Locator
|
||||
|
||||
constructor(public override readonly page: Page) {
|
||||
super(page, 'assets')
|
||||
this.generatedTab = page.getByRole('tab', { name: 'Generated' })
|
||||
this.importedTab = page.getByRole('tab', { name: 'Imported' })
|
||||
this.emptyStateMessage = page.getByText(
|
||||
}
|
||||
|
||||
// --- Tab navigation ---
|
||||
|
||||
get generatedTab() {
|
||||
return this.page.getByRole('tab', { name: 'Generated' })
|
||||
}
|
||||
|
||||
get importedTab() {
|
||||
return this.page.getByRole('tab', { name: 'Imported' })
|
||||
}
|
||||
|
||||
// --- Empty state ---
|
||||
|
||||
get emptyStateMessage() {
|
||||
return this.page.getByText(
|
||||
'Upload files or generate content to see them here'
|
||||
)
|
||||
this.searchInput = page.getByPlaceholder('Search Assets...')
|
||||
this.settingsButton = page.getByRole('button', { name: 'View settings' })
|
||||
this.listViewOption = page.getByText('List view')
|
||||
this.gridViewOption = page.getByText('Grid view')
|
||||
this.sortNewestFirst = page.getByText('Newest first')
|
||||
this.sortOldestFirst = page.getByText('Oldest first')
|
||||
this.assetCards = page
|
||||
.getByRole('button')
|
||||
.and(page.locator('[data-selected]'))
|
||||
this.selectedCards = page.locator('[data-selected="true"]')
|
||||
this.listViewItems = page.locator(
|
||||
'.sidebar-content-container [role="button"][tabindex="0"]'
|
||||
)
|
||||
this.selectionFooter = page
|
||||
.locator('.sidebar-content-container')
|
||||
.locator('..')
|
||||
.locator('[class*="h-18"]')
|
||||
this.selectionCountButton = page.getByText(/Assets Selected: \d+/)
|
||||
this.deselectAllButton = page.getByText('Deselect all')
|
||||
this.deleteSelectedButton = page
|
||||
.getByTestId('assets-delete-selected')
|
||||
.or(page.locator('button:has(.icon-\\[lucide--trash-2\\])').last())
|
||||
.first()
|
||||
this.downloadSelectedButton = page
|
||||
.getByTestId('assets-download-selected')
|
||||
.or(page.locator('button:has(.icon-\\[lucide--download\\])').last())
|
||||
.first()
|
||||
this.backToAssetsButton = page.getByText('Back to all assets')
|
||||
this.skeletonLoaders = page.locator(
|
||||
'.sidebar-content-container .animate-pulse'
|
||||
)
|
||||
}
|
||||
|
||||
emptyStateTitle(title: string) {
|
||||
return this.page.getByText(title)
|
||||
}
|
||||
|
||||
getAssetCardByName(name: string) {
|
||||
return this.assetCards.filter({ hasText: name })
|
||||
// --- Search & filter ---
|
||||
|
||||
get searchInput() {
|
||||
return this.page.getByPlaceholder('Search Assets...')
|
||||
}
|
||||
|
||||
get settingsButton() {
|
||||
return this.page.getByRole('button', { name: 'View settings' })
|
||||
}
|
||||
|
||||
// --- View mode ---
|
||||
|
||||
get listViewOption() {
|
||||
return this.page.getByText('List view')
|
||||
}
|
||||
|
||||
get gridViewOption() {
|
||||
return this.page.getByText('Grid view')
|
||||
}
|
||||
|
||||
// --- Sort options (cloud-only, shown inside settings popover) ---
|
||||
|
||||
get sortNewestFirst() {
|
||||
return this.page.getByText('Newest first')
|
||||
}
|
||||
|
||||
get sortOldestFirst() {
|
||||
return this.page.getByText('Oldest first')
|
||||
}
|
||||
|
||||
// --- Asset cards ---
|
||||
|
||||
get assetCards() {
|
||||
return this.page.locator('[role="button"][data-selected]')
|
||||
}
|
||||
|
||||
getAssetCardByName(name: string) {
|
||||
return this.page.locator('[role="button"][data-selected]', {
|
||||
hasText: name
|
||||
})
|
||||
}
|
||||
|
||||
get selectedCards() {
|
||||
return this.page.locator('[data-selected="true"]')
|
||||
}
|
||||
|
||||
// --- List view items ---
|
||||
|
||||
get listViewItems() {
|
||||
return this.page.locator(
|
||||
'.sidebar-content-container [role="button"][tabindex="0"]'
|
||||
)
|
||||
}
|
||||
|
||||
// --- Selection footer ---
|
||||
|
||||
get selectionFooter() {
|
||||
return this.page
|
||||
.locator('.sidebar-content-container')
|
||||
.locator('..')
|
||||
.locator('[class*="h-18"]')
|
||||
}
|
||||
|
||||
get selectionCountButton() {
|
||||
return this.page.getByText(/Assets Selected: \d+/)
|
||||
}
|
||||
|
||||
get deselectAllButton() {
|
||||
return this.page.getByText('Deselect all')
|
||||
}
|
||||
|
||||
get deleteSelectedButton() {
|
||||
return this.page
|
||||
.getByTestId('assets-delete-selected')
|
||||
.or(this.page.locator('button:has(.icon-\\[lucide--trash-2\\])').last())
|
||||
.first()
|
||||
}
|
||||
|
||||
get downloadSelectedButton() {
|
||||
return this.page
|
||||
.getByTestId('assets-download-selected')
|
||||
.or(this.page.locator('button:has(.icon-\\[lucide--download\\])').last())
|
||||
.first()
|
||||
}
|
||||
|
||||
// --- Context menu ---
|
||||
|
||||
contextMenuItem(label: string) {
|
||||
return this.page.locator('.p-contextmenu').getByText(label)
|
||||
}
|
||||
|
||||
// --- Folder view ---
|
||||
|
||||
get backToAssetsButton() {
|
||||
return this.page.getByText('Back to all assets')
|
||||
}
|
||||
|
||||
// --- Loading ---
|
||||
|
||||
get skeletonLoaders() {
|
||||
return this.page.locator('.sidebar-content-container .animate-pulse')
|
||||
}
|
||||
|
||||
// --- Helpers ---
|
||||
|
||||
override async open() {
|
||||
// Remove any toast notifications that may overlay the sidebar button
|
||||
await this.dismissToasts()
|
||||
@@ -351,24 +423,28 @@ export class AssetsSidebarTab extends SidebarTab {
|
||||
async dismissToasts() {
|
||||
const closeButtons = this.page.locator('.p-toast-close-button')
|
||||
for (const btn of await closeButtons.all()) {
|
||||
await btn.click().catch(() => {})
|
||||
await btn.click({ force: true }).catch(() => {})
|
||||
}
|
||||
// Wait for all toast elements to fully animate out and detach from DOM
|
||||
await expect(this.page.locator('.p-toast-message'))
|
||||
.toHaveCount(0)
|
||||
.toHaveCount(0, { timeout: 5000 })
|
||||
.catch(() => {})
|
||||
}
|
||||
|
||||
async switchToImported() {
|
||||
await this.dismissToasts()
|
||||
await this.importedTab.click()
|
||||
await expect(this.importedTab).toHaveAttribute('aria-selected', 'true')
|
||||
await expect(this.importedTab).toHaveAttribute('aria-selected', 'true', {
|
||||
timeout: 3000
|
||||
})
|
||||
}
|
||||
|
||||
async switchToGenerated() {
|
||||
await this.dismissToasts()
|
||||
await this.generatedTab.click()
|
||||
await expect(this.generatedTab).toHaveAttribute('aria-selected', 'true')
|
||||
await expect(this.generatedTab).toHaveAttribute('aria-selected', 'true', {
|
||||
timeout: 3000
|
||||
})
|
||||
}
|
||||
|
||||
async openSettingsMenu() {
|
||||
@@ -391,7 +467,7 @@ export class AssetsSidebarTab extends SidebarTab {
|
||||
|
||||
async waitForAssets(count?: number) {
|
||||
if (count !== undefined) {
|
||||
await expect(this.assetCards).toHaveCount(count)
|
||||
await expect(this.assetCards).toHaveCount(count, { timeout: 5000 })
|
||||
} else {
|
||||
await this.assetCards.first().waitFor({ state: 'visible', timeout: 5000 })
|
||||
}
|
||||
|
||||
@@ -10,17 +10,6 @@ export class SignInDialog extends BaseDialog {
|
||||
readonly apiKeyButton: Locator
|
||||
readonly termsLink: Locator
|
||||
readonly privacyLink: Locator
|
||||
readonly heading: Locator
|
||||
readonly signUpLink: Locator
|
||||
readonly signInLink: Locator
|
||||
readonly signUpEmailInput: Locator
|
||||
readonly signUpPasswordInput: Locator
|
||||
readonly signUpConfirmPasswordInput: Locator
|
||||
readonly signUpButton: Locator
|
||||
readonly apiKeyHeading: Locator
|
||||
readonly apiKeyInput: Locator
|
||||
readonly backButton: Locator
|
||||
readonly dividerText: Locator
|
||||
|
||||
constructor(page: Page) {
|
||||
super(page)
|
||||
@@ -33,22 +22,6 @@ export class SignInDialog extends BaseDialog {
|
||||
})
|
||||
this.termsLink = this.root.getByRole('link', { name: 'Terms of Use' })
|
||||
this.privacyLink = this.root.getByRole('link', { name: 'Privacy Policy' })
|
||||
this.heading = this.root.getByRole('heading').first()
|
||||
this.signUpLink = this.root.getByText('Sign up', { exact: true })
|
||||
this.signInLink = this.root.getByText('Sign in', { exact: true })
|
||||
this.signUpEmailInput = this.root.locator('#comfy-org-sign-up-email')
|
||||
this.signUpPasswordInput = this.root.locator('#comfy-org-sign-up-password')
|
||||
this.signUpConfirmPasswordInput = this.root.locator(
|
||||
'#comfy-org-sign-up-confirm-password'
|
||||
)
|
||||
this.signUpButton = this.root.getByRole('button', {
|
||||
name: 'Sign up',
|
||||
exact: true
|
||||
})
|
||||
this.apiKeyHeading = this.root.getByRole('heading', { name: 'API Key' })
|
||||
this.apiKeyInput = this.root.locator('#comfy-org-api-key')
|
||||
this.backButton = this.root.getByRole('button', { name: 'Back' })
|
||||
this.dividerText = this.root.getByText('Or continue with')
|
||||
}
|
||||
|
||||
async open() {
|
||||
@@ -57,4 +30,48 @@ export class SignInDialog extends BaseDialog {
|
||||
})
|
||||
await this.waitForVisible()
|
||||
}
|
||||
|
||||
get heading() {
|
||||
return this.root.getByRole('heading').first()
|
||||
}
|
||||
|
||||
get signUpLink() {
|
||||
return this.root.getByText('Sign up', { exact: true })
|
||||
}
|
||||
|
||||
get signInLink() {
|
||||
return this.root.getByText('Sign in', { exact: true })
|
||||
}
|
||||
|
||||
get signUpEmailInput() {
|
||||
return this.root.locator('#comfy-org-sign-up-email')
|
||||
}
|
||||
|
||||
get signUpPasswordInput() {
|
||||
return this.root.locator('#comfy-org-sign-up-password')
|
||||
}
|
||||
|
||||
get signUpConfirmPasswordInput() {
|
||||
return this.root.locator('#comfy-org-sign-up-confirm-password')
|
||||
}
|
||||
|
||||
get signUpButton() {
|
||||
return this.root.getByRole('button', { name: 'Sign up', exact: true })
|
||||
}
|
||||
|
||||
get apiKeyHeading() {
|
||||
return this.root.getByRole('heading', { name: 'API Key' })
|
||||
}
|
||||
|
||||
get apiKeyInput() {
|
||||
return this.root.locator('#comfy-org-api-key')
|
||||
}
|
||||
|
||||
get backButton() {
|
||||
return this.root.getByRole('button', { name: 'Back' })
|
||||
}
|
||||
|
||||
get dividerText() {
|
||||
return this.root.getByText('Or continue with')
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,12 +5,10 @@ import type { WorkspaceStore } from '@e2e/types/globals'
|
||||
export class Topbar {
|
||||
private readonly menuLocator: Locator
|
||||
private readonly menuTrigger: Locator
|
||||
readonly newWorkflowButton: Locator
|
||||
|
||||
constructor(public readonly page: Page) {
|
||||
this.menuLocator = page.locator('.comfy-command-menu')
|
||||
this.menuTrigger = page.locator('.comfy-menu-button-wrapper')
|
||||
this.newWorkflowButton = page.locator('.new-blank-workflow-button')
|
||||
}
|
||||
|
||||
async getTabNames(): Promise<string[]> {
|
||||
@@ -52,6 +50,10 @@ export class Topbar {
|
||||
return classes ? !classes.includes('invisible') : false
|
||||
}
|
||||
|
||||
get newWorkflowButton(): Locator {
|
||||
return this.page.locator('.new-blank-workflow-button')
|
||||
}
|
||||
|
||||
getWorkflowTab(tabName: string): Locator {
|
||||
return this.page
|
||||
.locator(`.workflow-tabs .workflow-label:has-text("${tabName}")`)
|
||||
@@ -71,7 +73,7 @@ export class Topbar {
|
||||
async closeWorkflowTab(tabName: string) {
|
||||
const tab = this.getWorkflowTab(tabName)
|
||||
await tab.hover()
|
||||
await tab.locator('.close-button').click()
|
||||
await tab.locator('.close-button').click({ force: true })
|
||||
}
|
||||
|
||||
getSaveDialog(): Locator {
|
||||
@@ -105,7 +107,7 @@ export class Topbar {
|
||||
{ timeout: 3000 }
|
||||
)
|
||||
// Wait for the dialog to close.
|
||||
await this.getSaveDialog().waitFor({ state: 'hidden' })
|
||||
await this.getSaveDialog().waitFor({ state: 'hidden', timeout: 500 })
|
||||
|
||||
// Check if a confirmation dialog appeared (e.g., "Overwrite existing file?")
|
||||
// If so, return early to let the test handle the confirmation
|
||||
|
||||
@@ -3,7 +3,6 @@ import type { Locator, Page } from '@playwright/test'
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
import { OutputHistoryComponent } from '@e2e/fixtures/components/OutputHistory'
|
||||
import { AppModeWidgetHelper } from '@e2e/fixtures/helpers/AppModeWidgetHelper'
|
||||
import { BuilderFooterHelper } from '@e2e/fixtures/helpers/BuilderFooterHelper'
|
||||
import { BuilderSaveAsHelper } from '@e2e/fixtures/helpers/BuilderSaveAsHelper'
|
||||
@@ -15,66 +14,14 @@ export class AppModeHelper {
|
||||
readonly footer: BuilderFooterHelper
|
||||
readonly saveAs: BuilderSaveAsHelper
|
||||
readonly select: BuilderSelectHelper
|
||||
readonly outputHistory: OutputHistoryComponent
|
||||
readonly widgets: AppModeWidgetHelper
|
||||
/** The "Connect an output" popover shown when saving without outputs. */
|
||||
public readonly connectOutputPopover: Locator
|
||||
/** The empty-state placeholder shown when no outputs are selected. */
|
||||
public readonly outputPlaceholder: Locator
|
||||
/** The linear-mode widget list container (visible in app mode). */
|
||||
public readonly linearWidgets: Locator
|
||||
/** The PrimeVue Popover for the image picker (renders with role="dialog"). */
|
||||
public readonly imagePickerPopover: Locator
|
||||
/** The Run button in the app mode footer. */
|
||||
public readonly runButton: Locator
|
||||
/** The welcome screen shown when app mode has no outputs or no nodes. */
|
||||
public readonly welcome: Locator
|
||||
/** The empty workflow message shown when no nodes exist. */
|
||||
public readonly emptyWorkflowText: Locator
|
||||
/** The "Build app" button shown when nodes exist but no outputs. */
|
||||
public readonly buildAppButton: Locator
|
||||
/** The "Back to workflow" button on the welcome screen. */
|
||||
public readonly backToWorkflowButton: Locator
|
||||
/** The "Load template" button shown when no nodes exist. */
|
||||
public readonly loadTemplateButton: Locator
|
||||
/** The cancel button for an in-progress run in the output history. */
|
||||
public readonly cancelRunButton: Locator
|
||||
|
||||
constructor(private readonly comfyPage: ComfyPage) {
|
||||
this.steps = new BuilderStepsHelper(comfyPage)
|
||||
this.footer = new BuilderFooterHelper(comfyPage)
|
||||
this.saveAs = new BuilderSaveAsHelper(comfyPage)
|
||||
this.select = new BuilderSelectHelper(comfyPage)
|
||||
this.outputHistory = new OutputHistoryComponent(comfyPage.page)
|
||||
this.widgets = new AppModeWidgetHelper(comfyPage)
|
||||
this.connectOutputPopover = this.page.getByTestId(
|
||||
TestIds.builder.connectOutputPopover
|
||||
)
|
||||
this.outputPlaceholder = this.page.getByTestId(
|
||||
TestIds.builder.outputPlaceholder
|
||||
)
|
||||
this.linearWidgets = this.page.getByTestId('linear-widgets')
|
||||
this.imagePickerPopover = this.page
|
||||
.getByRole('dialog')
|
||||
.filter({ has: this.page.getByRole('button', { name: 'All' }) })
|
||||
.first()
|
||||
this.runButton = this.page
|
||||
.getByTestId('linear-run-button')
|
||||
.getByRole('button', { name: /run/i })
|
||||
this.welcome = this.page.getByTestId(TestIds.appMode.welcome)
|
||||
this.emptyWorkflowText = this.page.getByTestId(
|
||||
TestIds.appMode.emptyWorkflow
|
||||
)
|
||||
this.buildAppButton = this.page.getByTestId(TestIds.appMode.buildApp)
|
||||
this.backToWorkflowButton = this.page.getByTestId(
|
||||
TestIds.appMode.backToWorkflow
|
||||
)
|
||||
this.loadTemplateButton = this.page.getByTestId(
|
||||
TestIds.appMode.loadTemplate
|
||||
)
|
||||
this.cancelRunButton = this.page.getByTestId(
|
||||
TestIds.outputHistory.cancelRun
|
||||
)
|
||||
}
|
||||
|
||||
private get page(): Page {
|
||||
@@ -146,6 +93,61 @@ export class AppModeHelper {
|
||||
await this.toggleAppMode()
|
||||
}
|
||||
|
||||
/** The "Connect an output" popover shown when saving without outputs. */
|
||||
get connectOutputPopover(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.connectOutputPopover)
|
||||
}
|
||||
|
||||
/** The empty-state placeholder shown when no outputs are selected. */
|
||||
get outputPlaceholder(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.outputPlaceholder)
|
||||
}
|
||||
|
||||
/** The linear-mode widget list container (visible in app mode). */
|
||||
get linearWidgets(): Locator {
|
||||
return this.page.locator('[data-testid="linear-widgets"]')
|
||||
}
|
||||
|
||||
/** The PrimeVue Popover for the image picker (renders with role="dialog"). */
|
||||
get imagePickerPopover(): Locator {
|
||||
return this.page
|
||||
.getByRole('dialog')
|
||||
.filter({ has: this.page.getByRole('button', { name: 'All' }) })
|
||||
.first()
|
||||
}
|
||||
|
||||
/** The Run button in the app mode footer. */
|
||||
get runButton(): Locator {
|
||||
return this.page
|
||||
.getByTestId('linear-run-button')
|
||||
.getByRole('button', { name: /run/i })
|
||||
}
|
||||
|
||||
/** The welcome screen shown when app mode has no outputs or no nodes. */
|
||||
get welcome(): Locator {
|
||||
return this.page.getByTestId(TestIds.appMode.welcome)
|
||||
}
|
||||
|
||||
/** The empty workflow message shown when no nodes exist. */
|
||||
get emptyWorkflowText(): Locator {
|
||||
return this.page.getByTestId(TestIds.appMode.emptyWorkflow)
|
||||
}
|
||||
|
||||
/** The "Build app" button shown when nodes exist but no outputs. */
|
||||
get buildAppButton(): Locator {
|
||||
return this.page.getByTestId(TestIds.appMode.buildApp)
|
||||
}
|
||||
|
||||
/** The "Back to workflow" button on the welcome screen. */
|
||||
get backToWorkflowButton(): Locator {
|
||||
return this.page.getByTestId(TestIds.appMode.backToWorkflow)
|
||||
}
|
||||
|
||||
/** The "Load template" button shown when no nodes exist. */
|
||||
get loadTemplateButton(): Locator {
|
||||
return this.page.getByTestId(TestIds.appMode.loadTemplate)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the actions menu trigger for a widget in the app mode widget list.
|
||||
* @param widgetName Text shown in the widget label (e.g. "seed").
|
||||
|
||||
@@ -4,32 +4,48 @@ import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
export class BuilderFooterHelper {
|
||||
public readonly nav: Locator
|
||||
public readonly exitButton: Locator
|
||||
public readonly nextButton: Locator
|
||||
public readonly backButton: Locator
|
||||
public readonly saveButton: Locator
|
||||
public readonly saveGroup: Locator
|
||||
public readonly saveAsButton: Locator
|
||||
public readonly saveAsChevron: Locator
|
||||
public readonly opensAsPopover: Locator
|
||||
|
||||
constructor(private readonly comfyPage: ComfyPage) {
|
||||
this.nav = this.page.getByTestId(TestIds.builder.footerNav)
|
||||
this.exitButton = this.buttonByName('Exit app builder')
|
||||
this.nextButton = this.buttonByName('Next')
|
||||
this.backButton = this.buttonByName('Back')
|
||||
this.saveButton = this.page.getByTestId(TestIds.builder.saveButton)
|
||||
this.saveGroup = this.page.getByTestId(TestIds.builder.saveGroup)
|
||||
this.saveAsButton = this.page.getByTestId(TestIds.builder.saveAsButton)
|
||||
this.saveAsChevron = this.page.getByTestId(TestIds.builder.saveAsChevron)
|
||||
this.opensAsPopover = this.page.getByTestId(TestIds.builder.opensAs)
|
||||
}
|
||||
constructor(private readonly comfyPage: ComfyPage) {}
|
||||
|
||||
private get page(): Page {
|
||||
return this.comfyPage.page
|
||||
}
|
||||
|
||||
get nav(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.footerNav)
|
||||
}
|
||||
|
||||
get exitButton(): Locator {
|
||||
return this.buttonByName('Exit app builder')
|
||||
}
|
||||
|
||||
get nextButton(): Locator {
|
||||
return this.buttonByName('Next')
|
||||
}
|
||||
|
||||
get backButton(): Locator {
|
||||
return this.buttonByName('Back')
|
||||
}
|
||||
|
||||
get saveButton(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.saveButton)
|
||||
}
|
||||
|
||||
get saveGroup(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.saveGroup)
|
||||
}
|
||||
|
||||
get saveAsButton(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.saveAsButton)
|
||||
}
|
||||
|
||||
get saveAsChevron(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.saveAsChevron)
|
||||
}
|
||||
|
||||
get opensAsPopover(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.opensAs)
|
||||
}
|
||||
|
||||
private buttonByName(name: string): Locator {
|
||||
return this.nav.getByRole('button', { name })
|
||||
}
|
||||
|
||||
@@ -3,61 +3,73 @@ import type { Locator, Page } from '@playwright/test'
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
export class BuilderSaveAsHelper {
|
||||
/** The save-as dialog (scoped by aria-labelledby). */
|
||||
public readonly dialog: Locator
|
||||
/** The post-save success dialog (scoped by aria-labelledby). */
|
||||
public readonly successDialog: Locator
|
||||
public readonly title: Locator
|
||||
public readonly radioGroup: Locator
|
||||
public readonly nameInput: Locator
|
||||
public readonly saveButton: Locator
|
||||
public readonly successMessage: Locator
|
||||
public readonly viewAppButton: Locator
|
||||
public readonly closeButton: Locator
|
||||
/** The X button to dismiss the success dialog without any action. */
|
||||
public readonly dismissButton: Locator
|
||||
public readonly exitBuilderButton: Locator
|
||||
public readonly overwriteDialog: Locator
|
||||
public readonly overwriteButton: Locator
|
||||
|
||||
constructor(private readonly comfyPage: ComfyPage) {
|
||||
this.dialog = this.page.locator('[aria-labelledby="builder-save"]')
|
||||
this.successDialog = this.page.locator(
|
||||
'[aria-labelledby="builder-save-success"]'
|
||||
)
|
||||
this.title = this.dialog.getByText('Save as')
|
||||
this.radioGroup = this.dialog.getByRole('radiogroup')
|
||||
this.nameInput = this.dialog.getByRole('textbox')
|
||||
this.saveButton = this.dialog.getByRole('button', { name: 'Save' })
|
||||
this.successMessage = this.successDialog.getByText('Successfully saved')
|
||||
this.viewAppButton = this.successDialog.getByRole('button', {
|
||||
name: 'View app'
|
||||
})
|
||||
this.closeButton = this.successDialog
|
||||
.getByRole('button', { name: 'Close', exact: true })
|
||||
.filter({ hasText: 'Close' })
|
||||
this.dismissButton = this.successDialog.locator(
|
||||
'button.p-dialog-close-button'
|
||||
)
|
||||
this.exitBuilderButton = this.successDialog.getByRole('button', {
|
||||
name: 'Exit builder'
|
||||
})
|
||||
this.overwriteDialog = this.page.getByRole('dialog', {
|
||||
name: 'Overwrite existing file?'
|
||||
})
|
||||
this.overwriteButton = this.overwriteDialog.getByRole('button', {
|
||||
name: 'Overwrite'
|
||||
})
|
||||
}
|
||||
constructor(private readonly comfyPage: ComfyPage) {}
|
||||
|
||||
private get page(): Page {
|
||||
return this.comfyPage.page
|
||||
}
|
||||
|
||||
/** The save-as dialog (scoped by aria-labelledby). */
|
||||
get dialog(): Locator {
|
||||
return this.page.locator('[aria-labelledby="builder-save"]')
|
||||
}
|
||||
|
||||
/** The post-save success dialog (scoped by aria-labelledby). */
|
||||
get successDialog(): Locator {
|
||||
return this.page.locator('[aria-labelledby="builder-save-success"]')
|
||||
}
|
||||
|
||||
get title(): Locator {
|
||||
return this.dialog.getByText('Save as')
|
||||
}
|
||||
|
||||
get radioGroup(): Locator {
|
||||
return this.dialog.getByRole('radiogroup')
|
||||
}
|
||||
|
||||
get nameInput(): Locator {
|
||||
return this.dialog.getByRole('textbox')
|
||||
}
|
||||
|
||||
viewTypeRadio(viewType: 'App' | 'Node graph'): Locator {
|
||||
return this.dialog.getByRole('radio', { name: viewType })
|
||||
}
|
||||
|
||||
get saveButton(): Locator {
|
||||
return this.dialog.getByRole('button', { name: 'Save' })
|
||||
}
|
||||
|
||||
get successMessage(): Locator {
|
||||
return this.successDialog.getByText('Successfully saved')
|
||||
}
|
||||
|
||||
get viewAppButton(): Locator {
|
||||
return this.successDialog.getByRole('button', { name: 'View app' })
|
||||
}
|
||||
|
||||
get closeButton(): Locator {
|
||||
return this.successDialog
|
||||
.getByRole('button', { name: 'Close', exact: true })
|
||||
.filter({ hasText: 'Close' })
|
||||
}
|
||||
|
||||
/** The X button to dismiss the success dialog without any action. */
|
||||
get dismissButton(): Locator {
|
||||
return this.successDialog.locator('button.p-dialog-close-button')
|
||||
}
|
||||
|
||||
get exitBuilderButton(): Locator {
|
||||
return this.successDialog.getByRole('button', { name: 'Exit builder' })
|
||||
}
|
||||
|
||||
get overwriteDialog(): Locator {
|
||||
return this.page.getByRole('dialog', { name: 'Overwrite existing file?' })
|
||||
}
|
||||
|
||||
get overwriteButton(): Locator {
|
||||
return this.overwriteDialog.getByRole('button', { name: 'Overwrite' })
|
||||
}
|
||||
|
||||
async fillAndSave(workflowName: string, viewType: 'App' | 'Node graph') {
|
||||
await this.nameInput.fill(workflowName)
|
||||
await this.viewTypeRadio(viewType).click()
|
||||
|
||||
@@ -32,20 +32,7 @@ async function dragByIndex(items: Locator, fromIndex: number, toIndex: number) {
|
||||
}
|
||||
|
||||
export class BuilderSelectHelper {
|
||||
/** All IoItem locators in the current step sidebar. */
|
||||
public readonly inputItems: Locator
|
||||
/** All IoItem title locators in the inputs step sidebar. */
|
||||
public readonly inputItemTitles: Locator
|
||||
/** All widget label locators in the preview/arrange sidebar. */
|
||||
public readonly previewWidgetLabels: Locator
|
||||
|
||||
constructor(private readonly comfyPage: ComfyPage) {
|
||||
this.inputItems = this.page.getByTestId(TestIds.builder.ioItem)
|
||||
this.inputItemTitles = this.page.getByTestId(TestIds.builder.ioItemTitle)
|
||||
this.previewWidgetLabels = this.page.getByTestId(
|
||||
TestIds.builder.widgetLabel
|
||||
)
|
||||
}
|
||||
constructor(private readonly comfyPage: ComfyPage) {}
|
||||
|
||||
private get page(): Page {
|
||||
return this.comfyPage.page
|
||||
@@ -56,9 +43,12 @@ export class BuilderSelectHelper {
|
||||
* @param title The widget title shown in the IoItem.
|
||||
*/
|
||||
getInputItemMenu(title: string): Locator {
|
||||
return this.inputItems
|
||||
return this.page
|
||||
.getByTestId(TestIds.builder.ioItem)
|
||||
.filter({
|
||||
has: this.inputItemTitles.getByText(title, { exact: true })
|
||||
has: this.page
|
||||
.getByTestId(TestIds.builder.ioItemTitle)
|
||||
.getByText(title, { exact: true })
|
||||
})
|
||||
.getByTestId(TestIds.builder.widgetActionsMenu)
|
||||
}
|
||||
@@ -151,7 +141,6 @@ export class BuilderSelectHelper {
|
||||
const widgetLocator = this.comfyPage.vueNodes
|
||||
.getNodeLocator(String(nodeRef.id))
|
||||
.getByLabel(widgetName, { exact: true })
|
||||
// oxlint-disable-next-line playwright/no-force-option -- Node container has conditional pointer-events:none that blocks actionability
|
||||
await widgetLocator.click({ force: true })
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
@@ -161,19 +150,38 @@ export class BuilderSelectHelper {
|
||||
* Useful for asserting "Widget not visible" on disconnected inputs.
|
||||
*/
|
||||
getInputItemSubtitle(title: string): Locator {
|
||||
return this.inputItems
|
||||
return this.page
|
||||
.getByTestId(TestIds.builder.ioItem)
|
||||
.filter({
|
||||
has: this.inputItemTitles.getByText(title, { exact: true })
|
||||
has: this.page
|
||||
.getByTestId(TestIds.builder.ioItemTitle)
|
||||
.getByText(title, { exact: true })
|
||||
})
|
||||
.getByTestId(TestIds.builder.ioItemSubtitle)
|
||||
}
|
||||
|
||||
/** All IoItem locators in the current step sidebar. */
|
||||
get inputItems(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.ioItem)
|
||||
}
|
||||
|
||||
/** All IoItem title locators in the inputs step sidebar. */
|
||||
get inputItemTitles(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.ioItemTitle)
|
||||
}
|
||||
|
||||
/** All widget label locators in the preview/arrange sidebar. */
|
||||
get previewWidgetLabels(): Locator {
|
||||
return this.page.getByTestId(TestIds.builder.widgetLabel)
|
||||
}
|
||||
|
||||
/**
|
||||
* Drag an IoItem from one index to another in the inputs step.
|
||||
* Items are identified by their 0-based position among visible IoItems.
|
||||
*/
|
||||
async dragInputItem(fromIndex: number, toIndex: number) {
|
||||
await dragByIndex(this.inputItems, fromIndex, toIndex)
|
||||
const items = this.page.getByTestId(TestIds.builder.ioItem)
|
||||
await dragByIndex(items, fromIndex, toIndex)
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
@@ -200,7 +208,6 @@ export class BuilderSelectHelper {
|
||||
const nodeLocator = this.comfyPage.vueNodes.getNodeLocator(
|
||||
String(nodeRef.id)
|
||||
)
|
||||
// oxlint-disable-next-line playwright/no-force-option -- Node container has conditional pointer-events:none that blocks actionability
|
||||
await nodeLocator.click({ force: true })
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
@@ -3,16 +3,16 @@ import type { Locator, Page } from '@playwright/test'
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
export class BuilderStepsHelper {
|
||||
public readonly toolbar: Locator
|
||||
|
||||
constructor(private readonly comfyPage: ComfyPage) {
|
||||
this.toolbar = this.page.getByRole('navigation', { name: 'App Builder' })
|
||||
}
|
||||
constructor(private readonly comfyPage: ComfyPage) {}
|
||||
|
||||
private get page(): Page {
|
||||
return this.comfyPage.page
|
||||
}
|
||||
|
||||
get toolbar(): Locator {
|
||||
return this.page.getByRole('navigation', { name: 'App Builder' })
|
||||
}
|
||||
|
||||
async goToInputs() {
|
||||
await this.toolbar.getByRole('button', { name: 'Inputs' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
|
||||
@@ -74,51 +74,6 @@ export class CanvasHelper {
|
||||
await this.nextFrame()
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a canvas-element-relative position to absolute page coordinates.
|
||||
* Use with `page.mouse` APIs when Vue DOM overlays above the canvas would
|
||||
* cause Playwright's actionability check to fail on the canvas locator.
|
||||
*/
|
||||
private async toAbsolute(position: Position): Promise<Position> {
|
||||
const box = await this.canvas.boundingBox()
|
||||
if (!box) throw new Error('Canvas bounding box not available')
|
||||
return { x: box.x + position.x, y: box.y + position.y }
|
||||
}
|
||||
|
||||
/**
|
||||
* Click at canvas-element-relative coordinates using `page.mouse.click()`.
|
||||
* Bypasses Playwright's actionability checks on the canvas locator, which
|
||||
* can fail when Vue-rendered DOM nodes overlay the `<canvas>` element.
|
||||
*/
|
||||
async mouseClickAt(
|
||||
position: Position,
|
||||
options?: {
|
||||
button?: 'left' | 'right' | 'middle'
|
||||
modifiers?: ('Shift' | 'Control' | 'Alt' | 'Meta')[]
|
||||
}
|
||||
): Promise<void> {
|
||||
const abs = await this.toAbsolute(position)
|
||||
const modifiers = options?.modifiers ?? []
|
||||
for (const mod of modifiers) await this.page.keyboard.down(mod)
|
||||
try {
|
||||
await this.page.mouse.click(abs.x, abs.y, {
|
||||
button: options?.button
|
||||
})
|
||||
} finally {
|
||||
for (const mod of modifiers) await this.page.keyboard.up(mod)
|
||||
}
|
||||
await this.nextFrame()
|
||||
}
|
||||
|
||||
/**
|
||||
* Double-click at canvas-element-relative coordinates using `page.mouse`.
|
||||
*/
|
||||
async mouseDblclickAt(position: Position): Promise<void> {
|
||||
const abs = await this.toAbsolute(position)
|
||||
await this.page.mouse.dblclick(abs.x, abs.y)
|
||||
await this.nextFrame()
|
||||
}
|
||||
|
||||
async clickEmptySpace(): Promise<void> {
|
||||
await this.canvas.click({ position: DefaultGraphPositions.emptySpaceClick })
|
||||
await this.nextFrame()
|
||||
|
||||
@@ -1,211 +0,0 @@
|
||||
import type { WebSocketRoute } from '@playwright/test'
|
||||
|
||||
import type { RawJobListItem } from '@/platform/remote/comfyui/jobs/jobTypes'
|
||||
import type { ComfyPage } from '../ComfyPage'
|
||||
import { createMockJob } from './AssetsHelper'
|
||||
|
||||
/**
|
||||
* Helper for simulating prompt execution in e2e tests.
|
||||
*/
|
||||
export class ExecutionHelper {
|
||||
private jobCounter = 0
|
||||
private readonly completedJobs: RawJobListItem[] = []
|
||||
private readonly page: ComfyPage['page']
|
||||
private readonly command: ComfyPage['command']
|
||||
private readonly assets: ComfyPage['assets']
|
||||
|
||||
constructor(
|
||||
comfyPage: ComfyPage,
|
||||
private readonly ws: WebSocketRoute
|
||||
) {
|
||||
this.page = comfyPage.page
|
||||
this.command = comfyPage.command
|
||||
this.assets = comfyPage.assets
|
||||
}
|
||||
|
||||
/**
|
||||
* Intercept POST /api/prompt, execute Comfy.QueuePrompt, and return
|
||||
* the synthetic job ID.
|
||||
*
|
||||
* The app receives a valid PromptResponse so storeJob() fires
|
||||
* and registers the job against the active workflow path.
|
||||
*/
|
||||
async run(): Promise<string> {
|
||||
const jobId = `test-job-${++this.jobCounter}`
|
||||
|
||||
let fulfilled!: () => void
|
||||
const prompted = new Promise<void>((r) => {
|
||||
fulfilled = r
|
||||
})
|
||||
|
||||
await this.page.route(
|
||||
'**/api/prompt',
|
||||
async (route) => {
|
||||
await route.fulfill({
|
||||
status: 200,
|
||||
contentType: 'application/json',
|
||||
body: JSON.stringify({
|
||||
prompt_id: jobId,
|
||||
node_errors: {}
|
||||
})
|
||||
})
|
||||
fulfilled()
|
||||
},
|
||||
{ times: 1 }
|
||||
)
|
||||
|
||||
await this.command.executeCommand('Comfy.QueuePrompt')
|
||||
await prompted
|
||||
|
||||
return jobId
|
||||
}
|
||||
|
||||
/**
|
||||
* Send a binary `b_preview_with_metadata` WS message (type 4).
|
||||
* Encodes the metadata and a tiny 1x1 PNG so the app creates a blob URL.
|
||||
*/
|
||||
latentPreview(jobId: string, nodeId: string): void {
|
||||
const metadata = JSON.stringify({
|
||||
node_id: nodeId,
|
||||
display_node_id: nodeId,
|
||||
parent_node_id: nodeId,
|
||||
real_node_id: nodeId,
|
||||
prompt_id: jobId,
|
||||
image_type: 'image/png'
|
||||
})
|
||||
const metadataBytes = new TextEncoder().encode(metadata)
|
||||
|
||||
// 1x1 red PNG
|
||||
const png = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwADhQGAWjR9awAAAABJRU5ErkJggg==',
|
||||
'base64'
|
||||
)
|
||||
|
||||
// Binary format: [type:uint32][metadataLength:uint32][metadata][imageData]
|
||||
const buf = new ArrayBuffer(8 + metadataBytes.length + png.length)
|
||||
const view = new DataView(buf)
|
||||
view.setUint32(0, 4) // type 4 = PREVIEW_IMAGE_WITH_METADATA
|
||||
view.setUint32(4, metadataBytes.length)
|
||||
new Uint8Array(buf, 8, metadataBytes.length).set(metadataBytes)
|
||||
new Uint8Array(buf, 8 + metadataBytes.length).set(png)
|
||||
|
||||
this.ws.send(Buffer.from(buf))
|
||||
}
|
||||
|
||||
/** Send `execution_start` WS event. */
|
||||
executionStart(jobId: string): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'execution_start',
|
||||
data: { prompt_id: jobId, timestamp: Date.now() }
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/** Send `executing` WS event to signal which node is currently running. */
|
||||
executing(jobId: string, nodeId: string | null): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'executing',
|
||||
data: { prompt_id: jobId, node: nodeId }
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/** Send `executed` WS event with node output. */
|
||||
executed(
|
||||
jobId: string,
|
||||
nodeId: string,
|
||||
output: Record<string, unknown>
|
||||
): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'executed',
|
||||
data: {
|
||||
prompt_id: jobId,
|
||||
node: nodeId,
|
||||
display_node: nodeId,
|
||||
output
|
||||
}
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/** Send `execution_success` WS event. */
|
||||
executionSuccess(jobId: string): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'execution_success',
|
||||
data: { prompt_id: jobId, timestamp: Date.now() }
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/** Send `execution_error` WS event. */
|
||||
executionError(jobId: string, nodeId: string, message: string): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'execution_error',
|
||||
data: {
|
||||
prompt_id: jobId,
|
||||
timestamp: Date.now(),
|
||||
node_id: nodeId,
|
||||
node_type: 'Unknown',
|
||||
exception_message: message,
|
||||
exception_type: 'RuntimeError',
|
||||
traceback: []
|
||||
}
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/** Send `progress` WS event. */
|
||||
progress(jobId: string, nodeId: string, value: number, max: number): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'progress',
|
||||
data: { prompt_id: jobId, node: nodeId, value, max }
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete a job by adding it to mock history, sending execution_success,
|
||||
* and triggering a history refresh via a status event.
|
||||
*
|
||||
* Requires an {@link AssetsHelper} to be passed in the constructor.
|
||||
*/
|
||||
async completeWithHistory(
|
||||
jobId: string,
|
||||
nodeId: string,
|
||||
filename: string
|
||||
): Promise<void> {
|
||||
this.completedJobs.push(
|
||||
createMockJob({
|
||||
id: jobId,
|
||||
preview_output: {
|
||||
filename,
|
||||
subfolder: '',
|
||||
type: 'output',
|
||||
nodeId,
|
||||
mediaType: 'images'
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
await this.assets.mockOutputHistory(this.completedJobs)
|
||||
this.executionSuccess(jobId)
|
||||
// Trigger queue/history refresh
|
||||
this.status(0)
|
||||
}
|
||||
|
||||
/** Send `status` WS event to update queue count. */
|
||||
status(queueRemaining: number): void {
|
||||
this.ws.send(
|
||||
JSON.stringify({
|
||||
type: 'status',
|
||||
data: { status: { exec_info: { queue_remaining: queueRemaining } } }
|
||||
})
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -8,13 +8,7 @@ import type { Position, Size } from '@e2e/fixtures/types'
|
||||
import { NodeReference } from '@e2e/fixtures/utils/litegraphUtils'
|
||||
|
||||
export class NodeOperationsHelper {
|
||||
public readonly promptDialogInput: Locator
|
||||
|
||||
constructor(private comfyPage: ComfyPage) {
|
||||
this.promptDialogInput = this.page.locator(
|
||||
'.p-dialog-content input[type="text"]'
|
||||
)
|
||||
}
|
||||
constructor(private comfyPage: ComfyPage) {}
|
||||
|
||||
private get page() {
|
||||
return this.comfyPage.page
|
||||
@@ -161,6 +155,10 @@ export class NodeOperationsHelper {
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
get promptDialogInput(): Locator {
|
||||
return this.page.locator('.p-dialog-content input[type="text"]')
|
||||
}
|
||||
|
||||
async fillPromptDialog(value: string): Promise<void> {
|
||||
await this.promptDialogInput.fill(value)
|
||||
await this.page.keyboard.press('Enter')
|
||||
|
||||
@@ -157,7 +157,7 @@ export class SubgraphHelper {
|
||||
|
||||
// Wait for the appropriate UI element to appear
|
||||
if (action === 'rightClick') {
|
||||
await this.page.locator('.litemenu-entry').first().waitFor({
|
||||
await this.page.waitForSelector('.litemenu-entry', {
|
||||
state: 'visible',
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
@@ -2,12 +2,14 @@ import { expect } from '@playwright/test'
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
export class ToastHelper {
|
||||
public readonly visibleToasts: Locator
|
||||
public readonly toastErrors: Locator
|
||||
constructor(private readonly page: Page) {}
|
||||
|
||||
constructor(private readonly page: Page) {
|
||||
this.visibleToasts = page.locator('.p-toast-message:visible')
|
||||
this.toastErrors = page.locator('.p-toast-message.p-toast-message-error')
|
||||
get visibleToasts(): Locator {
|
||||
return this.page.locator('.p-toast-message:visible')
|
||||
}
|
||||
|
||||
get toastErrors(): Locator {
|
||||
return this.page.locator('.p-toast-message.p-toast-message-error')
|
||||
}
|
||||
|
||||
async closeToasts(requireCount = 0): Promise<void> {
|
||||
@@ -26,6 +28,6 @@ export class ToastHelper {
|
||||
}
|
||||
|
||||
// Assert all toasts are closed
|
||||
await expect(this.visibleToasts).toHaveCount(0)
|
||||
await expect(this.visibleToasts).toHaveCount(0, { timeout: 1000 })
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,10 +21,6 @@ export const TestIds = {
|
||||
contextMenu: 'canvas-context-menu',
|
||||
toggleMinimapButton: 'toggle-minimap-button',
|
||||
closeMinimapButton: 'close-minimap-button',
|
||||
minimapContainer: 'minimap-container',
|
||||
minimapCanvas: 'minimap-canvas',
|
||||
minimapViewport: 'minimap-viewport',
|
||||
minimapInteractionOverlay: 'minimap-interaction-overlay',
|
||||
toggleLinkVisibilityButton: 'toggle-link-visibility-button',
|
||||
zoomControlsButton: 'zoom-controls-button',
|
||||
zoomInAction: 'zoom-in-action',
|
||||
@@ -134,24 +130,6 @@ export const TestIds = {
|
||||
outputPlaceholder: 'builder-output-placeholder',
|
||||
connectOutputPopover: 'builder-connect-output-popover'
|
||||
},
|
||||
outputHistory: {
|
||||
outputs: 'linear-outputs',
|
||||
welcome: 'linear-welcome',
|
||||
outputInfo: 'linear-output-info',
|
||||
activeQueue: 'linear-job',
|
||||
queueBadge: 'linear-job-badge',
|
||||
inProgressItem: 'linear-in-progress-item',
|
||||
historyItem: 'linear-history-item',
|
||||
skeleton: 'linear-skeleton',
|
||||
latentPreview: 'linear-latent-preview',
|
||||
imageOutput: 'linear-image-output',
|
||||
videoOutput: 'linear-video-output',
|
||||
cancelRun: 'linear-cancel-run',
|
||||
headerProgressBar: 'linear-header-progress-bar',
|
||||
itemProgressBar: 'linear-item-progress-bar',
|
||||
progressOverall: 'linear-progress-overall',
|
||||
progressNode: 'linear-progress-node'
|
||||
},
|
||||
appMode: {
|
||||
widgetItem: 'app-mode-widget-item',
|
||||
welcome: 'linear-welcome',
|
||||
@@ -202,7 +180,6 @@ export type TestIdValue =
|
||||
| (typeof TestIds.selectionToolbox)[keyof typeof TestIds.selectionToolbox]
|
||||
| (typeof TestIds.widgets)[keyof typeof TestIds.widgets]
|
||||
| (typeof TestIds.builder)[keyof typeof TestIds.builder]
|
||||
| (typeof TestIds.outputHistory)[keyof typeof TestIds.outputHistory]
|
||||
| (typeof TestIds.appMode)[keyof typeof TestIds.appMode]
|
||||
| (typeof TestIds.breadcrumb)[keyof typeof TestIds.breadcrumb]
|
||||
| Exclude<
|
||||
|
||||
@@ -14,12 +14,11 @@ function makeMatcher<T>(
|
||||
) {
|
||||
await expect(async () => {
|
||||
const value = await getValue(node)
|
||||
if (this.isNot) {
|
||||
expect(value, 'Node is ' + type).not.toBeTruthy()
|
||||
} else {
|
||||
expect(value, 'Node is not ' + type).toBeTruthy()
|
||||
}
|
||||
}).toPass({ timeout: 5000, ...options })
|
||||
const assertion = this.isNot
|
||||
? expect(value, 'Node is ' + type).not
|
||||
: expect(value, 'Node is not ' + type)
|
||||
assertion.toBeTruthy()
|
||||
}).toPass({ timeout: 250, ...options })
|
||||
return {
|
||||
pass: !this.isNot,
|
||||
message: () => 'Node is ' + (this.isNot ? 'not ' : '') + type
|
||||
@@ -31,7 +30,7 @@ export const comfyExpect = expect.extend({
|
||||
toBePinned: makeMatcher((n) => n.isPinned(), 'pinned'),
|
||||
toBeBypassed: makeMatcher((n) => n.isBypassed(), 'bypassed'),
|
||||
toBeCollapsed: makeMatcher((n) => n.isCollapsed(), 'collapsed'),
|
||||
async toHaveFocus(locator: Locator, options = {}) {
|
||||
async toHaveFocus(locator: Locator, options = { timeout: 256 }) {
|
||||
await expect
|
||||
.poll(
|
||||
() => locator.evaluate((el) => el === document.activeElement),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { expect } from '@playwright/test'
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import type { NodeId } from '@/platform/workflow/validation/schemas/workflowSchema'
|
||||
import { ManageGroupNode } from '@e2e/helpers/manageGroupNode'
|
||||
@@ -355,11 +356,7 @@ export class NodeReference {
|
||||
}
|
||||
async click(
|
||||
position: 'title' | 'collapse',
|
||||
options?: {
|
||||
button?: 'left' | 'right' | 'middle'
|
||||
modifiers?: ('Shift' | 'Control' | 'Alt' | 'Meta')[]
|
||||
moveMouseToEmptyArea?: boolean
|
||||
}
|
||||
options?: Parameters<Page['click']>[1] & { moveMouseToEmptyArea?: boolean }
|
||||
) {
|
||||
let clickPos: Position
|
||||
switch (position) {
|
||||
@@ -380,7 +377,12 @@ export class NodeReference {
|
||||
delete options.moveMouseToEmptyArea
|
||||
}
|
||||
|
||||
await this.comfyPage.canvasOps.mouseClickAt(clickPos, options)
|
||||
await this.comfyPage.canvas.click({
|
||||
...options,
|
||||
position: clickPos,
|
||||
force: true
|
||||
})
|
||||
await this.comfyPage.nextFrame()
|
||||
if (moveMouseToEmptyArea) {
|
||||
await this.comfyPage.canvasOps.moveMouseToEmptyArea()
|
||||
}
|
||||
@@ -497,18 +499,31 @@ export class NodeReference {
|
||||
|
||||
await expect(async () => {
|
||||
// Try just clicking the enter button first
|
||||
await this.comfyPage.canvasOps.mouseClickAt({ x: 250, y: 250 })
|
||||
await this.comfyPage.canvas.click({
|
||||
position: { x: 250, y: 250 },
|
||||
force: true
|
||||
})
|
||||
await this.comfyPage.nextFrame()
|
||||
|
||||
await this.comfyPage.canvasOps.mouseClickAt(subgraphButtonPos)
|
||||
await this.comfyPage.canvas.click({
|
||||
position: subgraphButtonPos,
|
||||
force: true
|
||||
})
|
||||
await this.comfyPage.nextFrame()
|
||||
|
||||
if (await checkIsInSubgraph()) return
|
||||
|
||||
for (const position of clickPositions) {
|
||||
// Clear any selection first
|
||||
await this.comfyPage.canvasOps.mouseClickAt({ x: 250, y: 250 })
|
||||
await this.comfyPage.canvas.click({
|
||||
position: { x: 250, y: 250 },
|
||||
force: true
|
||||
})
|
||||
await this.comfyPage.nextFrame()
|
||||
|
||||
// Double-click to enter subgraph
|
||||
await this.comfyPage.canvasOps.mouseDblclickAt(position)
|
||||
await this.comfyPage.canvas.dblclick({ position, force: true })
|
||||
await this.comfyPage.nextFrame()
|
||||
|
||||
if (await checkIsInSubgraph()) return
|
||||
}
|
||||
|
||||
@@ -4,24 +4,38 @@ import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
/** DOM-centric helper for a single Vue-rendered node on the canvas. */
|
||||
export class VueNodeFixture {
|
||||
public readonly header: Locator
|
||||
public readonly title: Locator
|
||||
public readonly titleInput: Locator
|
||||
public readonly body: Locator
|
||||
public readonly pinIndicator: Locator
|
||||
public readonly collapseButton: Locator
|
||||
public readonly collapseIcon: Locator
|
||||
public readonly root: Locator
|
||||
constructor(private readonly locator: Locator) {}
|
||||
|
||||
constructor(private readonly locator: Locator) {
|
||||
this.header = locator.locator('[data-testid^="node-header-"]')
|
||||
this.title = locator.getByTestId('node-title')
|
||||
this.titleInput = locator.getByTestId('node-title-input')
|
||||
this.body = locator.locator('[data-testid^="node-body-"]')
|
||||
this.pinIndicator = locator.getByTestId(TestIds.node.pinIndicator)
|
||||
this.collapseButton = locator.getByTestId('node-collapse-button')
|
||||
this.collapseIcon = this.collapseButton.locator('i')
|
||||
this.root = locator
|
||||
get header(): Locator {
|
||||
return this.locator.locator('[data-testid^="node-header-"]')
|
||||
}
|
||||
|
||||
get title(): Locator {
|
||||
return this.locator.locator('[data-testid="node-title"]')
|
||||
}
|
||||
|
||||
get titleInput(): Locator {
|
||||
return this.locator.locator('[data-testid="node-title-input"]')
|
||||
}
|
||||
|
||||
get body(): Locator {
|
||||
return this.locator.locator('[data-testid^="node-body-"]')
|
||||
}
|
||||
|
||||
get pinIndicator(): Locator {
|
||||
return this.locator.getByTestId(TestIds.node.pinIndicator)
|
||||
}
|
||||
|
||||
get collapseButton(): Locator {
|
||||
return this.locator.locator('[data-testid="node-collapse-button"]')
|
||||
}
|
||||
|
||||
get collapseIcon(): Locator {
|
||||
return this.collapseButton.locator('i')
|
||||
}
|
||||
|
||||
get root(): Locator {
|
||||
return this.locator
|
||||
}
|
||||
|
||||
async getTitle(): Promise<string> {
|
||||
|
||||
@@ -1,31 +1,53 @@
|
||||
import { test as base } from '@playwright/test'
|
||||
import type { WebSocketRoute } from '@playwright/test'
|
||||
|
||||
export const webSocketFixture = base.extend<{
|
||||
getWebSocket: () => Promise<WebSocketRoute>
|
||||
ws: { trigger(data: unknown, url?: string): Promise<void> }
|
||||
}>({
|
||||
getWebSocket: [
|
||||
async ({ context }, use) => {
|
||||
let latest: WebSocketRoute | undefined
|
||||
let resolve: ((ws: WebSocketRoute) => void) | undefined
|
||||
|
||||
await context.routeWebSocket(/\/ws/, (ws) => {
|
||||
const server = ws.connectToServer()
|
||||
server.onMessage((message) => {
|
||||
ws.send(message)
|
||||
ws: [
|
||||
async ({ page }, use) => {
|
||||
// Each time a page loads, to catch navigations
|
||||
page.on('load', async () => {
|
||||
await page.evaluate(function () {
|
||||
// Create a wrapper for WebSocket that stores them globally
|
||||
// so we can look it up to trigger messages
|
||||
const store: Record<string, WebSocket> = (window.__ws__ = {})
|
||||
window.WebSocket = class extends window.WebSocket {
|
||||
constructor(
|
||||
...rest: ConstructorParameters<typeof window.WebSocket>
|
||||
) {
|
||||
super(...rest)
|
||||
store[this.url] = this
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
latest = ws
|
||||
resolve?.(ws)
|
||||
})
|
||||
|
||||
await use(() => {
|
||||
if (latest) return Promise.resolve(latest)
|
||||
return new Promise<WebSocketRoute>((r) => {
|
||||
resolve = r
|
||||
})
|
||||
await use({
|
||||
async trigger(data, url) {
|
||||
// Trigger a websocket event on the page
|
||||
await page.evaluate(
|
||||
function ([data, url]) {
|
||||
if (!url) {
|
||||
// If no URL specified, use page URL
|
||||
const u = new URL(window.location.href)
|
||||
u.hash = ''
|
||||
u.protocol = 'ws:'
|
||||
u.pathname = '/'
|
||||
url = u.toString() + 'ws'
|
||||
}
|
||||
const ws: WebSocket = window.__ws__![url]
|
||||
ws.dispatchEvent(
|
||||
new MessageEvent('message', {
|
||||
data
|
||||
})
|
||||
)
|
||||
},
|
||||
[JSON.stringify(data), url]
|
||||
)
|
||||
}
|
||||
})
|
||||
},
|
||||
// We need this to run automatically as the first thing so it adds handlers as soon as the page loads
|
||||
{ auto: true }
|
||||
]
|
||||
})
|
||||
|
||||
@@ -82,9 +82,11 @@ export async function builderSaveAs(
|
||||
viewType: 'App' | 'Node graph' = 'App'
|
||||
) {
|
||||
await appMode.footer.saveAsButton.click()
|
||||
await comfyExpect(appMode.saveAs.nameInput).toBeVisible()
|
||||
await comfyExpect(appMode.saveAs.nameInput).toBeVisible({ timeout: 5000 })
|
||||
await appMode.saveAs.fillAndSave(workflowName, viewType)
|
||||
await comfyExpect(appMode.saveAs.successMessage).toBeVisible()
|
||||
await comfyExpect(appMode.saveAs.successMessage).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -122,21 +124,3 @@ export async function saveAndReopenInAppMode(
|
||||
|
||||
await comfyPage.appMode.toggleAppMode()
|
||||
}
|
||||
|
||||
/**
|
||||
* Enter builder, select the given widgets as inputs + SaveImage as output,
|
||||
* save as an app, and close the success dialog.
|
||||
*
|
||||
* Returns on the builder arrange/preview step.
|
||||
*/
|
||||
export async function createAndSaveApp(
|
||||
comfyPage: ComfyPage,
|
||||
appName: string,
|
||||
widgetNames: string[] = ['seed']
|
||||
): Promise<void> {
|
||||
await setupBuilder(comfyPage, undefined, widgetNames)
|
||||
await comfyPage.appMode.steps.goToPreview()
|
||||
await builderSaveAs(comfyPage.appMode, appName)
|
||||
await comfyPage.appMode.saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
import type { TestGraphAccess } from '@e2e/types/globals'
|
||||
|
||||
export async function drawStroke(
|
||||
page: Page,
|
||||
canvas: Locator,
|
||||
opts: { startXPct?: number; endXPct?: number; yPct?: number } = {}
|
||||
): Promise<void> {
|
||||
const { startXPct = 0.3, endXPct = 0.7, yPct = 0.5 } = opts
|
||||
const box = await canvas.boundingBox()
|
||||
if (!box) throw new Error('Canvas bounding box not found')
|
||||
await page.mouse.move(
|
||||
box.x + box.width * startXPct,
|
||||
box.y + box.height * yPct
|
||||
)
|
||||
await page.mouse.down()
|
||||
await page.mouse.move(
|
||||
box.x + box.width * endXPct,
|
||||
box.y + box.height * yPct,
|
||||
{ steps: 10 }
|
||||
)
|
||||
await page.mouse.up()
|
||||
}
|
||||
|
||||
export async function hasCanvasContent(canvas: Locator): Promise<boolean> {
|
||||
return canvas.evaluate((el: HTMLCanvasElement) => {
|
||||
const ctx = el.getContext('2d')
|
||||
if (!ctx) return false
|
||||
const { data } = ctx.getImageData(0, 0, el.width, el.height)
|
||||
for (let i = 3; i < data.length; i += 4) {
|
||||
if (data[i] > 0) return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
}
|
||||
|
||||
export async function triggerSerialization(page: Page): Promise<void> {
|
||||
await page.evaluate(async () => {
|
||||
const graph = window.graph as TestGraphAccess | undefined
|
||||
if (!graph) {
|
||||
throw new Error(
|
||||
'Global window.graph is absent. Ensure workflow fixture is loaded.'
|
||||
)
|
||||
}
|
||||
|
||||
const node = graph._nodes_by_id?.['1']
|
||||
if (!node) {
|
||||
throw new Error(
|
||||
'Target node with ID "1" not found in graph._nodes_by_id.'
|
||||
)
|
||||
}
|
||||
|
||||
const widget = node.widgets?.find((w) => w.name === 'mask')
|
||||
if (!widget) {
|
||||
throw new Error('Widget "mask" not found on target node 1.')
|
||||
}
|
||||
|
||||
if (typeof widget.serializeValue !== 'function') {
|
||||
throw new Error(
|
||||
'mask widget on node 1 does not have a serializeValue function.'
|
||||
)
|
||||
}
|
||||
|
||||
await widget.serializeValue(node, 0)
|
||||
})
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { Response } from '@playwright/test'
|
||||
import { expect, mergeTests } from '@playwright/test'
|
||||
|
||||
import type { StatusWsMessage } from '@/schemas/apiSchema'
|
||||
import { comfyPageFixture } from '@e2e/fixtures/ComfyPage'
|
||||
import { webSocketFixture } from '@e2e/fixtures/ws'
|
||||
import type { WorkspaceStore } from '@e2e/types/globals'
|
||||
@@ -17,10 +18,8 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
|
||||
*/
|
||||
test('Does not auto-queue multiple changes at a time', async ({
|
||||
comfyPage,
|
||||
getWebSocket
|
||||
ws
|
||||
}) => {
|
||||
const ws = await getWebSocket()
|
||||
|
||||
// Enable change auto-queue mode
|
||||
const queueOpts = await comfyPage.actionbar.queueButton.toggleOptions()
|
||||
await expect.poll(() => queueOpts.getMode()).toBe('disabled')
|
||||
@@ -63,19 +62,17 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
|
||||
}
|
||||
|
||||
// Trigger a status websocket message
|
||||
const triggerStatus = (queueSize: number) => {
|
||||
ws.send(
|
||||
JSON.stringify({
|
||||
type: 'status',
|
||||
data: {
|
||||
status: {
|
||||
exec_info: {
|
||||
queue_remaining: queueSize
|
||||
}
|
||||
const triggerStatus = async (queueSize: number) => {
|
||||
await ws.trigger({
|
||||
type: 'status',
|
||||
data: {
|
||||
status: {
|
||||
exec_info: {
|
||||
queue_remaining: queueSize
|
||||
}
|
||||
}
|
||||
})
|
||||
)
|
||||
}
|
||||
} as StatusWsMessage)
|
||||
}
|
||||
|
||||
// Extract the width from the queue response
|
||||
@@ -107,8 +104,8 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
|
||||
).toBe(1)
|
||||
|
||||
// Trigger a status update so auto-queue re-runs
|
||||
triggerStatus(1)
|
||||
triggerStatus(0)
|
||||
await triggerStatus(1)
|
||||
await triggerStatus(0)
|
||||
|
||||
// Ensure the queued width is the last queued value
|
||||
expect(
|
||||
|
||||
@@ -75,7 +75,9 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
]
|
||||
await comfyPage.appMode.enterAppModeWithInputs(inputs)
|
||||
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Scroll to bottom so the codec widget is at the clipping edge
|
||||
const widgetList = comfyPage.appMode.linearWidgets
|
||||
@@ -88,7 +90,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
await codecSelect.click()
|
||||
|
||||
const overlay = comfyPage.page.locator('.p-select-overlay').first()
|
||||
await expect(overlay).toBeVisible()
|
||||
await expect(overlay).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
@@ -121,7 +123,9 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
]
|
||||
await comfyPage.appMode.enterAppModeWithInputs(inputs)
|
||||
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Scroll to bottom so the image widget is at the clipping edge
|
||||
const widgetList = comfyPage.appMode.linearWidgets
|
||||
@@ -140,7 +144,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
// The unstyled PrimeVue Popover renders with role="dialog".
|
||||
// Locate the one containing the image grid (filter buttons like "All", "Inputs").
|
||||
const popover = comfyPage.appMode.imagePickerPopover
|
||||
await expect(popover).toBeVisible()
|
||||
await expect(popover).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
|
||||
@@ -16,7 +16,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
|
||||
|
||||
await expect(comfyPage.appMode.welcome).toBeVisible()
|
||||
await expect(comfyPage.appMode.emptyWorkflowText).toBeVisible()
|
||||
await expect(comfyPage.appMode.buildAppButton).toBeHidden()
|
||||
await expect(comfyPage.appMode.buildAppButton).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Build app button is visible when no outputs selected', async ({
|
||||
@@ -26,7 +26,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
|
||||
|
||||
await expect(comfyPage.appMode.welcome).toBeVisible()
|
||||
await expect(comfyPage.appMode.buildAppButton).toBeVisible()
|
||||
await expect(comfyPage.appMode.emptyWorkflowText).toBeHidden()
|
||||
await expect(comfyPage.appMode.emptyWorkflowText).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Empty workflow and build app are hidden when app has outputs', async ({
|
||||
@@ -35,8 +35,8 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([['3', 'seed']])
|
||||
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
await expect(comfyPage.appMode.emptyWorkflowText).toBeHidden()
|
||||
await expect(comfyPage.appMode.buildAppButton).toBeHidden()
|
||||
await expect(comfyPage.appMode.emptyWorkflowText).not.toBeVisible()
|
||||
await expect(comfyPage.appMode.buildAppButton).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Back to workflow returns to graph mode', async ({ comfyPage }) => {
|
||||
@@ -46,7 +46,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
|
||||
await comfyPage.appMode.backToWorkflowButton.click()
|
||||
|
||||
await expect(comfyPage.canvas).toBeVisible()
|
||||
await expect(comfyPage.appMode.welcome).toBeHidden()
|
||||
await expect(comfyPage.appMode.welcome).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Load template opens template selector', async ({ comfyPage }) => {
|
||||
|
||||
@@ -26,7 +26,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
await appMode.steps.goToInputs()
|
||||
|
||||
const menu = appMode.select.getInputItemMenu('seed')
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(menu).toBeVisible({ timeout: 5000 })
|
||||
await appMode.select.renameInputViaMenu('seed', 'Builder Input Seed')
|
||||
|
||||
// Verify in app mode after save/reload
|
||||
@@ -34,7 +34,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
const workflowName = `${new Date().getTime()} builder-input-menu`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(
|
||||
appMode.linearWidgets.getByText('Builder Input Seed')
|
||||
).toBeVisible()
|
||||
@@ -54,7 +54,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
const workflowName = `${new Date().getTime()} builder-input-dblclick`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('Dblclick Seed')).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -65,7 +65,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
await appMode.steps.goToPreview()
|
||||
|
||||
const menu = appMode.select.getPreviewWidgetMenu('seed — New Subgraph')
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(menu).toBeVisible({ timeout: 5000 })
|
||||
await appMode.select.renameWidget(menu, 'Preview Seed')
|
||||
|
||||
// Verify in app mode after save/reload
|
||||
@@ -73,7 +73,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
const workflowName = `${new Date().getTime()} builder-preview`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('Preview Seed')).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -85,7 +85,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
await appMode.footer.exitBuilder()
|
||||
await appMode.toggleAppMode()
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
|
||||
const menu = appMode.getAppModeWidgetMenu('seed')
|
||||
await appMode.select.renameWidget(menu, 'App Mode Seed')
|
||||
@@ -97,7 +97,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
const workflowName = `${new Date().getTime()} app-mode`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('App Mode Seed')).toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -63,7 +63,7 @@ test.describe('App mode widget values in prompt', { tag: '@ui' }, () => {
|
||||
({ nodeId, widgetName }) => [nodeId, widgetName]
|
||||
)
|
||||
await appMode.enterAppModeWithInputs(inputs)
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
|
||||
for (const { nodeId, widgetName, type, fill } of WIDGET_TEST_DATA) {
|
||||
const key = `${nodeId}:${widgetName}`
|
||||
|
||||
@@ -11,7 +11,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
|
||||
test('should open bottom panel via toggle button', async ({ comfyPage }) => {
|
||||
const { bottomPanel } = comfyPage
|
||||
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
await bottomPanel.toggleButton.click()
|
||||
await expect(bottomPanel.root).toBeVisible()
|
||||
})
|
||||
@@ -35,7 +35,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
|
||||
await expect(bottomPanel.root).toBeVisible()
|
||||
|
||||
await bottomPanel.toggleButton.click()
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('should switch between shortcuts and terminal panels', async ({
|
||||
@@ -55,7 +55,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
|
||||
await expect(logsTab).toBeVisible()
|
||||
await expect(
|
||||
comfyPage.page.locator('[id*="tab_shortcuts-essentials"]')
|
||||
).toBeHidden()
|
||||
).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('should persist Logs tab content in bottom panel', async ({
|
||||
|
||||
@@ -10,11 +10,11 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
test('should toggle shortcuts panel visibility', async ({ comfyPage }) => {
|
||||
const { bottomPanel } = comfyPage
|
||||
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
await bottomPanel.keyboardShortcutsButton.click()
|
||||
await expect(bottomPanel.root).toBeVisible()
|
||||
await bottomPanel.keyboardShortcutsButton.click()
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('should display essentials shortcuts tab', async ({ comfyPage }) => {
|
||||
@@ -182,7 +182,7 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
await expect(bottomPanel.root).toBeVisible()
|
||||
|
||||
await bottomPanel.keyboardShortcutsButton.click()
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('should display shortcuts in organized columns', async ({
|
||||
@@ -192,7 +192,9 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
|
||||
await bottomPanel.keyboardShortcutsButton.click()
|
||||
|
||||
await expect(comfyPage.page.getByTestId('shortcuts-columns')).toBeVisible()
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="shortcuts-columns"]')
|
||||
).toBeVisible()
|
||||
|
||||
const subcategoryTitles = bottomPanel.shortcuts.subcategoryTitles
|
||||
await expect.poll(() => subcategoryTitles.count()).toBeGreaterThanOrEqual(2)
|
||||
@@ -203,7 +205,7 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
}) => {
|
||||
const { bottomPanel } = comfyPage
|
||||
|
||||
await expect(bottomPanel.root).toBeHidden()
|
||||
await expect(bottomPanel.root).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Shift+KeyK')
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import type { AppModeHelper } from '@e2e/fixtures/helpers/AppModeHelper'
|
||||
import {
|
||||
builderSaveAs,
|
||||
createAndSaveApp,
|
||||
openWorkflowFromSidebar,
|
||||
setupBuilder
|
||||
} from '@e2e/helpers/builderTestUtils'
|
||||
@@ -22,7 +21,7 @@ async function saveCloseAndReopenAsApp(
|
||||
await appMode.steps.goToPreview()
|
||||
await builderSaveAs(appMode, workflowName)
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await expect(appMode.saveAs.successDialog).toBeHidden()
|
||||
await expect(appMode.saveAs.successDialog).not.toBeVisible()
|
||||
|
||||
await appMode.footer.exitBuilder()
|
||||
await openWorkflowFromSidebar(comfyPage, workflowName)
|
||||
@@ -123,7 +122,7 @@ test.describe('Builder input reordering', { tag: '@ui' }, () => {
|
||||
const workflowName = `${Date.now()} reorder-preview`
|
||||
await saveCloseAndReopenAsApp(comfyPage, appMode, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.select.previewWidgetLabels).toHaveText([
|
||||
'steps',
|
||||
'cfg',
|
||||
@@ -148,58 +147,11 @@ test.describe('Builder input reordering', { tag: '@ui' }, () => {
|
||||
const workflowName = `${Date.now()} reorder-persist`
|
||||
await saveCloseAndReopenAsApp(comfyPage, appMode, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible()
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.select.previewWidgetLabels).toHaveText([
|
||||
'steps',
|
||||
'cfg',
|
||||
'seed'
|
||||
])
|
||||
})
|
||||
|
||||
test('Reordering inputs in one app does not corrupt another app', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { appMode } = comfyPage
|
||||
const suffix = String(Date.now())
|
||||
const app1Name = `app1-${suffix}`
|
||||
const app2Name = `app2-${suffix}`
|
||||
const app2Widgets = ['seed', 'steps']
|
||||
|
||||
// Create and save app1 with [seed, steps, cfg]
|
||||
await createAndSaveApp(comfyPage, app1Name, WIDGETS)
|
||||
await appMode.footer.exitBuilder()
|
||||
|
||||
// Create app2 in a new tab so both apps are open simultaneously
|
||||
await comfyPage.menu.topbar.triggerTopbarCommand(['New'])
|
||||
await createAndSaveApp(comfyPage, app2Name, app2Widgets)
|
||||
await appMode.footer.exitBuilder()
|
||||
|
||||
// Switch to app1 tab and enter builder
|
||||
await comfyPage.menu.topbar.getWorkflowTab(app1Name).click()
|
||||
await appMode.enterBuilder()
|
||||
await appMode.steps.goToInputs()
|
||||
await expect(appMode.select.inputItemTitles).toHaveText(WIDGETS)
|
||||
|
||||
// Reorder app1 inputs: drag 'seed' from first to last
|
||||
await appMode.select.dragInputItem(0, 2)
|
||||
const app1Reordered = ['steps', 'cfg', 'seed']
|
||||
await expect(appMode.select.inputItemTitles).toHaveText(app1Reordered)
|
||||
|
||||
// Switch to app2 tab and enter builder
|
||||
await appMode.footer.exitBuilder()
|
||||
await comfyPage.menu.topbar.getWorkflowTab(app2Name).click()
|
||||
await appMode.enterBuilder()
|
||||
await appMode.steps.goToInputs()
|
||||
|
||||
// Verify app2 inputs are not corrupted — still [seed, steps]
|
||||
await expect(appMode.select.inputItemTitles).toHaveText(app2Widgets)
|
||||
|
||||
// Switch back to app1 and verify reorder persisted
|
||||
await appMode.footer.exitBuilder()
|
||||
await comfyPage.menu.topbar.getWorkflowTab(app1Name).click()
|
||||
await appMode.enterBuilder()
|
||||
await appMode.steps.goToInputs()
|
||||
|
||||
await expect(appMode.select.inputItemTitles).toHaveText(app1Reordered)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -21,7 +21,7 @@ async function reSaveAs(
|
||||
viewType: 'App' | 'Node graph'
|
||||
) {
|
||||
await appMode.footer.openSaveAsFromChevron()
|
||||
await expect(appMode.saveAs.nameInput).toBeVisible()
|
||||
await expect(appMode.saveAs.nameInput).toBeVisible({ timeout: 5000 })
|
||||
await appMode.saveAs.fillAndSave(workflowName, viewType)
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ async function dismissSuccessDialog(
|
||||
) {
|
||||
const btn = button === 'close' ? saveAs.closeButton : saveAs.dismissButton
|
||||
await btn.click()
|
||||
await expect(saveAs.successDialog).toBeHidden()
|
||||
await expect(saveAs.successDialog).not.toBeVisible()
|
||||
}
|
||||
|
||||
test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
@@ -48,7 +48,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
await comfyPage.appMode.footer.saveAsButton.click()
|
||||
|
||||
await expect(saveAs.dialog).toBeVisible()
|
||||
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
|
||||
await expect(saveAs.nameInput).toBeVisible()
|
||||
await expect(saveAs.title).toBeVisible()
|
||||
await expect(saveAs.radioGroup).toBeVisible()
|
||||
@@ -68,7 +68,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
await comfyPage.appMode.footer.saveAsButton.click()
|
||||
|
||||
await expect(saveAs.dialog).toBeVisible()
|
||||
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
|
||||
await saveAs.nameInput.fill('')
|
||||
await expect(saveAs.saveButton).toBeDisabled()
|
||||
})
|
||||
@@ -78,7 +78,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
await comfyPage.appMode.footer.saveAsButton.click()
|
||||
|
||||
await expect(saveAs.dialog).toBeVisible()
|
||||
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
|
||||
|
||||
const appRadio = saveAs.viewTypeRadio('App')
|
||||
await expect(appRadio).toHaveAttribute('aria-checked', 'true')
|
||||
@@ -113,7 +113,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
|
||||
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Exit builder button exits builder mode', async ({ comfyPage }) => {
|
||||
@@ -121,7 +121,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
await expect(comfyPage.appMode.steps.toolbar).toBeVisible()
|
||||
await comfyPage.appMode.footer.exitBuilder()
|
||||
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
|
||||
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Save button directly saves for previously saved workflow', async ({
|
||||
@@ -136,12 +136,12 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
// Modify the workflow so the save button becomes enabled
|
||||
await comfyPage.appMode.steps.goToInputs()
|
||||
await comfyPage.appMode.select.deleteInput('seed')
|
||||
await expect(footer.saveButton).toBeEnabled()
|
||||
await expect(footer.saveButton).toBeEnabled({ timeout: 5000 })
|
||||
|
||||
await footer.saveButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(saveAs.dialog).toBeHidden()
|
||||
await expect(saveAs.dialog).not.toBeVisible({ timeout: 2000 })
|
||||
await expect(footer.saveButton).toBeDisabled()
|
||||
})
|
||||
|
||||
@@ -156,7 +156,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
await footer.openSaveAsFromChevron()
|
||||
|
||||
await expect(saveAs.title).toBeVisible()
|
||||
await expect(saveAs.title).toBeVisible({ timeout: 5000 })
|
||||
await expect(saveAs.nameInput).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -209,7 +209,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByText('Connect an output', { exact: false })
|
||||
).toBeVisible()
|
||||
).toBeVisible({ timeout: 5000 })
|
||||
})
|
||||
|
||||
test('save as app produces correct extension and linearMode', async ({
|
||||
@@ -253,7 +253,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await builderSaveAs(comfyPage.appMode, `${Date.now()} app-view`, 'App')
|
||||
|
||||
await comfyPage.appMode.saveAs.viewAppButton.click()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).toBeHidden()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowActiveAppMode())
|
||||
@@ -271,9 +271,9 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
)
|
||||
|
||||
await comfyPage.appMode.saveAs.exitBuilderButton.click()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).toBeHidden()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
|
||||
|
||||
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
|
||||
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('save as with different mode does not modify the original workflow', async ({
|
||||
@@ -291,7 +291,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
// Re-save as node graph — creates a copy
|
||||
await reSaveAs(appMode, `${Date.now()} copy`, 'Node graph')
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible()
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
@@ -325,11 +325,11 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
await reSaveAs(appMode, name, 'App')
|
||||
|
||||
await expect(appMode.saveAs.overwriteDialog).toBeVisible()
|
||||
await expect(appMode.saveAs.overwriteDialog).toBeVisible({ timeout: 5000 })
|
||||
await appMode.saveAs.overwriteButton.click()
|
||||
await expect(appMode.saveAs.overwriteDialog).toBeHidden()
|
||||
await expect(appMode.saveAs.overwriteDialog).not.toBeVisible()
|
||||
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible()
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
@@ -351,7 +351,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await dismissSuccessDialog(appMode.saveAs)
|
||||
|
||||
await reSaveAs(appMode, name, 'Node graph')
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible()
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
|
||||
@@ -70,7 +70,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
|
||||
await expect(menu).toBeVisible()
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeHidden()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
|
||||
})
|
||||
|
||||
@@ -81,7 +81,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
|
||||
await expect(menu).toBeVisible()
|
||||
await handItem.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeHidden()
|
||||
await expect(menu).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('closes when Escape is pressed', async ({ comfyPage }) => {
|
||||
@@ -91,7 +91,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
|
||||
await expect(menu).toBeVisible()
|
||||
await selectItem.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeHidden()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
|
||||
})
|
||||
})
|
||||
@@ -197,7 +197,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
|
||||
await selectItem.press('ArrowDown')
|
||||
await handItem.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeHidden()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toBeFocused()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -206,8 +206,8 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
|
||||
|
||||
// Ensure undo reverts both changes
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(node).not.toBeBypassed()
|
||||
await expect(node).not.toBeCollapsed()
|
||||
await expect(node).not.toBeBypassed({ timeout: 5000 })
|
||||
await expect(node).not.toBeCollapsed({ timeout: 5000 })
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: false,
|
||||
redoQueueSize: 1,
|
||||
|
||||
@@ -75,18 +75,15 @@ test.describe('Asset-supported node default value', { tag: '@cloud' }, () => {
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(
|
||||
async () => {
|
||||
return await comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.graph.getNodeById(id)
|
||||
const widget = node?.widgets?.find(
|
||||
(w: { name: string }) => w.name === 'ckpt_name'
|
||||
)
|
||||
return String(widget?.value ?? '')
|
||||
}, nodeId)
|
||||
},
|
||||
{ timeout: 10_000 }
|
||||
)
|
||||
.poll(async () => {
|
||||
return await comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.graph.getNodeById(id)
|
||||
const widget = node?.widgets?.find(
|
||||
(w: { name: string }) => w.name === 'ckpt_name'
|
||||
)
|
||||
return String(widget?.value ?? '')
|
||||
}, nodeId)
|
||||
})
|
||||
.toBe(CLOUD_ASSETS[0].name)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -149,7 +149,11 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
|
||||
await comfyPage.canvas.click({ position: { x: 50, y: 500 } })
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.clipboard.paste()
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(3)
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5_000
|
||||
})
|
||||
.toBe(3)
|
||||
|
||||
// Step 2: Paste image onto selected LoadImage node
|
||||
const loadImageNodes =
|
||||
@@ -167,10 +171,13 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
|
||||
await uploadPromise
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const fileWidget = await loadImageNodes[0].getWidget(0)
|
||||
return fileWidget.getValue()
|
||||
})
|
||||
.poll(
|
||||
async () => {
|
||||
const fileWidget = await loadImageNodes[0].getWidget(0)
|
||||
return fileWidget.getValue()
|
||||
},
|
||||
{ timeout: 5_000 }
|
||||
)
|
||||
.toContain('image32x32')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(3)
|
||||
|
||||
@@ -187,7 +194,11 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
|
||||
)
|
||||
await uploadPromise2
|
||||
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(4)
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5_000
|
||||
})
|
||||
.toBe(4)
|
||||
const allLoadImageNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByType('LoadImage')
|
||||
expect(allLoadImageNodes).toHaveLength(2)
|
||||
|
||||
@@ -38,13 +38,13 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
`.${tabId}-tab-button.side-bar-button-selected`
|
||||
)
|
||||
|
||||
await expect(selectedButton).toBeHidden()
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await expect(selectedButton).toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await expect(selectedButton).toBeHidden()
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
})
|
||||
}
|
||||
})
|
||||
@@ -172,7 +172,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
|
||||
// Toggle off with Alt+m
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeHidden()
|
||||
await expect(comfyPage.appMode.linearWidgets).not.toBeVisible()
|
||||
|
||||
// Toggle on again
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
@@ -189,7 +189,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await expect(minimap).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await expect(minimap).toBeHidden()
|
||||
await expect(minimap).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await expect(minimap).toBeVisible()
|
||||
@@ -198,13 +198,13 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
test("'Ctrl+`' toggles terminal/logs panel", async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
await expect(comfyPage.bottomPanel.root).toBeHidden()
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await expect(comfyPage.bottomPanel.root).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await expect(comfyPage.bottomPanel.root).toBeHidden()
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -250,7 +250,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
|
||||
// The Save As dialog should appear (p-dialog overlay)
|
||||
const dialogOverlay = comfyPage.page.locator('.p-dialog-mask')
|
||||
await expect(dialogOverlay).toBeVisible()
|
||||
await expect(dialogOverlay).toBeVisible({ timeout: 3000 })
|
||||
|
||||
// Dismiss the dialog
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
@@ -303,7 +303,9 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
// After conversion, node count should decrease
|
||||
// (multiple nodes replaced by single subgraph node)
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5000
|
||||
})
|
||||
.toBeLessThan(initialCount)
|
||||
})
|
||||
|
||||
|
||||
@@ -11,7 +11,9 @@ test.beforeEach(async ({ comfyPage }) => {
|
||||
test.describe('Settings', () => {
|
||||
test('@mobile Should be visible on mobile', async ({ comfyPage }) => {
|
||||
await comfyPage.page.keyboard.press('Control+,')
|
||||
const settingsDialog = comfyPage.page.getByTestId('settings-dialog')
|
||||
const settingsDialog = comfyPage.page.locator(
|
||||
'[data-testid="settings-dialog"]'
|
||||
)
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
const contentArea = settingsDialog.locator('main')
|
||||
await expect(contentArea).toBeVisible()
|
||||
@@ -24,16 +26,17 @@ test.describe('Settings', () => {
|
||||
await comfyPage.page.keyboard.down('ControlOrMeta')
|
||||
await comfyPage.page.keyboard.press(',')
|
||||
await comfyPage.page.keyboard.up('ControlOrMeta')
|
||||
const settingsLocator = comfyPage.page.getByTestId('settings-dialog')
|
||||
const settingsLocator = comfyPage.page.locator(
|
||||
'[data-testid="settings-dialog"]'
|
||||
)
|
||||
await expect(settingsLocator).toBeVisible()
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await expect(settingsLocator).toBeHidden()
|
||||
await expect(settingsLocator).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Can change canvas zoom speed setting', async ({ comfyPage }) => {
|
||||
const maxSpeed = 2.5
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.ZoomSpeed', maxSpeed)
|
||||
|
||||
await test.step('Setting should persist', async () => {
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.Graph.ZoomSpeed'))
|
||||
@@ -46,7 +49,9 @@ test.describe('Settings', () => {
|
||||
await comfyPage.page.keyboard.press('Control+,')
|
||||
|
||||
// Open the keybinding tab
|
||||
const settingsDialog = comfyPage.page.getByTestId('settings-dialog')
|
||||
const settingsDialog = comfyPage.page.locator(
|
||||
'[data-testid="settings-dialog"]'
|
||||
)
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
await settingsDialog
|
||||
.locator('nav [role="button"]', { hasText: 'Keybinding' })
|
||||
|
||||
@@ -307,7 +307,7 @@ test.describe('ManagerDialog', { tag: '@ui' }, () => {
|
||||
await searchInput.fill('Test Pack B')
|
||||
|
||||
await expect(dialog.getByText('Test Pack B')).toBeVisible()
|
||||
await expect(dialog.getByText('Test Pack A')).toBeHidden()
|
||||
await expect(dialog.getByText('Test Pack A')).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Clicking a pack card opens the info panel', async ({ comfyPage }) => {
|
||||
@@ -360,7 +360,7 @@ test.describe('ManagerDialog', { tag: '@ui' }, () => {
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await expect(dialog).toBeHidden()
|
||||
await expect(dialog).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Empty search shows no results message', async ({ comfyPage }) => {
|
||||
|
||||
@@ -60,7 +60,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
await dialog.getByRole('button', { name: 'Cancel' }).click()
|
||||
await expect(dialog).toBeHidden()
|
||||
await expect(dialog).not.toBeVisible()
|
||||
expect(clearCalled).toBe(false)
|
||||
|
||||
await comfyPage.page.unroute('**/api/history')
|
||||
@@ -83,7 +83,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
await dialog.getByLabel('Close').click()
|
||||
await expect(dialog).toBeHidden()
|
||||
await expect(dialog).not.toBeVisible()
|
||||
expect(clearCalled).toBe(false)
|
||||
|
||||
await comfyPage.page.unroute('**/api/history')
|
||||
@@ -106,7 +106,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
|
||||
const request = await clearPromise
|
||||
expect(request.postDataJSON()).toEqual({ clear: true })
|
||||
|
||||
await expect(dialog).toBeHidden()
|
||||
await expect(dialog).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Dialog state resets after close and reopen', async ({ comfyPage }) => {
|
||||
@@ -114,7 +114,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
|
||||
const dialog = comfyPage.confirmDialog.root
|
||||
await expect(dialog).toBeVisible()
|
||||
await dialog.getByRole('button', { name: 'Cancel' }).click()
|
||||
await expect(dialog).toBeHidden()
|
||||
await expect(dialog).not.toBeVisible()
|
||||
|
||||
await comfyPage.queuePanel.openClearHistoryDialog()
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
@@ -61,7 +61,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
|
||||
await expect(dialog.root).toBeVisible()
|
||||
|
||||
await dialog.close()
|
||||
await expect(dialog.root).toBeHidden()
|
||||
await expect(dialog.root).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Escape key closes dialog', async ({ comfyPage }) => {
|
||||
@@ -70,7 +70,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
|
||||
await expect(dialog.root).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await expect(dialog.root).toBeHidden()
|
||||
await expect(dialog.root).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Search filters settings list', async ({ comfyPage }) => {
|
||||
@@ -153,7 +153,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
|
||||
const expanded = await select.getAttribute('aria-expanded')
|
||||
if (expanded !== 'true') await select.click()
|
||||
await expect(select).toHaveAttribute('aria-expanded', 'true')
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass({ timeout: 3000 })
|
||||
|
||||
// Pick the option that is not the current value
|
||||
const targetValue = initialValue === 'Top' ? 'Disabled' : 'Top'
|
||||
|
||||
@@ -10,7 +10,7 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
|
||||
test('Collapsed multiline textarea is not visible', async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('widgets/collapsed_multiline')
|
||||
const textareaWidget = comfyPage.page.locator('.comfy-multiline-input')
|
||||
await expect(textareaWidget).toBeHidden()
|
||||
await expect(textareaWidget).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Multiline textarea correctly collapses', async ({ comfyPage }) => {
|
||||
@@ -25,8 +25,8 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
|
||||
for (const node of nodes) {
|
||||
await node.click('collapse')
|
||||
}
|
||||
await expect(firstMultiline).toBeHidden()
|
||||
await expect(lastMultiline).toBeHidden()
|
||||
await expect(firstMultiline).not.toBeVisible()
|
||||
await expect(lastMultiline).not.toBeVisible()
|
||||
})
|
||||
|
||||
test(
|
||||
@@ -35,7 +35,7 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('focus-mode-on.png')
|
||||
}
|
||||
)
|
||||
|
||||
@@ -74,7 +74,7 @@ test.describe('Error dialog', () => {
|
||||
}) => {
|
||||
const errorDialog = await triggerConfigureError(comfyPage)
|
||||
await expect(errorDialog).toBeVisible()
|
||||
await expect(errorDialog.locator('pre')).toBeHidden()
|
||||
await expect(errorDialog.locator('pre')).not.toBeVisible()
|
||||
|
||||
await errorDialog.getByTestId(TestIds.dialogs.errorDialogShowReport).click()
|
||||
|
||||
@@ -83,7 +83,7 @@ test.describe('Error dialog', () => {
|
||||
await expect(reportPre).toHaveText(/\S/)
|
||||
await expect(
|
||||
errorDialog.getByTestId(TestIds.dialogs.errorDialogShowReport)
|
||||
).toBeHidden()
|
||||
).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Should copy report to clipboard when "Copy to Clipboard" is clicked', async ({
|
||||
|
||||
@@ -100,7 +100,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
await errorOverlay
|
||||
.getByTestId(TestIds.dialogs.errorOverlayDismiss)
|
||||
.click()
|
||||
await expect(errorOverlay).toBeHidden()
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
|
||||
await comfyPage.canvas.click()
|
||||
await comfyPage.nextFrame()
|
||||
@@ -112,10 +112,10 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(errorOverlay).toBeHidden()
|
||||
await expect(errorOverlay).not.toBeVisible({ timeout: 5000 })
|
||||
|
||||
await comfyPage.keyboard.redo()
|
||||
await expect(errorOverlay).toBeHidden()
|
||||
await expect(errorOverlay).not.toBeVisible({ timeout: 5000 })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -156,7 +156,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
|
||||
await overlay.getByTestId(TestIds.dialogs.errorOverlaySeeErrors).click()
|
||||
|
||||
await expect(overlay).toBeHidden()
|
||||
await expect(overlay).not.toBeVisible()
|
||||
await expect(comfyPage.page.getByTestId('properties-panel')).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -168,7 +168,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
|
||||
await overlay.getByTestId(TestIds.dialogs.errorOverlaySeeErrors).click()
|
||||
|
||||
await expect(overlay).toBeHidden()
|
||||
await expect(overlay).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('"Dismiss" closes overlay without opening panel', async ({
|
||||
@@ -181,8 +181,10 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
|
||||
await overlay.getByTestId(TestIds.dialogs.errorOverlayDismiss).click()
|
||||
|
||||
await expect(overlay).toBeHidden()
|
||||
await expect(comfyPage.page.getByTestId('properties-panel')).toBeHidden()
|
||||
await expect(overlay).not.toBeVisible()
|
||||
await expect(
|
||||
comfyPage.page.getByTestId('properties-panel')
|
||||
).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Close button (X) dismisses overlay', async ({ comfyPage }) => {
|
||||
@@ -193,7 +195,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
|
||||
await overlay.getByRole('button', { name: /close/i }).click()
|
||||
|
||||
await expect(overlay).toBeHidden()
|
||||
await expect(overlay).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -63,13 +63,19 @@ test.describe(
|
||||
|
||||
await comfyPage.command.executeCommand('Comfy.QueueSelectedOutputNodes')
|
||||
await expect
|
||||
.poll(async () => (await input.getWidget(0)).getValue())
|
||||
.poll(async () => (await input.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('foo')
|
||||
await expect
|
||||
.poll(async () => (await output1.getWidget(0)).getValue())
|
||||
.poll(async () => (await output1.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('foo')
|
||||
await expect
|
||||
.poll(async () => (await output2.getWidget(0)).getValue())
|
||||
.poll(async () => (await output2.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('')
|
||||
})
|
||||
}
|
||||
|
||||
@@ -69,7 +69,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
expect(flags?.data).not.toBeNull()
|
||||
expect(flags?.data).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags?.data?.supports_preview_metadata).toBe('boolean')
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
|
||||
// Verify server sent feature flags back
|
||||
await expect(async () => {
|
||||
@@ -82,7 +82,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
expect(typeof flags?.max_upload_size).toBe('number')
|
||||
expect(Object.keys(flags ?? {}).length).toBeGreaterThan(0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
|
||||
await newPage.close()
|
||||
})
|
||||
@@ -102,7 +102,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
expect(typeof flags.supports_preview_metadata).toBe('boolean')
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
expect(typeof flags.max_upload_size).toBe('number')
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
})
|
||||
|
||||
test('serverSupportsFeature method works with real backend flags', async ({
|
||||
@@ -182,7 +182,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
)
|
||||
expect(typeof maxUpload).toBe('number')
|
||||
expect(maxUpload as number).toBeGreaterThan(0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
|
||||
// Test getServerFeature with default value for non-existent feature
|
||||
await expect
|
||||
@@ -210,7 +210,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
expect(typeof features.supports_preview_metadata).toBe('boolean')
|
||||
expect(features).toHaveProperty('max_upload_size')
|
||||
expect(Object.keys(features).length).toBeGreaterThan(0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
})
|
||||
|
||||
test('Client feature flags are immutable', async ({ comfyPage }) => {
|
||||
@@ -348,14 +348,14 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
expect(flags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags?.supports_preview_metadata).toBe('boolean')
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
|
||||
// Verify feature flags were received and API was initialized
|
||||
await expect(async () => {
|
||||
const readiness = await newPage.evaluate(() => window.__appReadiness)
|
||||
expect(readiness?.featureFlagsReceived).toBe(true)
|
||||
expect(readiness?.apiInitialized).toBe(true)
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
|
||||
await newPage.close()
|
||||
})
|
||||
|
||||
@@ -14,12 +14,12 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
|
||||
|
||||
await comfyPage.setFocusMode(true)
|
||||
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Focus mode restores UI chrome', async ({ comfyPage }) => {
|
||||
await comfyPage.setFocusMode(true)
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
|
||||
await comfyPage.setFocusMode(false)
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
@@ -29,7 +29,7 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
@@ -41,7 +41,7 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
|
||||
|
||||
await comfyPage.setFocusMode(true)
|
||||
|
||||
await expect(topMenu).toBeHidden()
|
||||
await expect(topMenu).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Canvas remains visible in focus mode', async ({ comfyPage }) => {
|
||||
@@ -52,12 +52,12 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
|
||||
|
||||
test('Focus mode can be toggled multiple times', async ({ comfyPage }) => {
|
||||
await comfyPage.setFocusMode(true)
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
|
||||
await comfyPage.setFocusMode(false)
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
|
||||
await comfyPage.setFocusMode(true)
|
||||
await expect(comfyPage.menu.sideToolbar).toBeHidden()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -109,6 +109,6 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
|
||||
expect(r.switchOutputLinkIds).toEqual(
|
||||
expect.arrayContaining([r.cfg85LinkId, r.cfg86LinkId])
|
||||
)
|
||||
}).toPass({ timeout: 5000 })
|
||||
}).toPass()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -94,6 +94,6 @@ test.describe('Graph Canvas Menu', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
await backdrop.click()
|
||||
|
||||
// Modal should be hidden
|
||||
await expect(zoomModal).toBeHidden()
|
||||
await expect(zoomModal).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -88,10 +88,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
.getNode(groupNodeName)
|
||||
.locator('.bookmark-button')
|
||||
.click()
|
||||
await comfyPage.page
|
||||
.locator('.p-tree-node-label.tree-explorer-node-label')
|
||||
.first()
|
||||
.hover()
|
||||
await comfyPage.page.hover('.p-tree-node-label.tree-explorer-node-label')
|
||||
await expect(
|
||||
comfyPage.page.locator('.node-lib-node-preview')
|
||||
).toBeVisible()
|
||||
@@ -102,7 +99,6 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
.click()
|
||||
})
|
||||
})
|
||||
|
||||
test(
|
||||
'Can be added to canvas using search',
|
||||
{ tag: '@screenshot' },
|
||||
@@ -158,7 +154,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await expect(manage1.selectedNodeTypeSelect).toHaveValue('g1')
|
||||
await manage1.close()
|
||||
await expect(manage1.root).toBeHidden()
|
||||
await expect(manage1.root).not.toBeVisible()
|
||||
|
||||
const manage2 = await group2.manageGroupNode()
|
||||
await expect(manage2.selectedNodeTypeSelect).toHaveValue('g2')
|
||||
@@ -245,7 +241,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
await expect(
|
||||
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
|
||||
).toBeHidden()
|
||||
).not.toBeVisible()
|
||||
})
|
||||
|
||||
test.describe('Copy and paste', () => {
|
||||
@@ -353,7 +349,6 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
await comfyPage.page.keyboard.press('Alt+g')
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(1)
|
||||
})
|
||||
|
||||
test('Convert to group node, selected 1 node', async ({ comfyPage }) => {
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await comfyPage.canvas.click({
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
test.describe('Image Compare', { tag: '@widget' }, () => {
|
||||
test.describe('Image Compare', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
await comfyPage.workflow.loadWorkflow('widgets/image_compare_widget')
|
||||
@@ -22,12 +21,7 @@ test.describe('Image Compare', { tag: '@widget' }, () => {
|
||||
|
||||
async function setImageCompareValue(
|
||||
comfyPage: ComfyPage,
|
||||
value: {
|
||||
beforeImages: string[]
|
||||
afterImages: string[]
|
||||
beforeAlt?: string
|
||||
afterAlt?: string
|
||||
}
|
||||
value: { beforeImages: string[]; afterImages: string[] }
|
||||
) {
|
||||
await comfyPage.page.evaluate(
|
||||
({ value }) => {
|
||||
@@ -43,48 +37,6 @@ test.describe('Image Compare', { tag: '@widget' }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
async function moveToPercentage(
|
||||
page: Page,
|
||||
containerLocator: Locator,
|
||||
percentage: number
|
||||
) {
|
||||
const box = await containerLocator.boundingBox()
|
||||
if (!box) throw new Error('Container not found')
|
||||
await page.mouse.move(
|
||||
box.x + box.width * (percentage / 100),
|
||||
box.y + box.height / 2
|
||||
)
|
||||
}
|
||||
|
||||
async function waitForImagesLoaded(node: Locator) {
|
||||
await expect
|
||||
.poll(() =>
|
||||
node.evaluate((el) => {
|
||||
const imgs = el.querySelectorAll('img')
|
||||
return (
|
||||
imgs.length > 0 &&
|
||||
Array.from(imgs).every(
|
||||
(img) => img.complete && img.naturalWidth > 0
|
||||
)
|
||||
)
|
||||
})
|
||||
)
|
||||
.toBe(true)
|
||||
}
|
||||
|
||||
async function getClipPathInsetRightPercent(imgLocator: Locator) {
|
||||
return imgLocator.evaluate((el) => {
|
||||
// Accessing raw style avoids cross-browser getComputedStyle normalization issues
|
||||
// Format is uniformly "inset(0 60% 0 0)" per Vue runtime inline style bindings
|
||||
const parts = (el as HTMLElement).style.clipPath.split(' ')
|
||||
return parts.length > 1 ? parseFloat(parts[1]) : -1
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Rendering
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test(
|
||||
'Shows empty state when no images are set',
|
||||
{ tag: '@smoke' },
|
||||
@@ -94,14 +46,10 @@ test.describe('Image Compare', { tag: '@widget' }, () => {
|
||||
|
||||
await expect(node).toContainText('No images to compare')
|
||||
await expect(node.locator('img')).toHaveCount(0)
|
||||
await expect(node.getByRole('presentation')).toHaveCount(0)
|
||||
await expect(node.locator('[role="presentation"]')).toHaveCount(0)
|
||||
}
|
||||
)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slider defaults
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test(
|
||||
'Slider defaults to 50% with both images set',
|
||||
{ tag: ['@smoke', '@screenshot'] },
|
||||
@@ -119,444 +67,15 @@ test.describe('Image Compare', { tag: '@widget' }, () => {
|
||||
await expect(beforeImg).toBeVisible()
|
||||
await expect(afterImg).toBeVisible()
|
||||
|
||||
const handle = node.getByRole('presentation')
|
||||
const handle = node.locator('[role="presentation"]')
|
||||
await expect(handle).toBeVisible()
|
||||
|
||||
expect(
|
||||
await handle.evaluate((el) => (el as HTMLElement).style.left),
|
||||
'Slider should default to 50% before screenshot'
|
||||
await handle.evaluate((el) => (el as HTMLElement).style.left)
|
||||
).toBe('50%')
|
||||
await expect
|
||||
.poll(() => getClipPathInsetRightPercent(beforeImg))
|
||||
.toBeCloseTo(50, 0)
|
||||
await expect(beforeImg).toHaveCSS('clip-path', /50%/)
|
||||
|
||||
await waitForImagesLoaded(node)
|
||||
await comfyPage.page.mouse.move(0, 0)
|
||||
await expect(node).toHaveScreenshot('image-compare-default-50.png')
|
||||
}
|
||||
)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Slider interaction
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test(
|
||||
'Mouse hover moves slider position',
|
||||
{ tag: '@smoke' },
|
||||
async ({ comfyPage }) => {
|
||||
const beforeUrl = createTestImageDataUrl('Before', '#c00')
|
||||
const afterUrl = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [beforeUrl],
|
||||
afterImages: [afterUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const handle = node.getByRole('presentation')
|
||||
const beforeImg = node.locator('img[alt="Before image"]')
|
||||
const afterImg = node.locator('img[alt="After image"]')
|
||||
await expect(afterImg).toBeVisible()
|
||||
|
||||
// Left edge: sliderPosition ≈ 5 → clip-path inset right ≈ 95%
|
||||
await moveToPercentage(comfyPage.page, afterImg, 5)
|
||||
await expect
|
||||
.poll(() => getClipPathInsetRightPercent(beforeImg))
|
||||
.toBeGreaterThan(90)
|
||||
await expect
|
||||
.poll(() =>
|
||||
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
|
||||
)
|
||||
.toBeLessThan(10)
|
||||
|
||||
// Right edge: sliderPosition ≈ 95 → clip-path inset right ≈ 5%
|
||||
await moveToPercentage(comfyPage.page, afterImg, 95)
|
||||
await expect
|
||||
.poll(() => getClipPathInsetRightPercent(beforeImg))
|
||||
.toBeLessThan(10)
|
||||
await expect
|
||||
.poll(() =>
|
||||
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
|
||||
)
|
||||
.toBeGreaterThan(90)
|
||||
}
|
||||
)
|
||||
|
||||
test('Slider preserves last position when mouse leaves widget', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const beforeUrl = createTestImageDataUrl('Before', '#c00')
|
||||
const afterUrl = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [beforeUrl],
|
||||
afterImages: [afterUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const handle = node.getByRole('presentation')
|
||||
const afterImg = node.locator('img[alt="After image"]')
|
||||
await expect(afterImg).toBeVisible()
|
||||
|
||||
await moveToPercentage(comfyPage.page, afterImg, 30)
|
||||
// Wait for Vue to commit the slider update
|
||||
await expect
|
||||
.poll(() =>
|
||||
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
|
||||
)
|
||||
.toBeCloseTo(30, 0)
|
||||
const positionWhileInside = parseFloat(
|
||||
await handle.evaluate((el) => (el as HTMLElement).style.left)
|
||||
)
|
||||
|
||||
await comfyPage.page.mouse.move(0, 0)
|
||||
|
||||
// Position must not reset to default 50%
|
||||
await expect
|
||||
.poll(() =>
|
||||
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
|
||||
)
|
||||
.toBeCloseTo(positionWhileInside, 0)
|
||||
})
|
||||
|
||||
test('Slider clamps to 0% at left edge of container', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const beforeUrl = createTestImageDataUrl('Before', '#c00')
|
||||
const afterUrl = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [beforeUrl],
|
||||
afterImages: [afterUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const handle = node.getByRole('presentation')
|
||||
const afterImg = node.locator('img[alt="After image"]')
|
||||
await expect(afterImg).toBeVisible()
|
||||
|
||||
const box = await afterImg.boundingBox()
|
||||
if (!box) throw new Error('Container not found')
|
||||
|
||||
// Move to the leftmost pixel (elementX = 0 → sliderPosition = 0)
|
||||
await comfyPage.page.mouse.move(box.x, box.y + box.height / 2)
|
||||
await expect
|
||||
.poll(() => handle.evaluate((el) => (el as HTMLElement).style.left))
|
||||
.toBe('0%')
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Single image modes
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('Only before image shows without slider when afterImages is empty', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const url = createTestImageDataUrl('Before', '#c00')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url],
|
||||
afterImages: []
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.locator('img')).toHaveCount(1)
|
||||
await expect(node.getByRole('presentation')).toBeHidden()
|
||||
})
|
||||
|
||||
test('Only after image shows without slider when beforeImages is empty', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const url = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [],
|
||||
afterImages: [url]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.locator('img')).toHaveCount(1)
|
||||
await expect(node.getByRole('presentation')).toBeHidden()
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Batch navigation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test(
|
||||
'Batch navigation appears when before side has multiple images',
|
||||
{ tag: '@smoke' },
|
||||
async ({ comfyPage }) => {
|
||||
const url1 = createTestImageDataUrl('A1', '#c00')
|
||||
const url2 = createTestImageDataUrl('A2', '#0c0')
|
||||
const url3 = createTestImageDataUrl('A3', '#00c')
|
||||
const afterUrl = createTestImageDataUrl('B1', '#888')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url1, url2, url3],
|
||||
afterImages: [afterUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const beforeBatch = node.getByTestId('before-batch')
|
||||
|
||||
await expect(node.getByTestId('batch-nav')).toBeVisible()
|
||||
await expect(beforeBatch.getByTestId('batch-counter')).toHaveText('1 / 3')
|
||||
// after-batch renders only when afterBatchCount > 1
|
||||
await expect(node.getByTestId('after-batch')).toBeHidden()
|
||||
await expect(beforeBatch.getByTestId('batch-prev')).toBeDisabled()
|
||||
}
|
||||
)
|
||||
|
||||
test('Batch navigation is hidden when both sides have single images', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const url = createTestImageDataUrl('Image', '#c00')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url],
|
||||
afterImages: [url]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.getByTestId('batch-nav')).toBeHidden()
|
||||
})
|
||||
|
||||
test(
|
||||
'Navigate forward through before images',
|
||||
{ tag: '@smoke' },
|
||||
async ({ comfyPage }) => {
|
||||
const url1 = createTestImageDataUrl('A1', '#c00')
|
||||
const url2 = createTestImageDataUrl('A2', '#0c0')
|
||||
const url3 = createTestImageDataUrl('A3', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url1, url2, url3],
|
||||
afterImages: [createTestImageDataUrl('B1', '#888')]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const beforeBatch = node.getByTestId('before-batch')
|
||||
const counter = beforeBatch.getByTestId('batch-counter')
|
||||
const nextBtn = beforeBatch.getByTestId('batch-next')
|
||||
const prevBtn = beforeBatch.getByTestId('batch-prev')
|
||||
|
||||
await nextBtn.click()
|
||||
await expect(counter).toHaveText('2 / 3')
|
||||
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
|
||||
'src',
|
||||
url2
|
||||
)
|
||||
await expect(prevBtn).toBeEnabled()
|
||||
|
||||
await nextBtn.click()
|
||||
await expect(counter).toHaveText('3 / 3')
|
||||
await expect(nextBtn).toBeDisabled()
|
||||
}
|
||||
)
|
||||
|
||||
test('Navigate backward through before images', async ({ comfyPage }) => {
|
||||
const url1 = createTestImageDataUrl('A1', '#c00')
|
||||
const url2 = createTestImageDataUrl('A2', '#0c0')
|
||||
const url3 = createTestImageDataUrl('A3', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url1, url2, url3],
|
||||
afterImages: [createTestImageDataUrl('B1', '#888')]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const beforeBatch = node.getByTestId('before-batch')
|
||||
const counter = beforeBatch.getByTestId('batch-counter')
|
||||
const nextBtn = beforeBatch.getByTestId('batch-next')
|
||||
const prevBtn = beforeBatch.getByTestId('batch-prev')
|
||||
|
||||
await nextBtn.click()
|
||||
await nextBtn.click()
|
||||
await expect(counter).toHaveText('3 / 3')
|
||||
|
||||
await prevBtn.click()
|
||||
await expect(counter).toHaveText('2 / 3')
|
||||
await expect(prevBtn).toBeEnabled()
|
||||
await expect(nextBtn).toBeEnabled()
|
||||
})
|
||||
|
||||
test('Before and after batch navigation are independent', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const url1 = createTestImageDataUrl('A1', '#c00')
|
||||
const url2 = createTestImageDataUrl('A2', '#0c0')
|
||||
const url3 = createTestImageDataUrl('A3', '#00c')
|
||||
const urlA = createTestImageDataUrl('B1', '#880')
|
||||
const urlB = createTestImageDataUrl('B2', '#008')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [url1, url2, url3],
|
||||
afterImages: [urlA, urlB]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const beforeBatch = node.getByTestId('before-batch')
|
||||
const afterBatch = node.getByTestId('after-batch')
|
||||
|
||||
await beforeBatch.getByTestId('batch-next').click()
|
||||
await afterBatch.getByTestId('batch-next').click()
|
||||
|
||||
await expect(beforeBatch.getByTestId('batch-counter')).toHaveText('2 / 3')
|
||||
await expect(afterBatch.getByTestId('batch-counter')).toHaveText('2 / 2')
|
||||
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
|
||||
'src',
|
||||
url2
|
||||
)
|
||||
await expect(node.locator('img[alt="After image"]')).toHaveAttribute(
|
||||
'src',
|
||||
urlB
|
||||
)
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Visual regression screenshots
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
for (const { pct, expectedClipMin, expectedClipMax } of [
|
||||
{ pct: 25, expectedClipMin: 70, expectedClipMax: 80 },
|
||||
{ pct: 75, expectedClipMin: 20, expectedClipMax: 30 }
|
||||
]) {
|
||||
test(
|
||||
`Screenshot at ${pct}% slider position`,
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
const beforeUrl = createTestImageDataUrl('Before', '#c00')
|
||||
const afterUrl = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [beforeUrl],
|
||||
afterImages: [afterUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
const beforeImg = node.locator('img[alt="Before image"]')
|
||||
const afterImg = node.locator('img[alt="After image"]')
|
||||
await waitForImagesLoaded(node)
|
||||
await moveToPercentage(comfyPage.page, afterImg, pct)
|
||||
await expect
|
||||
.poll(() => getClipPathInsetRightPercent(beforeImg))
|
||||
.toBeGreaterThan(expectedClipMin)
|
||||
await expect
|
||||
.poll(() => getClipPathInsetRightPercent(beforeImg))
|
||||
.toBeLessThan(expectedClipMax)
|
||||
|
||||
await expect(node).toHaveScreenshot(`image-compare-slider-${pct}.png`)
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Edge cases
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('Widget remains stable with broken image URLs', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: ['https://example.invalid/broken.png'],
|
||||
afterImages: ['https://example.invalid/broken2.png']
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.locator('img')).toHaveCount(2)
|
||||
await expect(node.getByRole('presentation')).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
node.evaluate((el) => {
|
||||
const imgs = el.querySelectorAll('img')
|
||||
let errors = 0
|
||||
imgs.forEach((img) => {
|
||||
if (img.complete && img.naturalWidth === 0 && img.src) errors++
|
||||
})
|
||||
return errors
|
||||
})
|
||||
)
|
||||
.toBe(2)
|
||||
})
|
||||
|
||||
test('Rapid value updates show latest images and reset batch index', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const redUrl = createTestImageDataUrl('Red', '#c00')
|
||||
const green1Url = createTestImageDataUrl('G1', '#0c0')
|
||||
const green2Url = createTestImageDataUrl('G2', '#090')
|
||||
const blueUrl = createTestImageDataUrl('Blue', '#00c')
|
||||
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [redUrl, green1Url],
|
||||
afterImages: [blueUrl]
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await node.getByTestId('before-batch').getByTestId('batch-next').click()
|
||||
await expect(
|
||||
node.getByTestId('before-batch').getByTestId('batch-counter')
|
||||
).toHaveText('2 / 2')
|
||||
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [green1Url, green2Url],
|
||||
afterImages: [blueUrl]
|
||||
})
|
||||
|
||||
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
|
||||
'src',
|
||||
green1Url
|
||||
)
|
||||
await expect(
|
||||
node.getByTestId('before-batch').getByTestId('batch-counter')
|
||||
).toHaveText('1 / 2')
|
||||
})
|
||||
|
||||
test('Legacy string value shows single image without slider', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const url = createTestImageDataUrl('Legacy', '#c00')
|
||||
await comfyPage.page.evaluate(
|
||||
({ url }) => {
|
||||
const node = window.app!.graph.getNodeById(1)
|
||||
const widget = node?.widgets?.find((w) => w.type === 'imagecompare')
|
||||
if (widget) {
|
||||
widget.value = url
|
||||
widget.callback?.(url)
|
||||
}
|
||||
},
|
||||
{ url }
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.locator('img')).toHaveCount(1)
|
||||
await expect(node.getByRole('presentation')).toBeHidden()
|
||||
})
|
||||
|
||||
test('Custom beforeAlt and afterAlt are used as img alt text', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const beforeUrl = createTestImageDataUrl('Before', '#c00')
|
||||
const afterUrl = createTestImageDataUrl('After', '#00c')
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: [beforeUrl],
|
||||
afterImages: [afterUrl],
|
||||
beforeAlt: 'Custom before',
|
||||
afterAlt: 'Custom after'
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(node.locator('img[alt="Custom before"]')).toBeVisible()
|
||||
await expect(node.locator('img[alt="Custom after"]')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Large batch sizes show correct counter', async ({ comfyPage }) => {
|
||||
const images = Array.from({ length: 20 }, (_, i) =>
|
||||
createTestImageDataUrl(String(i + 1), '#c00')
|
||||
)
|
||||
await setImageCompareValue(comfyPage, {
|
||||
beforeImages: images,
|
||||
afterImages: images
|
||||
})
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await expect(
|
||||
node.getByTestId('before-batch').getByTestId('batch-counter')
|
||||
).toHaveText('1 / 20')
|
||||
await expect(
|
||||
node.getByTestId('after-batch').getByTestId('batch-counter')
|
||||
).toHaveText('1 / 20')
|
||||
})
|
||||
})
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 14 KiB |
@@ -180,48 +180,6 @@ test.describe('Node Interaction', () => {
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Node Duplication', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
// Pin this suite to the legacy canvas path so Alt+drag exercises
|
||||
// LGraphCanvas, not the Vue node drag handler.
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
|
||||
await comfyPage.nextFrame()
|
||||
})
|
||||
|
||||
test('Can duplicate a regular node via Alt+drag', async ({ comfyPage }) => {
|
||||
const before = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
expect(
|
||||
before,
|
||||
'Expected exactly 2 CLIPTextEncode nodes in default graph'
|
||||
).toHaveLength(2)
|
||||
|
||||
const target = before[0]
|
||||
const pos = await target.getPosition()
|
||||
const src = { x: pos.x + 16, y: pos.y + 16 }
|
||||
|
||||
await comfyPage.page.mouse.move(src.x, src.y)
|
||||
await comfyPage.page.keyboard.down('Alt')
|
||||
try {
|
||||
await comfyPage.page.mouse.down()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.mouse.move(src.x + 120, src.y + 80, { steps: 20 })
|
||||
await comfyPage.page.mouse.up()
|
||||
await comfyPage.nextFrame()
|
||||
} finally {
|
||||
await comfyPage.page.keyboard.up('Alt')
|
||||
}
|
||||
await comfyPage.canvasOps.moveMouseToEmptyArea()
|
||||
|
||||
await expect
|
||||
.poll(
|
||||
async () =>
|
||||
(await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')).length
|
||||
)
|
||||
.toBe(3)
|
||||
expect(await target.exists()).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Edge Interaction', { tag: '@screenshot' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting(
|
||||
@@ -393,20 +351,16 @@ test.describe('Node Interaction', () => {
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'text-encode-toggled-off.png'
|
||||
)
|
||||
// Wait for the double-click window (300ms) to expire so the next
|
||||
// click at the same position isn't interpreted as a double-click.
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const pointer = window.app!.canvas.pointer
|
||||
if (!pointer.eLastDown) return true
|
||||
return performance.now() - pointer.eLastDown.timeStamp > 300
|
||||
})
|
||||
)
|
||||
.toBe(true)
|
||||
await comfyPage.canvas.click({
|
||||
position: togglerPos
|
||||
})
|
||||
// Re-expand: clicking the canvas toggler on a collapsed node is
|
||||
// unreliable because DOM widget overlays may intercept the pointer
|
||||
// event. Use programmatic collapse() for the expand step.
|
||||
// TODO(#11006): Restore click-to-expand once DOM widget overlay pointer interception is fixed
|
||||
await comfyPage.page.evaluate((nodeId) => {
|
||||
const node = window.app!.graph.getNodeById(nodeId)!
|
||||
node.collapse()
|
||||
window.app!.canvas.setDirty(true, true)
|
||||
}, targetNode.id)
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => targetNode.isCollapsed()).toBe(false)
|
||||
// Move mouse away to avoid hover highlight differences.
|
||||
await comfyPage.canvasOps.moveMouseToEmptyArea()
|
||||
@@ -1294,7 +1248,7 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
|
||||
test('Space + left-click drag should pan canvas', async ({ comfyPage }) => {
|
||||
// Click canvas to focus it
|
||||
await comfyPage.canvas.click()
|
||||
await comfyPage.page.click('canvas')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.page.keyboard.down('Space')
|
||||
@@ -1363,7 +1317,7 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
'panning'
|
||||
)
|
||||
|
||||
await comfyPage.canvas.click()
|
||||
await comfyPage.page.click('canvas')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('standard-initial.png')
|
||||
|
||||
@@ -25,7 +25,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByTestId('docked-job-history-action')
|
||||
comfyPage.page.locator('[data-testid="docked-job-history-action"]')
|
||||
).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -34,7 +34,9 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
}) => {
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
const action = comfyPage.page.getByTestId('docked-job-history-action')
|
||||
const action = comfyPage.page.locator(
|
||||
'[data-testid="docked-job-history-action"]'
|
||||
)
|
||||
await expect(action).toBeVisible()
|
||||
await expect(action).not.toBeEmpty()
|
||||
})
|
||||
@@ -43,7 +45,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByTestId('show-run-progress-bar-action')
|
||||
comfyPage.page.locator('[data-testid="show-run-progress-bar-action"]')
|
||||
).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -51,18 +53,20 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByTestId('clear-history-action')
|
||||
comfyPage.page.locator('[data-testid="clear-history-action"]')
|
||||
).toBeVisible()
|
||||
})
|
||||
|
||||
test('Clicking docked job history closes popover', async ({ comfyPage }) => {
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
const action = comfyPage.page.getByTestId('docked-job-history-action')
|
||||
const action = comfyPage.page.locator(
|
||||
'[data-testid="docked-job-history-action"]'
|
||||
)
|
||||
await expect(action).toBeVisible()
|
||||
await action.click()
|
||||
|
||||
await expect(action).toBeHidden()
|
||||
await expect(action).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Clicking show run progress bar toggles setting', async ({
|
||||
@@ -74,7 +78,9 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
|
||||
await openMoreOptionsPopover(comfyPage)
|
||||
|
||||
const action = comfyPage.page.getByTestId('show-run-progress-bar-action')
|
||||
const action = comfyPage.page.locator(
|
||||
'[data-testid="show-run-progress-bar-action"]'
|
||||
)
|
||||
await action.click()
|
||||
|
||||
await expect
|
||||
|
||||
@@ -79,7 +79,9 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss, then close settings via Escape
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
|
||||
@@ -131,7 +133,9 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Export via ellipsis menu
|
||||
await menuButton.click()
|
||||
@@ -179,7 +183,9 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Delete via ellipsis menu
|
||||
await menuButton.click()
|
||||
@@ -217,7 +223,9 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Save as new preset via ellipsis menu
|
||||
await menuButton.click()
|
||||
@@ -229,7 +237,9 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
await promptInput.press('Enter')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Verify preset trigger shows my-custom-preset
|
||||
await expect(presetTrigger).toContainText('my-custom-preset')
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user