mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-04-13 19:20:37 +00:00
Compare commits
16 Commits
sno-qa-901
...
fix/dropdo
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
07e892bcc5 | ||
|
|
b77f0551bd | ||
|
|
04a172c880 | ||
|
|
b29c56dd55 | ||
|
|
8f000fe8da | ||
|
|
8618661bab | ||
|
|
43a24cc869 | ||
|
|
3f3d6e4ebe | ||
|
|
0a4d3e307b | ||
|
|
83da733a5f | ||
|
|
eed1fbbb8a | ||
|
|
c514b6a825 | ||
|
|
88dfe6d749 | ||
|
|
e98df0a577 | ||
|
|
15226f7730 | ||
|
|
b495372511 |
@@ -1,278 +0,0 @@
|
||||
---
|
||||
name: reproduce-issue
|
||||
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
|
||||
---
|
||||
|
||||
# Issue Reproduction Skill
|
||||
|
||||
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two videos are produced:
|
||||
|
||||
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
|
||||
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
|
||||
|
||||
```
|
||||
Phase 1: Research → Read issue, understand prerequisites
|
||||
Phase 2: Environment → Install custom nodes, load workflows, configure settings
|
||||
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
|
||||
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
|
||||
Phase 5: Report → Generate a structured reproduction report
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
|
||||
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
|
||||
- `gh` CLI (authenticated, for reading issues)
|
||||
- ComfyUI backend with Python environment (for installing custom nodes)
|
||||
|
||||
## Phase 1: Research the Issue
|
||||
|
||||
1. Fetch the issue details:
|
||||
|
||||
```bash
|
||||
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
|
||||
```
|
||||
|
||||
2. Extract from the issue body:
|
||||
- **Reproduction steps** (the exact sequence)
|
||||
- **Prerequisites**: specific workflows, custom nodes, settings, models
|
||||
- **Environment**: OS, browser, ComfyUI version
|
||||
- **Media**: screenshots or videos showing the bug
|
||||
|
||||
3. Search the codebase for related code:
|
||||
- Find the feature/component mentioned in the issue
|
||||
- Understand how it works currently
|
||||
- Identify what state the UI needs to be in
|
||||
|
||||
## Phase 2: Environment Setup
|
||||
|
||||
Set up everything the issue requires BEFORE attempting reproduction.
|
||||
|
||||
### Custom Nodes
|
||||
|
||||
If the issue mentions custom nodes:
|
||||
|
||||
```bash
|
||||
# Find the custom node repo
|
||||
# Clone into ComfyUI's custom_nodes directory
|
||||
cd <comfyui_path>/custom_nodes
|
||||
git clone <custom_node_repo_url>
|
||||
|
||||
# Install dependencies if needed
|
||||
cd <custom_node_name>
|
||||
pip install -r requirements.txt 2>/dev/null || true
|
||||
|
||||
# Restart ComfyUI server to load the new nodes
|
||||
```
|
||||
|
||||
### Workflows
|
||||
|
||||
If the issue references a specific workflow:
|
||||
|
||||
```bash
|
||||
# Download workflow JSON if a URL is provided
|
||||
curl -L "<workflow_url>" -o /tmp/test-workflow.json
|
||||
|
||||
# Load it via the API
|
||||
curl -X POST http://127.0.0.1:8188/api/workflow \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @/tmp/test-workflow.json
|
||||
```
|
||||
|
||||
Or load via playwright-cli:
|
||||
|
||||
```bash
|
||||
playwright-cli goto "http://127.0.0.1:8188"
|
||||
# Drag-and-drop or use File > Open to load the workflow
|
||||
```
|
||||
|
||||
### Settings
|
||||
|
||||
If the issue requires specific settings:
|
||||
|
||||
```bash
|
||||
# Use playwright-cli to open settings and change them
|
||||
playwright-cli press "Control+,"
|
||||
playwright-cli snapshot
|
||||
# Find and modify the relevant setting
|
||||
```
|
||||
|
||||
## Phase 3: Interactive Exploration — Research Video
|
||||
|
||||
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
|
||||
|
||||
```bash
|
||||
# Open browser and start video recording
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
|
||||
# Take a snapshot to see current state
|
||||
playwright-cli snapshot
|
||||
|
||||
# Interact based on what you see
|
||||
playwright-cli click <ref>
|
||||
playwright-cli fill <ref> "text"
|
||||
playwright-cli press "Control+s"
|
||||
|
||||
# Check results
|
||||
playwright-cli snapshot
|
||||
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
|
||||
```
|
||||
|
||||
### Key Principles
|
||||
|
||||
- **Observe before acting**: Always `snapshot` before interacting
|
||||
- **Retry and adapt**: If a step fails, try a different approach
|
||||
- **Document what works**: Keep notes on which steps trigger the bug
|
||||
- **Don't give up**: Try multiple approaches if the first doesn't work
|
||||
- **Establish prerequisites**: Many bugs require specific UI state:
|
||||
- Save a workflow first (File > Save)
|
||||
- Make changes to dirty the workflow
|
||||
- Open multiple tabs
|
||||
- Add specific node types
|
||||
- Change settings
|
||||
- Resize the window
|
||||
|
||||
### Common ComfyUI Interactions via playwright-cli
|
||||
|
||||
| Action | Command |
|
||||
| ------------------- | -------------------------------------------------------------- |
|
||||
| Open hamburger menu | `playwright-cli click` on the C logo button |
|
||||
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
|
||||
| Add node | Double-click canvas → type node name → select from results |
|
||||
| Connect nodes | Drag from output slot to input slot |
|
||||
| Save workflow | `playwright-cli press "Control+s"` |
|
||||
| Save As | Menu > File > Save As |
|
||||
| Select node | Click on the node |
|
||||
| Delete node | Select → `playwright-cli press "Delete"` |
|
||||
| Right-click menu | `playwright-cli click <ref> --button right` |
|
||||
| Keyboard shortcut | `playwright-cli press "Control+z"` |
|
||||
|
||||
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
|
||||
|
||||
Once the bug is confirmed, **stop the research video** and **close the research browser**:
|
||||
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
Now start a **fresh browser session** for the clean reproduce video (Video 2).
|
||||
|
||||
**IMPORTANT constraints:**
|
||||
|
||||
- **Max 5 minutes** — the reproduce video must be short and focused
|
||||
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
|
||||
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
|
||||
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
|
||||
|
||||
1. **Open browser and start recording**:
|
||||
|
||||
```bash
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
```
|
||||
|
||||
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
|
||||
|
||||
3. **Take key screenshots** at critical moments:
|
||||
|
||||
```bash
|
||||
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
|
||||
# ... trigger the bug ...
|
||||
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
|
||||
```
|
||||
|
||||
4. **Stop recording and close** immediately after the bug is demonstrated:
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
## Phase 5: Generate Report
|
||||
|
||||
Create a reproduction report at `tmp/qa/reproduce-report.md`:
|
||||
|
||||
```markdown
|
||||
# Issue Reproduction Report
|
||||
|
||||
- **Issue**: <issue_url>
|
||||
- **Title**: <issue_title>
|
||||
- **Date**: <today>
|
||||
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
|
||||
|
||||
## Environment
|
||||
|
||||
- ComfyUI Server: <url>
|
||||
- OS: <os>
|
||||
- Custom Nodes Installed: <list or "none">
|
||||
- Settings Changed: <list or "none">
|
||||
|
||||
## Prerequisites
|
||||
|
||||
List everything that had to be set up before the bug could be triggered:
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
## Reproduction Steps
|
||||
|
||||
Minimal steps to reproduce (the clean sequence):
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
3. ...
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
<from the issue>
|
||||
|
||||
## Actual Behavior
|
||||
|
||||
<what actually happened>
|
||||
|
||||
## Evidence
|
||||
|
||||
- Research video: `research-video/video.webm` (full exploration session)
|
||||
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
|
||||
- Screenshots: `before-bug.png`, `bug-visible.png`
|
||||
|
||||
## Root Cause Analysis (if identified)
|
||||
|
||||
<code pointers, hypothesis about what's going wrong>
|
||||
|
||||
## Notes
|
||||
|
||||
<any additional observations, workarounds discovered, related issues>
|
||||
```
|
||||
|
||||
## Handling Failures
|
||||
|
||||
If the bug **cannot be reproduced**:
|
||||
|
||||
1. Document what you tried and why it didn't work
|
||||
2. Check if the issue was already fixed (search git log for related commits)
|
||||
3. Check if it's environment-specific (OS, browser, specific version)
|
||||
4. Set report status to "Not Reproduced" with detailed notes
|
||||
5. The report is still valuable — it saves others from repeating the same investigation
|
||||
|
||||
## CI Integration
|
||||
|
||||
In CI, this skill runs as a Claude Code agent with:
|
||||
|
||||
- `ANTHROPIC_API_KEY` for Claude
|
||||
- `GEMINI_API_KEY` for initial issue analysis (optional)
|
||||
- ComfyUI server pre-started in the container
|
||||
- `playwright-cli` pre-installed
|
||||
|
||||
The CI workflow:
|
||||
|
||||
1. Gemini generates a reproduce guide (markdown) from the issue
|
||||
2. Claude agent receives the guide and runs this skill
|
||||
3. Claude explores interactively, installs dependencies, retries
|
||||
4. Claude records a clean demo once reproduced
|
||||
5. Video and report are uploaded as artifacts
|
||||
@@ -1,133 +0,0 @@
|
||||
---
|
||||
name: comfy-qa
|
||||
description: 'Comprehensive QA of ComfyUI frontend. Reproduces bugs via E2E tests, records narrated demo videos, deploys reports. Works in CI and locally via `pnpm qa`.'
|
||||
---
|
||||
|
||||
# ComfyUI Frontend QA Skill
|
||||
|
||||
Automated quality assurance pipeline that reproduces reported bugs using Playwright E2E tests, records narrated demo videos with demowright, and deploys reports to Cloudflare Pages.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Reproduce an issue
|
||||
pnpm qa 10253
|
||||
|
||||
# Test a PR
|
||||
pnpm qa 10270
|
||||
|
||||
# Test PR base (reproduce bug)
|
||||
pnpm qa 10270 -t base
|
||||
|
||||
# Test both base + head
|
||||
pnpm qa 10270 -t both
|
||||
|
||||
# Test local uncommitted changes
|
||||
pnpm qa --uncommitted
|
||||
|
||||
# Help
|
||||
pnpm qa -h
|
||||
```
|
||||
|
||||
Auto-loads `.env.local` / `.env` for `GEMINI_API_KEY` and `ANTHROPIC_API_KEY`.
|
||||
Auto-detects issue vs PR via GitHub API. Auto-starts ComfyUI if not running.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three-phase pipeline:
|
||||
|
||||
1. **RESEARCH** — Claude Sonnet 4.6 (Agent SDK) writes Playwright E2E tests to reproduce bugs
|
||||
2. **RECORD** — Re-runs test with demowright for narrated demo video (title cards, TTS, subtitles)
|
||||
3. **REPORT** — Gemini reviews video, deploys to Cloudflare Pages with badge + verdict
|
||||
|
||||
Key principle: **Playwright assertions are truth** — no AI hallucination. If the test passes, the bug is proven.
|
||||
|
||||
## Pipeline Scripts
|
||||
|
||||
All scripts live in `.claude/skills/comfy-qa/scripts/`:
|
||||
|
||||
| Script | Role |
|
||||
| ------------------------- | ----------------------------------------------------- |
|
||||
| `qa.ts` | CLI entry point (`pnpm qa`) |
|
||||
| `qa-agent.ts` | Research phase: Claude writes E2E tests via Agent SDK |
|
||||
| `qa-record.ts` | Orchestrator: 3-phase pipeline |
|
||||
| `qa-deploy-pages.sh` | Cloudflare Pages deploy + badge generation |
|
||||
| `qa-report-template.html` | Report site template |
|
||||
| `qa-video-review.ts` | Gemini video review |
|
||||
| `qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide |
|
||||
|
||||
## Triggering QA
|
||||
|
||||
### Via CLI (`pnpm qa`)
|
||||
|
||||
```bash
|
||||
pnpm qa 10253 # issue (auto-detect)
|
||||
pnpm qa 10270 # PR head
|
||||
pnpm qa 10270 -t base # PR base
|
||||
pnpm qa 10270 -t both # both
|
||||
pnpm qa --uncommitted # local changes
|
||||
```
|
||||
|
||||
### Via GitHub Labels
|
||||
|
||||
- **`qa-issue`** — Reproduce a bug from an issue
|
||||
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after)
|
||||
- **`qa-full`** — Full QA (3-OS matrix)
|
||||
|
||||
### Via Push to trigger branches
|
||||
|
||||
```bash
|
||||
git push origin sno-skills:sno-qa-10253 --force
|
||||
```
|
||||
|
||||
## Research Phase (`qa-agent.ts`)
|
||||
|
||||
Claude receives the issue/PR context + a11y tree snapshot + ComfyPage fixture API docs.
|
||||
|
||||
Tools:
|
||||
|
||||
- **`inspect(selector?)`** — Read a11y tree
|
||||
- **`readFixture(path)`** — Read fixture source code
|
||||
- **`readTest(path)`** — Read existing tests for patterns
|
||||
- **`writeTest(code)`** — Write a Playwright .spec.ts
|
||||
- **`runTest()`** — Execute and get pass/fail + errors
|
||||
- **`done(verdict, summary, evidence, testCode, videoScript?)`** — Finish
|
||||
|
||||
When `verdict=REPRODUCED`, Claude also provides a `videoScript` — a separate test file using demowright's `createVideoScript()` for professional narrated demo video with title cards, TTS segments, and outro.
|
||||
|
||||
## Video Recording (demowright)
|
||||
|
||||
Phase 2 uses the video script to record with:
|
||||
|
||||
- `showTitleCard()` / `hideTitleCard()` — covers setup/loading screen
|
||||
- `createVideoScript().title().segment().outro()` — structured narration
|
||||
- `pace()` — narration-then-action timing
|
||||
- TTS audio + subtitles + cursor overlay + key badges
|
||||
|
||||
## Report Site
|
||||
|
||||
Deployed to `https://sno-qa-{number}.comfy-qa.pages.dev/`
|
||||
|
||||
Features:
|
||||
|
||||
- Video player (1x default, adjustable speed)
|
||||
- Research log (verdict, tool calls, timing)
|
||||
- E2E test code + video script code
|
||||
- Verdict banner for NOT_REPRODUCIBLE/INCONCLUSIVE with failure reason
|
||||
- Copy badge button (markdown)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `GEMINI_API_KEY` — video review, TTS
|
||||
- `ANTHROPIC_API_KEY` — Claude Agent SDK (research phase)
|
||||
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — report deployment (CI only)
|
||||
- ComfyUI server running (auto-detected, or auto-started)
|
||||
|
||||
## CI Workflow (`.github/workflows/pr-qa.yaml`)
|
||||
|
||||
```
|
||||
resolve-matrix → analyze-pr ──┐
|
||||
├→ qa-before (main branch)
|
||||
├→ qa-after (PR branch)
|
||||
└→ report (video review, deploy, comment)
|
||||
```
|
||||
@@ -1,673 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
|
||||
*
|
||||
* Instead of driving a browser interactively, Claude:
|
||||
* 1. Reads the issue + a11y snapshot of the UI
|
||||
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
|
||||
* 3. Runs the test → reads errors → rewrites → repeats until it works
|
||||
* 4. Outputs the passing test + verdict
|
||||
*
|
||||
* Tools:
|
||||
* - inspect(selector) — read a11y tree to understand UI state
|
||||
* - writeTest(code) — write a Playwright test file
|
||||
* - runTest() — execute the test and get results
|
||||
* - done(verdict, summary, testCode) — finish with the working test
|
||||
*/
|
||||
|
||||
import type { Page } from '@playwright/test'
|
||||
/* eslint-disable import-x/no-unresolved */
|
||||
// @ts-expect-error — claude-agent-sdk has no type declarations for vue-tsc
|
||||
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
|
||||
/* eslint-enable import-x/no-unresolved */
|
||||
import { z } from 'zod'
|
||||
import { mkdirSync, readFileSync, writeFileSync } from 'fs'
|
||||
import { execSync } from 'child_process'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface ResearchOptions {
|
||||
page: Page
|
||||
issueContext: string
|
||||
qaGuide: string
|
||||
outputDir: string
|
||||
serverUrl: string
|
||||
anthropicApiKey?: string
|
||||
maxTurns?: number
|
||||
timeBudgetMs?: number
|
||||
}
|
||||
|
||||
export type ReproMethod = 'e2e_test' | 'video' | 'both' | 'none'
|
||||
|
||||
export interface ResearchResult {
|
||||
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
|
||||
reproducedBy: ReproMethod
|
||||
summary: string
|
||||
evidence: string
|
||||
testCode: string
|
||||
videoScript?: string
|
||||
log: Array<{
|
||||
turn: number
|
||||
timestampMs: number
|
||||
toolName: string
|
||||
toolInput: unknown
|
||||
toolResult: string
|
||||
}>
|
||||
}
|
||||
|
||||
// ── Main research function ──
|
||||
|
||||
export async function runResearchPhase(
|
||||
opts: ResearchOptions
|
||||
): Promise<ResearchResult> {
|
||||
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
|
||||
opts
|
||||
const maxTurns = opts.maxTurns ?? 50
|
||||
|
||||
let agentDone = false
|
||||
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
|
||||
let finalReproducedBy: ReproMethod = 'none'
|
||||
let finalSummary = 'Agent did not complete'
|
||||
let finalEvidence = ''
|
||||
let finalTestCode = ''
|
||||
let finalVideoScript = ''
|
||||
let turnCount = 0
|
||||
let lastPassedTurn = -1
|
||||
const startTime = Date.now()
|
||||
const researchLog: ResearchResult['log'] = []
|
||||
|
||||
const testDir = `${outputDir}/research`
|
||||
mkdirSync(testDir, { recursive: true })
|
||||
const testPath = `${testDir}/reproduce.spec.ts`
|
||||
|
||||
// Get initial a11y snapshot for context
|
||||
let initialA11y = ''
|
||||
try {
|
||||
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
|
||||
initialA11y = initialA11y.slice(0, 3000)
|
||||
} catch {
|
||||
initialA11y = '(could not capture initial a11y snapshot)'
|
||||
}
|
||||
|
||||
// ── Tool: inspect ──
|
||||
const inspectTool = tool(
|
||||
'inspect',
|
||||
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
|
||||
{
|
||||
selector: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Optional filter — only show elements matching this name/role. Omit for full tree.'
|
||||
)
|
||||
},
|
||||
async (args: { selector?: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const ariaText = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 5000 })
|
||||
if (args.selector) {
|
||||
const lines = ariaText.split('\n')
|
||||
const matches = lines.filter((l: string) =>
|
||||
l.toLowerCase().includes(args.selector!.toLowerCase())
|
||||
)
|
||||
resultText =
|
||||
matches.length > 0
|
||||
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
|
||||
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
|
||||
} else {
|
||||
resultText = ariaText.slice(0, 3000)
|
||||
}
|
||||
} catch (e) {
|
||||
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'inspect',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: readFixture ──
|
||||
const readFixtureTool = tool(
|
||||
'readFixture',
|
||||
'Read a fixture or helper file from browser_tests/fixtures/ to understand the API. Use this to discover available methods on comfyPage helpers before writing your test.',
|
||||
{
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
'Relative path within browser_tests/fixtures/, e.g. "helpers/CanvasHelper.ts" or "components/Topbar.ts" or "ComfyPage.ts"'
|
||||
)
|
||||
},
|
||||
async (args: { path: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const fullPath = `${projectRoot}/browser_tests/fixtures/${args.path}`
|
||||
const content = readFileSync(fullPath, 'utf-8')
|
||||
resultText = content.slice(0, 4000)
|
||||
if (content.length > 4000) {
|
||||
resultText += `\n\n... (truncated, ${content.length} total chars)`
|
||||
}
|
||||
} catch (e) {
|
||||
resultText = `Could not read fixture: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'readFixture',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: readTest ──
|
||||
const readTestTool = tool(
|
||||
'readTest',
|
||||
'Read an existing E2E test file from browser_tests/tests/ to learn patterns and conventions used in this project.',
|
||||
{
|
||||
path: z
|
||||
.string()
|
||||
.describe(
|
||||
'Relative path within browser_tests/tests/, e.g. "workflow.spec.ts" or "subgraph.spec.ts"'
|
||||
)
|
||||
},
|
||||
async (args: { path: string }) => {
|
||||
let resultText: string
|
||||
try {
|
||||
const fullPath = `${projectRoot}/browser_tests/tests/${args.path}`
|
||||
const content = readFileSync(fullPath, 'utf-8')
|
||||
resultText = content.slice(0, 4000)
|
||||
if (content.length > 4000) {
|
||||
resultText += `\n\n... (truncated, ${content.length} total chars)`
|
||||
}
|
||||
} catch (e) {
|
||||
// List available test files if the path doesn't exist
|
||||
try {
|
||||
const { readdirSync } = await import('fs')
|
||||
const files = readdirSync(`${projectRoot}/browser_tests/tests/`)
|
||||
.filter((f: string) => f.endsWith('.spec.ts'))
|
||||
.slice(0, 30)
|
||||
resultText = `File not found: ${args.path}\n\nAvailable test files:\n${files.join('\n')}`
|
||||
} catch {
|
||||
resultText = `Could not read test: ${e instanceof Error ? e.message : e}`
|
||||
}
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'readTest',
|
||||
toolInput: args,
|
||||
toolResult: resultText.slice(0, 500)
|
||||
})
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: writeTest ──
|
||||
const writeTestTool = tool(
|
||||
'writeTest',
|
||||
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
|
||||
{
|
||||
code: z
|
||||
.string()
|
||||
.describe('Complete Playwright test file content (.spec.ts)')
|
||||
},
|
||||
async (args: { code: string }) => {
|
||||
writeFileSync(testPath, args.code)
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'writeTest',
|
||||
toolInput: { path: testPath, codeLength: args.code.length },
|
||||
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
|
||||
})
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text' as const,
|
||||
text: `Test written to ${testPath}. Use runTest() to execute it.`
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: runTest ──
|
||||
// Place test in browser_tests/ so Playwright config finds fixtures
|
||||
const projectRoot = process.cwd()
|
||||
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
|
||||
|
||||
const runTestTool = tool(
|
||||
'runTest',
|
||||
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
|
||||
{},
|
||||
async () => {
|
||||
turnCount++
|
||||
// Copy the test to browser_tests/tests/ where Playwright expects it
|
||||
const { copyFileSync } = await import('fs')
|
||||
try {
|
||||
copyFileSync(testPath, browserTestPath)
|
||||
} catch {
|
||||
// directory may not exist
|
||||
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
|
||||
copyFileSync(testPath, browserTestPath)
|
||||
}
|
||||
|
||||
let resultText: string
|
||||
try {
|
||||
const output = execSync(
|
||||
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
|
||||
{
|
||||
timeout: 90000,
|
||||
encoding: 'utf-8',
|
||||
env: {
|
||||
...process.env,
|
||||
COMFYUI_BASE_URL: serverUrl
|
||||
}
|
||||
}
|
||||
)
|
||||
resultText = `TEST PASSED:\n${output.slice(-1500)}`
|
||||
} catch (e) {
|
||||
const err = e as { stdout?: string; stderr?: string; message?: string }
|
||||
const output = (err.stdout || '') + '\n' + (err.stderr || '')
|
||||
resultText = `TEST FAILED:\n${output.slice(-2000)}`
|
||||
}
|
||||
|
||||
researchLog.push({
|
||||
turn: turnCount,
|
||||
timestampMs: Date.now() - startTime,
|
||||
toolName: 'runTest',
|
||||
toolInput: { testPath },
|
||||
toolResult: resultText.slice(0, 1000)
|
||||
})
|
||||
|
||||
// Auto-save passing test code for fallback completion — but only if
|
||||
// the test contains a bug-specific assertion (not just a discovery/debug test)
|
||||
if (resultText.startsWith('TEST PASSED')) {
|
||||
try {
|
||||
const code = readFileSync(browserTestPath, 'utf-8')
|
||||
const hasBugAssertion =
|
||||
/expect\s*\(/.test(code) &&
|
||||
!/^\s*expect\([^)]+\)\.toBeDefined\(\)/m.test(code) &&
|
||||
!/^\s*expect\([^)]+\)\.toBeGreaterThan\(0\)/m.test(code) &&
|
||||
!/Inspect|Find|Debug|discover/i.test(
|
||||
code.match(/test\(['"`]([^'"`]+)/)?.[1] ?? ''
|
||||
)
|
||||
if (hasBugAssertion) {
|
||||
finalTestCode = code
|
||||
lastPassedTurn = turnCount
|
||||
}
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
resultText +=
|
||||
'\n\n⚠️ Test PASSED — call done() now with verdict REPRODUCED and the test code. Do NOT write more tests.'
|
||||
}
|
||||
|
||||
return { content: [{ type: 'text' as const, text: resultText }] }
|
||||
}
|
||||
)
|
||||
|
||||
// ── Tool: done ──
|
||||
const doneTool = tool(
|
||||
'done',
|
||||
'Finish research with verdict and the final test code.',
|
||||
{
|
||||
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
|
||||
reproducedBy: z
|
||||
.enum(['e2e_test', 'video', 'both', 'none'])
|
||||
.describe(
|
||||
'How the bug was proven: e2e_test = Playwright assertion passed, video = visual evidence only, both = both methods, none = not reproduced'
|
||||
),
|
||||
summary: z.string().describe('What you found and why'),
|
||||
evidence: z.string().describe('Test output that proves the verdict'),
|
||||
testCode: z
|
||||
.string()
|
||||
.describe(
|
||||
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
|
||||
),
|
||||
videoScript: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'Demowright video script for Phase 2 demo recording. REQUIRED when verdict is REPRODUCED. A separate test file using createVideoScript with title, segments, and outro. Do NOT include demowright imports in testCode.'
|
||||
)
|
||||
},
|
||||
async (args: {
|
||||
verdict: ResearchResult['verdict']
|
||||
reproducedBy: ReproMethod
|
||||
summary: string
|
||||
evidence: string
|
||||
testCode: string
|
||||
videoScript?: string
|
||||
}) => {
|
||||
agentDone = true
|
||||
finalVerdict = args.verdict
|
||||
finalReproducedBy = args.reproducedBy
|
||||
finalSummary = args.summary
|
||||
finalEvidence = args.evidence
|
||||
finalTestCode = args.testCode
|
||||
finalVideoScript = args.videoScript ?? ''
|
||||
writeFileSync(testPath, args.testCode)
|
||||
if (args.videoScript) {
|
||||
writeFileSync(`${outputDir}/video-script.spec.ts`, args.videoScript)
|
||||
}
|
||||
return {
|
||||
content: [
|
||||
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
|
||||
]
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
// ── MCP Server ──
|
||||
const server = createSdkMcpServer({
|
||||
name: 'qa-research',
|
||||
version: '1.0.0',
|
||||
tools: [
|
||||
inspectTool,
|
||||
readFixtureTool,
|
||||
readTestTool,
|
||||
writeTestTool,
|
||||
runTestTool,
|
||||
doneTool
|
||||
]
|
||||
})
|
||||
|
||||
// ── System prompt ──
|
||||
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
|
||||
|
||||
## Your tools
|
||||
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
|
||||
- readFixture(path) — Read fixture source code from browser_tests/fixtures/. Use to discover available methods. E.g. "helpers/CanvasHelper.ts", "components/Topbar.ts", "ComfyPage.ts"
|
||||
- readTest(path) — Read an existing test from browser_tests/tests/ to learn patterns. E.g. "workflow.spec.ts". Pass any name to list available files.
|
||||
- writeTest(code) — Write a Playwright test file (.spec.ts)
|
||||
- runTest() — Execute the test and get results (pass/fail + errors)
|
||||
- done(verdict, summary, evidence, testCode) — Finish with the final test
|
||||
|
||||
## Workflow
|
||||
1. Read the issue description carefully
|
||||
2. Use inspect() to understand the current UI state and discover element selectors
|
||||
3. If unsure about the fixture API, use readFixture() to read the relevant helper source code
|
||||
4. If unsure about test patterns, use readTest() to read an existing test for reference
|
||||
5. Write a Playwright test that:
|
||||
- Performs the exact reproduction steps from the issue
|
||||
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
|
||||
6. Run the test with runTest()
|
||||
7. If it fails: read the error, fix the test, run again (max 5 attempts)
|
||||
8. Call done() with the final verdict and test code
|
||||
|
||||
## Test writing guidelines
|
||||
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
|
||||
- Import expect: \`import { expect } from '@playwright/test'\`
|
||||
- The fixture provides \`comfyPage\` which has all the helpers listed below
|
||||
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
|
||||
- Keep tests focused and minimal — test ONLY the reported bug
|
||||
- Write ONE test, not multiple. Focus on the single clearest reproduction.
|
||||
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
|
||||
- Use \`comfyPage.nextFrame()\` after interactions that trigger UI updates
|
||||
- NEVER use \`page.waitForTimeout()\` — use Locator actions and retrying assertions instead
|
||||
- ALWAYS call done() when finished, even if the test passed — do not keep iterating after a passing test
|
||||
- Use \`expect.poll()\` for async assertions: \`await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(8)\`
|
||||
- CRITICAL: Your assertions must be SPECIFIC TO THE BUG. A test that asserts \`expect(count).toBeGreaterThan(0)\` proves nothing — it would pass even without the bug. Instead assert the exact broken state, e.g. \`expect(clonedWidgets).toHaveLength(0)\` (missing widgets) or \`expect(zIndex).toBeLessThan(parentZIndex)\` (wrong z-order). If a test passes trivially, it's a false positive.
|
||||
- NEVER write "debug", "discovery", or "inspect node types" tests. These waste turns and produce false REPRODUCED verdicts. If you need to discover node type names, use inspect() or readFixture() — not a passing test.
|
||||
- If you cannot write a bug-specific assertion, call done() with verdict NOT_REPRODUCIBLE and explain why.
|
||||
|
||||
## ComfyPage Fixture API Reference
|
||||
|
||||
### Core properties
|
||||
- \`comfyPage.page\` — raw Playwright Page
|
||||
- \`comfyPage.canvas\` — Locator for #graph-canvas
|
||||
- \`comfyPage.queueButton\` — "Queue Prompt" button
|
||||
- \`comfyPage.runButton\` — "Run" button (new UI)
|
||||
- \`comfyPage.confirmDialog\` — ConfirmDialog (has .confirm, .delete, .overwrite, .reject locators + .click(name) method)
|
||||
- \`comfyPage.nextFrame()\` — wait for next requestAnimationFrame
|
||||
- \`comfyPage.setup()\` — navigate + wait for app ready (called automatically by fixture)
|
||||
|
||||
### Menu (comfyPage.menu)
|
||||
- \`comfyPage.menu.topbar\` — Topbar helper:
|
||||
- \`.triggerTopbarCommand(['File', 'Save As'])\` — navigate menu hierarchy
|
||||
- \`.openTopbarMenu()\` / \`.closeTopbarMenu()\` — open/close hamburger
|
||||
- \`.openSubmenu('File')\` — hover to open submenu, returns submenu Locator
|
||||
- \`.getTabNames()\` — get all open workflow tab names
|
||||
- \`.getActiveTabName()\` — get active tab name
|
||||
- \`.getWorkflowTab(name)\` — get tab Locator
|
||||
- \`.closeWorkflowTab(name)\` — close a tab
|
||||
- \`.saveWorkflow(name)\` / \`.saveWorkflowAs(name)\` / \`.exportWorkflow(name)\`
|
||||
- \`.switchTheme('dark' | 'light')\`
|
||||
- \`comfyPage.menu.workflowsTab\` — WorkflowsSidebarTab:
|
||||
- \`.open()\` / \`.close()\` — toggle workflows sidebar
|
||||
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflow names
|
||||
- \`comfyPage.menu.nodeLibraryTab\` — NodeLibrarySidebarTab
|
||||
- \`comfyPage.menu.assetsTab\` — AssetsSidebarTab
|
||||
|
||||
### Canvas (comfyPage.canvasOps)
|
||||
- \`.click({x, y})\` — click at position on canvas
|
||||
- \`.rightClick(x, y)\` — right-click (opens context menu)
|
||||
- \`.doubleClick()\` — double-click canvas (opens node search)
|
||||
- \`.clickEmptySpace()\` — click known empty area
|
||||
- \`.dragAndDrop(source, target)\` — drag from source to target position
|
||||
- \`.pan(offset, safeSpot?)\` — pan canvas by offset
|
||||
- \`.zoom(deltaY, steps?)\` — zoom via scroll wheel
|
||||
- \`.resetView()\` — reset zoom/pan to default
|
||||
- \`.getScale()\` / \`.setScale(n)\` — get/set canvas zoom
|
||||
- \`.getNodeCenterByTitle(title)\` — get screen coords of node center
|
||||
- \`.disconnectEdge()\` / \`.connectEdge()\` — default graph edge operations
|
||||
|
||||
### Node Operations (comfyPage.nodeOps)
|
||||
- \`.getGraphNodesCount()\` — count all nodes
|
||||
- \`.getSelectedGraphNodesCount()\` — count selected nodes
|
||||
- \`.getNodes()\` — get all nodes
|
||||
- \`.getFirstNodeRef()\` — get NodeReference for first node
|
||||
- \`.getNodeRefById(id)\` — get NodeReference by ID
|
||||
- \`.getNodeRefsByType(type)\` — get all nodes of a type
|
||||
- \`.waitForGraphNodes(count)\` — wait until node count matches
|
||||
|
||||
### Settings (comfyPage.settings)
|
||||
- \`.setSetting(id, value)\` — change a ComfyUI setting
|
||||
- \`.getSetting(id)\` — read current setting value
|
||||
|
||||
### Keyboard (comfyPage.keyboard)
|
||||
- \`.undo()\` / \`.redo()\` — Ctrl+Z / Ctrl+Y
|
||||
- \`.bypass()\` — Ctrl+B
|
||||
- \`.selectAll()\` — Ctrl+A
|
||||
- \`.ctrlSend(key)\` — send Ctrl+key
|
||||
|
||||
### Workflow (comfyPage.workflow)
|
||||
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
|
||||
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
|
||||
- \`.deleteWorkflow(name)\`
|
||||
- \`.isCurrentWorkflowModified()\` — check dirty state
|
||||
|
||||
### Context Menu (comfyPage.contextMenu)
|
||||
- \`.openFor(locator)\` — right-click locator and wait for menu
|
||||
- \`.clickMenuItem(name)\` — click a menu item by name
|
||||
- \`.isVisible()\` — check if context menu is showing
|
||||
- \`.assertHasItems(items)\` — assert menu contains items
|
||||
|
||||
### Other helpers
|
||||
- \`comfyPage.settingDialog\` — SettingDialog component
|
||||
- \`comfyPage.searchBox\` / \`comfyPage.searchBoxV2\` — node search
|
||||
- \`comfyPage.toast\` — ToastHelper (\`.visibleToasts\`)
|
||||
- \`comfyPage.subgraph\` — SubgraphHelper
|
||||
- \`comfyPage.vueNodes\` — VueNodeHelpers
|
||||
- \`comfyPage.bottomPanel\` — BottomPanel
|
||||
- \`comfyPage.clipboard\` — ClipboardHelper
|
||||
- \`comfyPage.dragDrop\` — DragDropHelper
|
||||
|
||||
### Available fixture files (use readFixture to explore)
|
||||
- ComfyPage.ts — main fixture with all helpers
|
||||
- helpers/CanvasHelper.ts, NodeOperationsHelper.ts, WorkflowHelper.ts
|
||||
- helpers/KeyboardHelper.ts, SettingsHelper.ts, SubgraphHelper.ts
|
||||
- components/Topbar.ts, ContextMenu.ts, SettingDialog.ts, SidebarTab.ts
|
||||
|
||||
## Video Script (IMPORTANT — provide via done() tool)
|
||||
|
||||
When calling done(), provide a \`videoScript\` parameter with a SEPARATE test file that uses demowright's createVideoScript.
|
||||
Do NOT put demowright imports in testCode — they won't resolve in Phase 1.
|
||||
|
||||
The videoScript is a complete, standalone Playwright test file for Phase 2 demo recording:
|
||||
|
||||
\`\`\`typescript
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { createVideoScript, showTitleCard, hideTitleCard } from 'demowright/video-script'
|
||||
|
||||
test('Demo: Bug Title', async ({ comfyPage }) => {
|
||||
// Show title card IMMEDIATELY — covers the screen while setup runs behind it
|
||||
await showTitleCard(comfyPage.page, 'Bug Title Here', { subtitle: 'Issue #NNNN' })
|
||||
|
||||
// Setup runs while title card is visible
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.workflow.setupWorkflowsDirectory({})
|
||||
|
||||
// Remove early title card before script starts (script will show its own)
|
||||
await hideTitleCard(comfyPage.page)
|
||||
|
||||
const script = createVideoScript()
|
||||
.title('Bug Title Here', { subtitle: 'Issue #NNNN', durationMs: 4000 })
|
||||
.segment('Step 1: description of what we do', async (pace) => {
|
||||
await pace() // narration finishes FIRST
|
||||
await comfyPage.menu.topbar.saveWorkflow('name') // THEN action
|
||||
await comfyPage.page.waitForTimeout(2000) // pause for viewer
|
||||
})
|
||||
.segment('Bug evidence: what we see proves the bug', async (pace) => {
|
||||
await pace()
|
||||
await comfyPage.page.waitForTimeout(5000) // hold on evidence
|
||||
})
|
||||
.outro({ text: 'Bug Reproduced', subtitle: 'Summary' })
|
||||
|
||||
await script.render(comfyPage.page)
|
||||
})
|
||||
\`\`\`
|
||||
|
||||
Key API:
|
||||
- \`.title(text, {subtitle?, durationMs?})\` — title card (4s default)
|
||||
- \`.segment(narrationText, async (pace) => { await pace(); ...actions... })\` — TTS narrated step
|
||||
- \`.outro({text?, subtitle?, durationMs?})\` — ending card
|
||||
- \`pace()\` — wait for narration audio to finish
|
||||
|
||||
CRITICAL TIMING: Call \`await pace()\` FIRST in each segment, BEFORE the Playwright actions.
|
||||
This makes the narration play and finish, THEN the actions execute — so viewers hear what's about
|
||||
to happen before it happens. Pattern:
|
||||
|
||||
\`\`\`typescript
|
||||
.segment('Now we save the workflow as a new name', async (pace) => {
|
||||
await pace() // narration finishes first
|
||||
await comfyPage.menu.topbar.saveWorkflowAs('new-name') // then action happens
|
||||
await comfyPage.page.waitForTimeout(2000) // pause so viewer sees the result
|
||||
})
|
||||
\`\`\`
|
||||
|
||||
IMPORTANT RULES for videoScript:
|
||||
1. You MUST provide videoScript when verdict is REPRODUCED — every reproduced bug needs a narrated demo
|
||||
2. Call showTitleCard() BEFORE setup, run setup behind it, call hideTitleCard() before createVideoScript() — see example
|
||||
3. Call \`await pace()\` FIRST in each segment callback, BEFORE actions
|
||||
4. Add \`waitForTimeout(2000)\` after each action so viewers can see the result
|
||||
5. Final evidence segment: hold for 5+ seconds
|
||||
6. Reproduce the same steps as testCode but slower with clear narration
|
||||
|
||||
## Current UI state (accessibility tree)
|
||||
${initialA11y}
|
||||
|
||||
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
|
||||
## Issue to Reproduce
|
||||
${issueContext}`
|
||||
|
||||
// ── Run the agent ──
|
||||
console.warn('Starting research phase (Claude writes E2E tests)...')
|
||||
|
||||
try {
|
||||
for await (const message of query({
|
||||
prompt:
|
||||
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, readFixture() or readTest() if you need to understand the fixture API or see existing test patterns, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
|
||||
options: {
|
||||
model: 'claude-sonnet-4-6',
|
||||
systemPrompt,
|
||||
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
|
||||
maxTurns,
|
||||
mcpServers: { 'qa-research': server },
|
||||
allowedTools: [
|
||||
'mcp__qa-research__inspect',
|
||||
'mcp__qa-research__readFixture',
|
||||
'mcp__qa-research__readTest',
|
||||
'mcp__qa-research__writeTest',
|
||||
'mcp__qa-research__runTest',
|
||||
'mcp__qa-research__done'
|
||||
]
|
||||
}
|
||||
})) {
|
||||
if (message.type === 'assistant' && message.message?.content) {
|
||||
for (const block of message.message.content) {
|
||||
if ('text' in block && block.text) {
|
||||
console.warn(` Claude: ${block.text.slice(0, 200)}`)
|
||||
}
|
||||
if ('name' in block) {
|
||||
console.warn(
|
||||
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (agentDone) break
|
||||
}
|
||||
} catch (e) {
|
||||
const errMsg = e instanceof Error ? e.message : String(e)
|
||||
console.warn(`Research error: ${errMsg}`)
|
||||
|
||||
// Detect billing/auth errors and surface them clearly
|
||||
if (
|
||||
errMsg.includes('Credit balance is too low') ||
|
||||
errMsg.includes('insufficient_quota') ||
|
||||
errMsg.includes('rate_limit')
|
||||
) {
|
||||
finalSummary = `API error: ${errMsg.slice(0, 200)}`
|
||||
finalEvidence = 'Agent could not start due to API billing/auth issue'
|
||||
console.warn(
|
||||
'::error::Anthropic API credits exhausted — cannot run research phase'
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-complete: if a test passed but done() was never called, use the passing test
|
||||
if (!agentDone && lastPassedTurn >= 0 && finalTestCode) {
|
||||
console.warn(
|
||||
`Auto-completing: test passed at turn ${lastPassedTurn} but done() was not called`
|
||||
)
|
||||
finalVerdict = 'REPRODUCED'
|
||||
finalReproducedBy = 'e2e_test'
|
||||
finalSummary = `Test passed at turn ${lastPassedTurn} (auto-completed — agent did not call done())`
|
||||
finalEvidence = `Test passed with exit code 0`
|
||||
}
|
||||
|
||||
const result: ResearchResult = {
|
||||
verdict: finalVerdict,
|
||||
reproducedBy: finalReproducedBy,
|
||||
summary: finalSummary,
|
||||
evidence: finalEvidence,
|
||||
testCode: finalTestCode,
|
||||
videoScript: finalVideoScript || undefined,
|
||||
log: researchLog
|
||||
}
|
||||
|
||||
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
|
||||
console.warn(
|
||||
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
|
||||
)
|
||||
|
||||
return result
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import { extractMediaUrls } from './qa-analyze-pr'
|
||||
|
||||
describe('extractMediaUrls', () => {
|
||||
it('extracts markdown image URLs', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
|
||||
})
|
||||
|
||||
it('extracts multiple markdown images', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Some text',
|
||||
''
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/before.png',
|
||||
'https://example.com/after.jpg'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts raw URLs with media extensions', () => {
|
||||
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
|
||||
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
|
||||
})
|
||||
|
||||
it('extracts GitHub user-attachments URLs', () => {
|
||||
const text =
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts private-user-images URLs', () => {
|
||||
const text =
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts URLs with query parameters', () => {
|
||||
const text = 'https://example.com/image.png?w=800&h=600'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/image.png?w=800&h=600'
|
||||
])
|
||||
})
|
||||
|
||||
it('deduplicates URLs', () => {
|
||||
const text = [
|
||||
'',
|
||||
'',
|
||||
'Also https://example.com/same.png'
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
|
||||
})
|
||||
|
||||
it('returns empty array for empty input', () => {
|
||||
expect(extractMediaUrls('')).toEqual([])
|
||||
})
|
||||
|
||||
it('returns empty array for text with no media URLs', () => {
|
||||
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
|
||||
})
|
||||
|
||||
it('handles mixed media types', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Video: https://example.com/demo.webm',
|
||||
''
|
||||
].join('\n')
|
||||
const urls = extractMediaUrls(text)
|
||||
expect(urls).toContain('https://example.com/screenshot.png')
|
||||
expect(urls).toContain('https://example.com/demo.webm')
|
||||
expect(urls).toContain('https://example.com/animation.gif')
|
||||
})
|
||||
|
||||
it('ignores non-http URLs in markdown', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -1,799 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA PR Analysis Script
|
||||
*
|
||||
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
|
||||
* for before/after recording sessions. Fetches PR thread, extracts media,
|
||||
* and produces structured test plans.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-analyze-pr.ts \
|
||||
* --pr-number 10270 \
|
||||
* --repo owner/repo \
|
||||
* --output-dir qa-guides/ \
|
||||
* [--model gemini-3.1-pro-preview]
|
||||
*
|
||||
* Env: GEMINI_API_KEY (required)
|
||||
*/
|
||||
|
||||
import { execSync } from 'node:child_process'
|
||||
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface QaGuideStep {
|
||||
action: string
|
||||
description: string
|
||||
expected_before?: string
|
||||
expected_after?: string
|
||||
}
|
||||
|
||||
interface QaGuide {
|
||||
summary: string
|
||||
test_focus: string
|
||||
prerequisites: string[]
|
||||
steps: QaGuideStep[]
|
||||
visual_checks: string[]
|
||||
}
|
||||
|
||||
interface PrThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
issueComments: string[]
|
||||
reviewComments: string[]
|
||||
reviews: string[]
|
||||
diff: string
|
||||
}
|
||||
|
||||
type TargetType = 'pr' | 'issue'
|
||||
|
||||
interface Options {
|
||||
prNumber: string
|
||||
repo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
apiKey: string
|
||||
mediaBudgetBytes: number
|
||||
maxVideoBytes: number
|
||||
type: TargetType
|
||||
}
|
||||
|
||||
// ── CLI parsing ──
|
||||
|
||||
function parseArgs(): Options {
|
||||
const args = process.argv.slice(2)
|
||||
const opts: Partial<Options> = {
|
||||
model: 'gemini-3.1-pro-preview',
|
||||
apiKey: process.env.GEMINI_API_KEY || '',
|
||||
mediaBudgetBytes: 20 * 1024 * 1024,
|
||||
maxVideoBytes: 10 * 1024 * 1024,
|
||||
type: 'pr'
|
||||
}
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--pr-number':
|
||||
opts.prNumber = args[++i]
|
||||
break
|
||||
case '--repo':
|
||||
opts.repo = args[++i]
|
||||
break
|
||||
case '--output-dir':
|
||||
opts.outputDir = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--type':
|
||||
opts.type = args[++i] as TargetType
|
||||
break
|
||||
case '--help':
|
||||
console.warn(
|
||||
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
|
||||
)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
|
||||
console.error(
|
||||
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!opts.apiKey) {
|
||||
console.error('GEMINI_API_KEY environment variable is required')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts as Options
|
||||
}
|
||||
|
||||
// ── PR thread fetching ──
|
||||
|
||||
function ghExec(cmd: string): string {
|
||||
try {
|
||||
return execSync(cmd, {
|
||||
encoding: 'utf-8',
|
||||
timeout: 30_000,
|
||||
stdio: ['pipe', 'pipe', 'pipe']
|
||||
}).trim()
|
||||
} catch (err) {
|
||||
console.warn(`gh command failed: ${cmd}`)
|
||||
console.warn((err as Error).message)
|
||||
return ''
|
||||
}
|
||||
}
|
||||
|
||||
function fetchPrThread(prNumber: string, repo: string): PrThread {
|
||||
console.warn('Fetching PR thread...')
|
||||
|
||||
const prView = ghExec(
|
||||
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const prData = prView
|
||||
? JSON.parse(prView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const issueCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
|
||||
)
|
||||
const issueComments: string[] = issueCommentsRaw
|
||||
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
|
||||
)
|
||||
const reviewComments: string[] = reviewCommentsRaw
|
||||
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
|
||||
)
|
||||
const reviews: string[] = reviewsRaw
|
||||
? JSON.parse(reviewsRaw)
|
||||
.filter((r: { body: string }) => r.body)
|
||||
.map((r: { body: string }) => r.body)
|
||||
: []
|
||||
|
||||
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
|
||||
|
||||
console.warn(
|
||||
`PR #${prNumber}: "${prData.title}" | ` +
|
||||
`${issueComments.length} issue comments, ` +
|
||||
`${reviewComments.length} review comments, ` +
|
||||
`${reviews.length} reviews, ` +
|
||||
`diff: ${diff.length} chars`
|
||||
)
|
||||
|
||||
return {
|
||||
title: prData.title || '',
|
||||
body: prData.body || '',
|
||||
labels: (prData.labels || []).map((l: { name: string }) => l.name),
|
||||
issueComments,
|
||||
reviewComments,
|
||||
reviews,
|
||||
diff
|
||||
}
|
||||
}
|
||||
|
||||
interface IssueThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
comments: string[]
|
||||
}
|
||||
|
||||
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
|
||||
console.warn('Fetching issue thread...')
|
||||
|
||||
const issueView = ghExec(
|
||||
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const issueData = issueView
|
||||
? JSON.parse(issueView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const commentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
|
||||
)
|
||||
const comments: string[] = commentsRaw
|
||||
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
console.warn(
|
||||
`Issue #${issueNumber}: "${issueData.title}" | ` +
|
||||
`${comments.length} comments`
|
||||
)
|
||||
|
||||
return {
|
||||
title: issueData.title || '',
|
||||
body: issueData.body || '',
|
||||
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
|
||||
comments
|
||||
}
|
||||
}
|
||||
|
||||
// ── Media extraction ──
|
||||
|
||||
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
|
||||
|
||||
const MEDIA_URL_PATTERNS = [
|
||||
// Markdown images: 
|
||||
/!\[[^\]]*\]\(([^)]+)\)/g,
|
||||
// GitHub user-attachments
|
||||
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
|
||||
// Private user images
|
||||
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
|
||||
// Raw URLs with media extensions (standalone or in text)
|
||||
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
|
||||
]
|
||||
|
||||
export function extractMediaUrls(text: string): string[] {
|
||||
if (!text) return []
|
||||
|
||||
const urls = new Set<string>()
|
||||
|
||||
for (const pattern of MEDIA_URL_PATTERNS) {
|
||||
// Reset lastIndex for global patterns
|
||||
pattern.lastIndex = 0
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = pattern.exec(text)) !== null) {
|
||||
// For markdown images, the URL is in capture group 1
|
||||
const url = match[1] || match[0]
|
||||
// Clean trailing markdown/html artifacts
|
||||
const cleaned = url.replace(/[)>"'\s]+$/, '')
|
||||
if (cleaned.startsWith('http')) {
|
||||
urls.add(cleaned)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [...urls]
|
||||
}
|
||||
|
||||
// ── Media downloading ──
|
||||
|
||||
const ALLOWED_MEDIA_DOMAINS = [
|
||||
'github.com',
|
||||
'raw.githubusercontent.com',
|
||||
'user-images.githubusercontent.com',
|
||||
'private-user-images.githubusercontent.com',
|
||||
'objects.githubusercontent.com',
|
||||
'github.githubassets.com'
|
||||
]
|
||||
|
||||
function isAllowedMediaDomain(url: string): boolean {
|
||||
try {
|
||||
const hostname = new URL(url).hostname
|
||||
return ALLOWED_MEDIA_DOMAINS.some(
|
||||
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
|
||||
)
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadMedia(
|
||||
urls: string[],
|
||||
outputDir: string,
|
||||
budgetBytes: number,
|
||||
maxVideoBytes: number
|
||||
): Promise<Array<{ path: string; mimeType: string }>> {
|
||||
const downloaded: Array<{ path: string; mimeType: string }> = []
|
||||
let totalBytes = 0
|
||||
|
||||
const mediaDir = resolve(outputDir, 'media')
|
||||
mkdirSync(mediaDir, { recursive: true })
|
||||
|
||||
for (const url of urls) {
|
||||
if (totalBytes >= budgetBytes) {
|
||||
console.warn(
|
||||
`Media budget exhausted (${totalBytes} bytes), skipping rest`
|
||||
)
|
||||
break
|
||||
}
|
||||
|
||||
if (!isAllowedMediaDomain(url)) {
|
||||
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
headers: { Accept: 'image/*,video/*' },
|
||||
redirect: 'follow'
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
console.warn(`Failed to download ${url}: ${response.status}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const contentLength = response.headers.get('content-length')
|
||||
if (contentLength) {
|
||||
const declaredSize = Number.parseInt(contentLength, 10)
|
||||
if (declaredSize > budgetBytes - totalBytes) {
|
||||
console.warn(
|
||||
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
|
||||
)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
const contentType = response.headers.get('content-type') || ''
|
||||
const buffer = Buffer.from(await response.arrayBuffer())
|
||||
|
||||
// Skip oversized videos
|
||||
const isVideo =
|
||||
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
|
||||
if (isVideo && buffer.length > maxVideoBytes) {
|
||||
console.warn(
|
||||
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (totalBytes + buffer.length > budgetBytes) {
|
||||
console.warn(`Would exceed budget, skipping ${url}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const ext = guessExtension(url, contentType)
|
||||
const filename = `media-${downloaded.length}${ext}`
|
||||
const filepath = resolve(mediaDir, filename)
|
||||
writeFileSync(filepath, buffer)
|
||||
totalBytes += buffer.length
|
||||
|
||||
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
|
||||
|
||||
downloaded.push({ path: filepath, mimeType })
|
||||
console.warn(
|
||||
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
|
||||
)
|
||||
} catch (err) {
|
||||
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Downloaded ${downloaded.length}/${urls.length} media files ` +
|
||||
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
|
||||
)
|
||||
return downloaded
|
||||
}
|
||||
|
||||
function guessExtension(url: string, contentType: string): string {
|
||||
const urlMatch = url.match(MEDIA_EXTENSIONS)
|
||||
if (urlMatch) return urlMatch[0].toLowerCase()
|
||||
|
||||
const typeMap: Record<string, string> = {
|
||||
'image/png': '.png',
|
||||
'image/jpeg': '.jpg',
|
||||
'image/gif': '.gif',
|
||||
'image/webp': '.webp',
|
||||
'video/mp4': '.mp4',
|
||||
'video/webm': '.webm'
|
||||
}
|
||||
return typeMap[contentType.split(';')[0]] || '.bin'
|
||||
}
|
||||
|
||||
function guessMimeType(ext: string): string {
|
||||
const map: Record<string, string> = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime'
|
||||
}
|
||||
return map[ext] || 'application/octet-stream'
|
||||
}
|
||||
|
||||
// ── Gemini analysis ──
|
||||
|
||||
function buildIssueAnalysisPrompt(issue: IssueThread): string {
|
||||
const allText = [
|
||||
`# Issue: ${issue.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
issue.body,
|
||||
'',
|
||||
issue.comments.length > 0
|
||||
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
|
||||
: ''
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
|
||||
|
||||
The UI has:
|
||||
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
|
||||
- Nodes are boxes with input/output slots connected by wires
|
||||
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
|
||||
- Sidebars (Workflows, Node Library, Models)
|
||||
- A topbar with workflow tabs and Queue button
|
||||
- The default workflow loads with these nodes (approximate center coordinates):
|
||||
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
|
||||
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
|
||||
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
|
||||
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
|
||||
|
||||
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
|
||||
### Menu actions
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
|
||||
### Element actions (by visible text)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "rightClick" — right-clicks an element to open context menu (text required)
|
||||
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
|
||||
|
||||
### Canvas actions (by coordinates — viewport is 1280x720)
|
||||
- "clickCanvas" — click at coordinates (x, y required)
|
||||
- "rightClickCanvas" — right-click at coordinates (x, y required)
|
||||
- "doubleClick" — double-click at coordinates to open node search (x, y)
|
||||
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
|
||||
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
|
||||
|
||||
### Utility
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Common ComfyUI interactions
|
||||
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
|
||||
- Double-click empty canvas → opens node search dialog
|
||||
- Ctrl+C / Ctrl+V → copy/paste selected nodes
|
||||
- Delete key → remove selected node
|
||||
- Ctrl+G → group selected nodes
|
||||
- Drag from output slot to input slot → create connection
|
||||
- Click a node to select it, Shift+click for multi-select
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly one key: "reproduce", containing:
|
||||
{
|
||||
"summary": "One sentence: what bug this issue reports",
|
||||
"test_focus": "Specific behavior to reproduce",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "clickCanvas",
|
||||
"description": "Click on first node to select it",
|
||||
"expected_before": "What should happen if the bug is present"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual evidence of the bug to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
|
||||
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
|
||||
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
|
||||
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
|
||||
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
|
||||
- Do NOT include login steps.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
function buildAnalysisPrompt(thread: PrThread): string {
|
||||
const allText = [
|
||||
`# PR: ${thread.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
thread.body,
|
||||
'',
|
||||
thread.issueComments.length > 0
|
||||
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviewComments.length > 0
|
||||
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviews.length > 0
|
||||
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
'',
|
||||
'## Diff (truncated)',
|
||||
'```',
|
||||
thread.diff.slice(0, 8000),
|
||||
'```'
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
|
||||
|
||||
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly two keys: "before" and "after", each containing:
|
||||
{
|
||||
"summary": "One sentence: what this PR changes",
|
||||
"test_focus": "Specific behaviors to verify in this recording",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "openMenu",
|
||||
"description": "Open the main menu to check file options",
|
||||
"expected_before": "Old behavior description (before key only)",
|
||||
"expected_after": "New behavior description (after key only)"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual elements to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
|
||||
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
|
||||
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
|
||||
- Use information from PR description, screenshots, and comments to understand intended behavior.
|
||||
- Include at least one screenshot step in each guide.
|
||||
- Do NOT include login steps.
|
||||
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
async function analyzeWithGemini(
|
||||
thread: PrThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<{ before: QaGuide; after: QaGuide }> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildAnalysisPrompt(thread)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
// Add media as inline data
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
// Strip markdown fences if present
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
// Handle different response shapes from Gemini
|
||||
let before: QaGuide
|
||||
let after: QaGuide
|
||||
|
||||
if (Array.isArray(parsed) && parsed.length >= 2) {
|
||||
// Array format: [before, after]
|
||||
before = parsed[0]
|
||||
after = parsed[1]
|
||||
} else if (parsed.before && parsed.after) {
|
||||
// Object format: { before, after }
|
||||
before = parsed.before
|
||||
after = parsed.after
|
||||
} else {
|
||||
// Try nested wrapper keys
|
||||
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
|
||||
if (inner.before && inner.after) {
|
||||
before = inner.before
|
||||
after = inner.after
|
||||
} else {
|
||||
console.warn(
|
||||
'Full response:',
|
||||
JSON.stringify(parsed, null, 2).slice(0, 2000)
|
||||
)
|
||||
throw new Error(
|
||||
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
return { before, after }
|
||||
}
|
||||
|
||||
async function analyzeIssueWithGemini(
|
||||
issue: IssueThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<QaGuide> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildIssueAnalysisPrompt(issue)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
const guide: QaGuide =
|
||||
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
|
||||
return guide
|
||||
}
|
||||
|
||||
// ── Main ──
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
mkdirSync(opts.outputDir, { recursive: true })
|
||||
|
||||
if (opts.type === 'issue') {
|
||||
await analyzeIssue(opts)
|
||||
} else {
|
||||
await analyzePr(opts)
|
||||
}
|
||||
}
|
||||
|
||||
async function analyzeIssue(opts: Options) {
|
||||
const issue = fetchIssueThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [issue.body, ...issue.comments].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guide = await analyzeIssueWithGemini(
|
||||
issue,
|
||||
media,
|
||||
opts.model,
|
||||
opts.apiKey
|
||||
)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guide:`)
|
||||
console.warn(` Reproduce: ${beforePath}`)
|
||||
}
|
||||
|
||||
async function analyzePr(opts: Options) {
|
||||
const thread = fetchPrThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [
|
||||
thread.body,
|
||||
...thread.issueComments,
|
||||
...thread.reviewComments,
|
||||
...thread.reviews
|
||||
].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
|
||||
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guides:`)
|
||||
console.warn(` Before: ${beforePath}`)
|
||||
console.warn(` After: ${afterPath}`)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
main().catch((err) => {
|
||||
console.error('PR analysis failed:', err)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
@@ -1,413 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deploy QA report to Cloudflare Pages.
|
||||
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
|
||||
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
|
||||
# Writes outputs to GITHUB_OUTPUT: badge_status, url
|
||||
set -euo pipefail
|
||||
|
||||
npm install -g wrangler@4.74.0 >/dev/null 2>&1
|
||||
|
||||
DEPLOY_DIR=$(mktemp -d)
|
||||
mkdir -p "$DEPLOY_DIR"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
|
||||
for prefix in qa qa-before; do
|
||||
VID="${DIR}/${prefix}-session.mp4"
|
||||
if [ -f "$VID" ]; then
|
||||
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
|
||||
cp "$VID" "$DEST"
|
||||
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
|
||||
fi
|
||||
done
|
||||
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
|
||||
for numbered in "$DIR"/qa-session-[0-9].mp4; do
|
||||
[ -f "$numbered" ] || continue
|
||||
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
|
||||
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
|
||||
cp "$numbered" "$DEST"
|
||||
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
|
||||
done
|
||||
# Generate GIF thumbnail from after video (or first pass)
|
||||
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
|
||||
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
|
||||
if [ -f "$THUMB_SRC" ]; then
|
||||
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
|
||||
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
|
||||
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|
||||
|| echo "GIF generation failed for ${os} (non-fatal)"
|
||||
fi
|
||||
done
|
||||
|
||||
# Build video cards and report sections
|
||||
CARDS=""
|
||||
# shellcheck disable=SC2034 # accessed via eval
|
||||
ICONS_Linux="🐧" ICONS_macOS="🍎" ICONS_Windows="🪟"
|
||||
CARD_COUNT=0
|
||||
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
eval "ICON=\$ICONS_${os}"
|
||||
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
|
||||
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
|
||||
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
|
||||
[ "$HAS_AFTER" = "0" ] && continue
|
||||
|
||||
# Collect all reports for this platform (single + multi-pass)
|
||||
REPORT_FILES=""
|
||||
REPORT_LINK=""
|
||||
REPORT_HTML=""
|
||||
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
|
||||
done
|
||||
|
||||
if [ -n "$REPORT_FILES" ]; then
|
||||
# Concatenate all reports into one combined report file
|
||||
COMBINED_MD=""
|
||||
for rpt in $REPORT_FILES; do
|
||||
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
|
||||
RPT_MD=$(sed 's/&/\&/g; s/</\</g; s/>/\>/g' "$rpt")
|
||||
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD} --- "
|
||||
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
|
||||
done
|
||||
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
|
||||
FIRST_BASENAME=$(basename "$FIRST_REPORT")
|
||||
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
|
||||
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
|
||||
fi
|
||||
|
||||
if [ "$HAS_BEFORE" = "1" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
|
||||
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
|
||||
else
|
||||
PASS_VIDEOS=""
|
||||
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
|
||||
[ -f "$pass_vid" ] || continue
|
||||
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
|
||||
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
|
||||
done
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
|
||||
fi
|
||||
CARD_COUNT=$((CARD_COUNT + 1))
|
||||
done
|
||||
|
||||
# Build commit info and target link for the report header
|
||||
COMMIT_HTML=""
|
||||
REPO_URL="https://github.com/${REPO}"
|
||||
if [ -n "${TARGET_NUM:-}" ]; then
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
|
||||
else
|
||||
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
|
||||
fi
|
||||
fi
|
||||
if [ -n "${BEFORE_SHA:-}" ]; then
|
||||
SHORT_BEFORE="${BEFORE_SHA:0:7}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
|
||||
fi
|
||||
if [ -n "${AFTER_SHA:-}" ]; then
|
||||
SHORT_AFTER="${AFTER_SHA:0:7}"
|
||||
AFTER_LABEL="PR"
|
||||
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
|
||||
fi
|
||||
if [ -n "${PIPELINE_SHA:-}" ]; then
|
||||
SHORT_PIPE="${PIPELINE_SHA:0:7}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
|
||||
fi
|
||||
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" · ${COMMIT_HTML}"
|
||||
|
||||
RUN_LINK=""
|
||||
if [ -n "${RUN_URL:-}" ]; then
|
||||
RUN_LINK=" · <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
|
||||
fi
|
||||
|
||||
# Timing info
|
||||
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
|
||||
TIMING_HTML=""
|
||||
if [ -n "${RUN_START_TIME:-}" ]; then
|
||||
TIMING_HTML=" · <span class=sha title='Pipeline timing'>${RUN_START_TIME} → ${DEPLOY_TIME}</span>"
|
||||
fi
|
||||
|
||||
# Generate index.html from template
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
|
||||
|
||||
# Write dynamic content to temp files for safe substitution
|
||||
# Cloudflare Pages _headers file — enable range requests for video seeking
|
||||
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
|
||||
/*.mp4
|
||||
Accept-Ranges: bytes
|
||||
Cache-Control: public, max-age=86400
|
||||
HEADERSEOF
|
||||
|
||||
# Build purpose description from pr-context.txt
|
||||
PURPOSE_HTML=""
|
||||
if [ -f pr-context.txt ]; then
|
||||
# Extract title line and first paragraph of description
|
||||
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt 2>/dev/null | sed 's/^Title: //' || true)
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
PURPOSE_LABEL="Issue #${TARGET_NUM}"
|
||||
PURPOSE_VERB="reports"
|
||||
else
|
||||
PURPOSE_LABEL="PR #${TARGET_NUM}"
|
||||
PURPOSE_VERB="aims to"
|
||||
fi
|
||||
# Get first ~300 chars of description body (after "Description:" line)
|
||||
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt 2>/dev/null | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&/g; s/</\</g; s/>/\>/g' | tr '\n' ' ' | head -c 400 || true)
|
||||
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt 2>/dev/null | sed 's/&/\&/g; s/</\</g; s/>/\>/g' | tr '\n' ' ' | head -c 400 || true)
|
||||
# Build requirements from QA guide JSON
|
||||
REQS_HTML=""
|
||||
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1 || true)
|
||||
if [ -f "$QA_GUIDE" ]; then
|
||||
PREREQS=$(python3 -c "
|
||||
import json, sys, html
|
||||
try:
|
||||
g = json.load(open(sys.argv[1]))
|
||||
prereqs = g.get('prerequisites', [])
|
||||
steps = g.get('steps', [])
|
||||
focus = g.get('test_focus', '')
|
||||
parts = []
|
||||
if focus:
|
||||
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
|
||||
if prereqs:
|
||||
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
|
||||
if steps:
|
||||
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
|
||||
if len(steps) > 6:
|
||||
parts[-1] += ' → ...'
|
||||
print('<br>'.join(parts))
|
||||
except: pass
|
||||
" "$QA_GUIDE" 2>/dev/null)
|
||||
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
|
||||
fi
|
||||
|
||||
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
|
||||
fi
|
||||
|
||||
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
|
||||
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
|
||||
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
|
||||
# Badge HTML with copy button (placeholder URL filled after deploy)
|
||||
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
|
||||
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
|
||||
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
|
||||
python3 -c "
|
||||
import sys, pathlib
|
||||
d = pathlib.Path(sys.argv[1])
|
||||
t = pathlib.Path(sys.argv[2]).read_text()
|
||||
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
|
||||
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
|
||||
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
|
||||
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
|
||||
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
|
||||
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
|
||||
sys.stdout.write(t)
|
||||
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
|
||||
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
|
||||
|
||||
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
|
||||
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
|
||||
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
|
||||
ERROREOF
|
||||
|
||||
# Copy research log to deploy dir if it exists
|
||||
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
|
||||
if [ -f "$rlog" ]; then
|
||||
cp "$rlog" "$DEPLOY_DIR/research-log.json"
|
||||
echo "Found research log: $rlog"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy generated test code to deploy dir
|
||||
for tfile in qa-artifacts/*/research/reproduce.spec.ts qa-artifacts/*/*/research/reproduce.spec.ts qa-artifacts/before/*/research/reproduce.spec.ts; do
|
||||
if [ -f "$tfile" ]; then
|
||||
cp "$tfile" "$DEPLOY_DIR/reproduce.spec.ts"
|
||||
echo "Found test code: $tfile"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Copy video script if available
|
||||
for vsfile in qa-artifacts/*/video-script.spec.ts qa-artifacts/*/*/video-script.spec.ts qa-artifacts/before/*/video-script.spec.ts; do
|
||||
if [ -f "$vsfile" ]; then
|
||||
cp "$vsfile" "$DEPLOY_DIR/video-script.spec.ts"
|
||||
echo "Found video script: $vsfile"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Generate badge SVGs into deploy dir
|
||||
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
|
||||
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
|
||||
|
||||
# Try research log first (ground truth from a11y assertions)
|
||||
RESEARCH_VERDICT=""
|
||||
REPRO_METHOD=""
|
||||
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
|
||||
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
|
||||
REPRO_METHOD=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('reproducedBy','none'))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
|
||||
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none} (by: ${REPRO_METHOD:-none})"
|
||||
if [ -n "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=1
|
||||
case "$RESEARCH_VERDICT" in
|
||||
REPRODUCED) REPRO_COUNT=1 ;;
|
||||
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
|
||||
INCONCLUSIVE) INCONC_COUNT=1 ;;
|
||||
esac
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check video review verdicts (always, not just as fallback)
|
||||
VIDEO_REPRODUCED=false
|
||||
if [ -d video-reviews ]; then
|
||||
for rpt in video-reviews/*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] || continue
|
||||
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
|
||||
|
||||
if [ -n "$VERDICT_JSON" ]; then
|
||||
echo "Video review verdict: $VERDICT_JSON ($(basename "$rpt"))"
|
||||
[ "$VERDICT_JSON" = "REPRODUCED" ] && VIDEO_REPRODUCED=true
|
||||
# Only count video as separate report if no research log
|
||||
if [ -z "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
|
||||
case "$VERDICT_JSON" in
|
||||
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
|
||||
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
|
||||
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
|
||||
esac
|
||||
fi
|
||||
elif [ -z "$RESEARCH_VERDICT" ]; then
|
||||
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
|
||||
# Fallback: grep Summary section (for older reports without ## Verdict)
|
||||
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
|
||||
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
|
||||
INCONC_COUNT=$((INCONC_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
|
||||
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
|
||||
REPRO_COUNT=$((REPRO_COUNT + 1))
|
||||
VIDEO_REPRODUCED=true
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Upgrade reproduction method to "both" when E2E and video agree
|
||||
if [ "$REPRO_METHOD" = "e2e_test" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
|
||||
REPRO_METHOD="both"
|
||||
echo "Upgraded reproducedBy to 'both' (E2E + video review agree)"
|
||||
elif [ -z "$RESEARCH_VERDICT" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
|
||||
REPRO_METHOD="video"
|
||||
fi
|
||||
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
|
||||
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
|
||||
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
|
||||
|
||||
# Warn on verdict mismatch between E2E and video review
|
||||
if [ -n "$RESEARCH_VERDICT" ]; then
|
||||
VIDEO_VERDICT=$(grep -oP '"verdict":\s*"[A-Z_]+' video-reviews/*-qa-video-report.md 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
|
||||
if [ -n "$VIDEO_VERDICT" ] && [ "$RESEARCH_VERDICT" != "$VIDEO_VERDICT" ]; then
|
||||
echo "⚠ Verdict mismatch: E2E=$RESEARCH_VERDICT vs Video=$VIDEO_VERDICT (E2E takes priority)"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Verdict: ${REPRO_COUNT}✓ ${NOT_REPRO_COUNT}✗ ${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
|
||||
|
||||
# Badge text:
|
||||
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
|
||||
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
|
||||
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
|
||||
if [ "$TOTAL_REPORTS" -le 1 ]; then
|
||||
# Single report — simple label
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
|
||||
elif [ "$FAIL_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
else
|
||||
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
|
||||
PARTS=""
|
||||
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}✓"
|
||||
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}✗"
|
||||
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}⚠"
|
||||
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
|
||||
# Color based on best outcome
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Badge label: #NUM QA0327 (with today's date)
|
||||
QA_DATE=$(date -u '+%m%d')
|
||||
BADGE_LABEL="QA${QA_DATE}"
|
||||
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
|
||||
|
||||
# For PRs, also extract fix quality from Overall Risk section
|
||||
FIX_RESULT="" FIX_COLOR="#4c1"
|
||||
if [ "$TARGET_TYPE" != "issue" ]; then
|
||||
# Try structured JSON risk first
|
||||
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
|
||||
if [ -n "$ALL_RISKS" ]; then
|
||||
# Use worst risk across all reports
|
||||
if echo "$ALL_RISKS" | grep -q 'high'; then
|
||||
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
|
||||
elif echo "$ALL_RISKS" | grep -q 'medium'; then
|
||||
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
|
||||
elif echo "$ALL_RISKS" | grep -q 'low'; then
|
||||
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
|
||||
fi
|
||||
else
|
||||
# Fallback: grep Overall Risk section
|
||||
RISK_TEXT=""
|
||||
if [ -d video-reviews ]; then
|
||||
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20 || true)
|
||||
fi
|
||||
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]' || true)
|
||||
if [ -n "$RISK_FIRST" ]; then
|
||||
case "$RISK_FIRST" in
|
||||
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
|
||||
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
|
||||
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
|
||||
esac
|
||||
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
|
||||
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# Always use vertical box badge
|
||||
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
|
||||
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
|
||||
"$FIX_RESULT" "$FIX_COLOR" "$REPRO_METHOD"
|
||||
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
|
||||
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Remove files exceeding Cloudflare Pages 25MB limit to prevent silent deploy failures
|
||||
MAX_SIZE=$((25 * 1024 * 1024))
|
||||
find "$DEPLOY_DIR" -type f -size +${MAX_SIZE}c | while read -r big_file; do
|
||||
SIZE_MB=$(( $(stat -c%s "$big_file") / 1024 / 1024 ))
|
||||
echo "Removing oversized file: $(basename "$big_file") (${SIZE_MB}MB > 25MB limit)"
|
||||
rm "$big_file"
|
||||
done
|
||||
|
||||
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
|
||||
|
||||
DEPLOY_OUTPUT=$(wrangler pages deploy "$DEPLOY_DIR" \
|
||||
--project-name="comfy-qa" \
|
||||
--branch="$BRANCH" 2>&1) || true
|
||||
echo "$DEPLOY_OUTPUT" | tail -5
|
||||
|
||||
URL=$(echo "$DEPLOY_OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1 || true)
|
||||
FALLBACK_URL="https://${BRANCH}.comfy-qa.pages.dev"
|
||||
|
||||
echo "url=${URL:-$FALLBACK_URL}" >> "$GITHUB_OUTPUT"
|
||||
echo "Deployed to: ${URL:-$FALLBACK_URL}"
|
||||
@@ -1,208 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
|
||||
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-generate-test.ts \
|
||||
* --qa-report <path> QA video review report (markdown)
|
||||
* --pr-diff <path> PR diff file
|
||||
* --output <path> Output .spec.ts file path
|
||||
* --model <name> Gemini model (default: gemini-3-flash-preview)
|
||||
*/
|
||||
import { readFile, writeFile } from 'node:fs/promises'
|
||||
import { basename, resolve } from 'node:path'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
interface CliOptions {
|
||||
qaReport: string
|
||||
prDiff: string
|
||||
output: string
|
||||
model: string
|
||||
}
|
||||
|
||||
const DEFAULTS: CliOptions = {
|
||||
qaReport: '',
|
||||
prDiff: '',
|
||||
output: '',
|
||||
model: 'gemini-3-flash-preview'
|
||||
}
|
||||
|
||||
// ── Fixture API reference for the prompt ────────────────────────────
|
||||
const FIXTURE_API = `
|
||||
## ComfyUI Playwright Test Fixture API
|
||||
|
||||
Import pattern:
|
||||
\`\`\`typescript
|
||||
import { expect } from '@playwright/test'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
\`\`\`
|
||||
|
||||
### Available helpers on \`comfyPage\`:
|
||||
- \`comfyPage.page\` — raw Playwright Page
|
||||
- \`comfyPage.menu.topbar\` — Topbar helper:
|
||||
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
|
||||
- \`.getActiveTabName(): Promise<string>\` — get active tab name
|
||||
- \`.saveWorkflow(name)\` — Save via File > Save dialog
|
||||
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
|
||||
- \`.exportWorkflow(name)\` — Export via File > Export dialog
|
||||
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
|
||||
- \`.getWorkflowTab(name)\` — get a tab locator by name
|
||||
- \`.closeWorkflowTab(name)\` — close a tab
|
||||
- \`.openTopbarMenu()\` — open the hamburger menu
|
||||
- \`.openSubmenu(label)\` — hover to open a submenu
|
||||
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
|
||||
- \`.open()\` / \`.close()\` — toggle sidebar
|
||||
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
|
||||
- \`.getPersistedItem(name)\` — get a workflow item locator
|
||||
- \`comfyPage.workflow\` — WorkflowHelper:
|
||||
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
|
||||
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
|
||||
- \`.deleteWorkflow(name)\` — delete a workflow
|
||||
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
|
||||
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
|
||||
- \`comfyPage.settings.setSetting(key, value)\` — change settings
|
||||
- \`comfyPage.keyboard\` — KeyboardHelper:
|
||||
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
|
||||
- \`comfyPage.nodeOps\` — NodeOperationsHelper
|
||||
- \`comfyPage.canvas\` — CanvasHelper
|
||||
- \`comfyPage.contextMenu\` — ContextMenu
|
||||
- \`comfyPage.toast\` — ToastHelper
|
||||
- \`comfyPage.confirmDialog\` — confirmation dialog
|
||||
- \`comfyPage.nextFrame()\` — wait for Vue re-render
|
||||
|
||||
### Test patterns:
|
||||
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
|
||||
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
|
||||
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
|
||||
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
|
||||
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
|
||||
`
|
||||
|
||||
// ── Prompt builder ──────────────────────────────────────────────────
|
||||
function buildPrompt(qaReport: string, prDiff: string): string {
|
||||
return `You are a Playwright test generator for the ComfyUI frontend.
|
||||
|
||||
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
|
||||
described in the QA report below. The test must:
|
||||
|
||||
1. Use the ComfyUI Playwright fixture API (documented below)
|
||||
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
|
||||
3. NOT test code implementation details
|
||||
4. Be concise — only test the behavior that the PR changed
|
||||
5. Follow existing test conventions (see API reference)
|
||||
|
||||
${FIXTURE_API}
|
||||
|
||||
## QA Video Review Report
|
||||
${qaReport}
|
||||
|
||||
## PR Diff (for context on what changed)
|
||||
${prDiff.slice(0, 8000)}
|
||||
|
||||
## Output Requirements
|
||||
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
|
||||
- Start with imports, end with closing brace
|
||||
- Use descriptive test names that explain the expected behavior
|
||||
- Add screenshot assertions where visual verification matters
|
||||
- Keep it focused: 2-5 test cases covering the core behavioral change
|
||||
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
|
||||
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
|
||||
`
|
||||
}
|
||||
|
||||
// ── Gemini call ─────────────────────────────────────────────────────
|
||||
async function generateTest(
|
||||
qaReport: string,
|
||||
prDiff: string,
|
||||
model: string
|
||||
): Promise<string> {
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
|
||||
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const genModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildPrompt(qaReport, prDiff)
|
||||
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
|
||||
|
||||
const result = await genModel.generateContent({
|
||||
contents: [{ role: 'user', parts: [{ text: prompt }] }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192
|
||||
}
|
||||
})
|
||||
|
||||
const text = result.response.text()
|
||||
|
||||
// Strip markdown fences if model wraps output
|
||||
return text
|
||||
.replace(/^```(?:typescript|ts)?\n?/, '')
|
||||
.replace(/\n?```$/, '')
|
||||
.trim()
|
||||
}
|
||||
|
||||
// ── CLI ─────────────────────────────────────────────────────────────
|
||||
function parseArgs(): CliOptions {
|
||||
const args = process.argv.slice(2)
|
||||
const opts = { ...DEFAULTS }
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--qa-report':
|
||||
opts.qaReport = args[++i]
|
||||
break
|
||||
case '--pr-diff':
|
||||
opts.prDiff = args[++i]
|
||||
break
|
||||
case '--output':
|
||||
opts.output = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--help':
|
||||
console.warn(`Usage:
|
||||
pnpm exec tsx scripts/qa-generate-test.ts [options]
|
||||
|
||||
Options:
|
||||
--qa-report <path> QA video review report (markdown) [required]
|
||||
--pr-diff <path> PR diff file [required]
|
||||
--output <path> Output .spec.ts path [required]
|
||||
--model <name> Gemini model (default: gemini-3-flash-preview)`)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.qaReport || !opts.prDiff || !opts.output) {
|
||||
console.error('Missing required args. Run with --help for usage.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
|
||||
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
|
||||
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
|
||||
|
||||
console.warn(
|
||||
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
|
||||
)
|
||||
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
|
||||
|
||||
const testCode = await generateTest(qaReport, prDiff, opts.model)
|
||||
|
||||
const outputPath = resolve(opts.output)
|
||||
await writeFile(outputPath, testCode + '\n')
|
||||
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err)
|
||||
process.exit(1)
|
||||
})
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,164 +0,0 @@
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
|
||||
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<style>
|
||||
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
|
||||
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
|
||||
*{margin:0;padding:0;box-sizing:border-box}
|
||||
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
|
||||
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
|
||||
.container{max-width:1200px;margin:0 auto}
|
||||
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
|
||||
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
|
||||
.header-icon svg{color:var(--primary)}
|
||||
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
|
||||
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
|
||||
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
|
||||
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
|
||||
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
|
||||
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
|
||||
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
|
||||
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
|
||||
.icon{font-size:1.125rem}
|
||||
.links{display:flex;gap:.5rem}
|
||||
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
|
||||
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
|
||||
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
|
||||
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
|
||||
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
|
||||
.comp-panel{border-right:1px solid var(--border-faint)}
|
||||
.comp-panel:last-child{border-right:none}
|
||||
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
|
||||
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
|
||||
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
|
||||
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
|
||||
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
|
||||
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
|
||||
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
|
||||
.report summary:hover{color:var(--fg)}
|
||||
.report summary svg{flex-shrink:0;opacity:.5}
|
||||
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
|
||||
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
|
||||
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
|
||||
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
|
||||
.report-body p{margin:.4rem 0}
|
||||
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
|
||||
.report-body li{margin:.25rem 0}
|
||||
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
|
||||
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
|
||||
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
|
||||
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
|
||||
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
|
||||
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
|
||||
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
|
||||
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
|
||||
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
|
||||
.report-body strong{color:var(--fg)}
|
||||
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
|
||||
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
|
||||
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
|
||||
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
|
||||
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
|
||||
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
|
||||
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
|
||||
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
|
||||
.badge-img{height:20px;display:block}
|
||||
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
|
||||
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
|
||||
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
|
||||
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
|
||||
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
|
||||
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
|
||||
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
|
||||
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
|
||||
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
|
||||
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
|
||||
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
|
||||
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
|
||||
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
|
||||
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
|
||||
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
|
||||
.purpose strong{color:var(--fg);font-weight:600}
|
||||
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
|
||||
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
|
||||
</style></head><body><div class=container>
|
||||
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend · Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
|
||||
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
|
||||
<div id=research-section style="margin-top:2rem"></div>
|
||||
</div><script>
|
||||
// Load research-log.json and reproduce.spec.ts if available
|
||||
(async()=>{
|
||||
const sec=document.getElementById('research-section');
|
||||
try{
|
||||
const [logRes,testRes,vsRes]=await Promise.allSettled([fetch('research-log.json'),fetch('reproduce.spec.ts'),fetch('video-script.spec.ts')]);
|
||||
let html='';
|
||||
if(logRes.status==='fulfilled'&&logRes.value.ok){
|
||||
const log=await logRes.value.json();
|
||||
// Show verdict banner for non-reproduced results
|
||||
if(log.verdict&&log.verdict!=='REPRODUCED'){
|
||||
const colors={NOT_REPRODUCIBLE:{bg:'oklch(25% 0.08 25)',border:'oklch(40% 0.15 25)',icon:'✗'},INCONCLUSIVE:{bg:'oklch(25% 0.06 80)',border:'oklch(40% 0.12 80)',icon:'⚠'}};
|
||||
const c=colors[log.verdict]||colors.INCONCLUSIVE;
|
||||
html+=`<div style="margin-bottom:1.5rem;padding:1.25rem;background:${c.bg};border:1px solid ${c.border};border-radius:var(--r-lg)"><div style="font-size:1.25rem;font-weight:700;margin-bottom:.5rem">${c.icon} ${log.verdict.replace(/_/g,' ')}</div><div style="font-size:.9rem;line-height:1.6;opacity:.9">${(log.summary||'No details available.').replace(/</g,'<')}</div>${log.evidence?`<div style="margin-top:.75rem;padding:.75rem;background:oklch(0% 0 0/.2);border-radius:var(--r);font-family:var(--font-mono);font-size:.8rem;white-space:pre-wrap;max-height:200px;overflow:auto">${log.evidence.replace(/</g,'<')}</div>`:''}</div>`;
|
||||
}
|
||||
html+=`<details style="margin-bottom:1.5rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Research Log — ${log.verdict||'?'} (${(log.log||[]).length||'?'} tool calls, ${((log.elapsedMs||0)/1000).toFixed(1)}s)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${JSON.stringify(log,null,2)}</pre></div></details>`;
|
||||
}
|
||||
if(testRes.status==='fulfilled'&&testRes.value.ok){
|
||||
const code=await testRes.value.text();
|
||||
html+=`<details><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">E2E Test Code (reproduce.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${code.replace(/</g,'<').replace(/>/g,'>')}</pre></div></details>`;
|
||||
}
|
||||
if(vsRes.status==='fulfilled'&&vsRes.value.ok){
|
||||
const vsCode=await vsRes.value.text();
|
||||
html+=`<details style="margin-top:1rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Video Script (video-script.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${vsCode.replace(/</g,'<').replace(/>/g,'>')}</pre></div></details>`;
|
||||
}
|
||||
if(html)sec.innerHTML=html;
|
||||
}catch(e){console.warn('research load failed',e)}
|
||||
})();
|
||||
</script><script>
|
||||
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
|
||||
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
|
||||
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
|
||||
document.querySelectorAll('.video-wrap video').forEach(v=>{
|
||||
v.playbackRate=1;
|
||||
const c=document.createElement('div');c.className='vctrl';
|
||||
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
|
||||
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
|
||||
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
|
||||
btn('\u23EE',()=>{v.currentTime=0});
|
||||
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
|
||||
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
|
||||
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
|
||||
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
|
||||
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
|
||||
sep();
|
||||
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===1)b.classList.add('active');return b});
|
||||
sep();c.appendChild(time);
|
||||
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
|
||||
// Custom seekbar — works even without server range request support
|
||||
const seekWrap=document.createElement('div');seekWrap.className='vseek';
|
||||
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
|
||||
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
|
||||
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
|
||||
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
|
||||
let seeking=false;
|
||||
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
|
||||
seekBar.onchange=()=>{seeking=false};
|
||||
v.closest('.video-wrap').after(seekWrap);
|
||||
seekWrap.after(c);
|
||||
v.ontimeupdate=()=>{
|
||||
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
|
||||
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
|
||||
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
|
||||
};
|
||||
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
|
||||
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
|
||||
v.parentElement.addEventListener('keydown',e=>{
|
||||
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
|
||||
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
|
||||
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
|
||||
});
|
||||
v.parentElement.setAttribute('tabindex','0');
|
||||
});
|
||||
</script></body></html>
|
||||
@@ -1,253 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA Reproduce Phase — Deterministic replay of research plan with narration
|
||||
*
|
||||
* Takes a reproduction plan from the research phase and replays it:
|
||||
* 1. Execute each action deterministically (no AI decisions)
|
||||
* 2. Capture a11y snapshot before/after each action
|
||||
* 3. Gemini describes what visually changed (narration for humans)
|
||||
* 4. Output: narration-log.json with full evidence chain
|
||||
*/
|
||||
|
||||
import type { Page } from '@playwright/test'
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
import { mkdirSync, writeFileSync } from 'fs'
|
||||
|
||||
import type { ActionResult } from './qa-record.js'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface ReproductionStep {
|
||||
action: Record<string, unknown> & { action: string }
|
||||
expectedAssertion: string
|
||||
}
|
||||
|
||||
interface NarrationEntry {
|
||||
step: number
|
||||
action: string
|
||||
params: Record<string, unknown>
|
||||
result: ActionResult
|
||||
a11yBefore: unknown
|
||||
a11yAfter: unknown
|
||||
assertionExpected: string
|
||||
assertionPassed: boolean
|
||||
assertionActual: string
|
||||
geminiNarration: string
|
||||
timestampMs: number
|
||||
}
|
||||
|
||||
export interface NarrationLog {
|
||||
entries: NarrationEntry[]
|
||||
allAssertionsPassed: boolean
|
||||
}
|
||||
|
||||
interface ReproduceOptions {
|
||||
page: Page
|
||||
plan: ReproductionStep[]
|
||||
geminiApiKey: string
|
||||
outputDir: string
|
||||
}
|
||||
|
||||
// ── A11y helpers ──
|
||||
|
||||
interface A11yNode {
|
||||
role: string
|
||||
name: string
|
||||
value?: string
|
||||
checked?: boolean
|
||||
disabled?: boolean
|
||||
expanded?: boolean
|
||||
children?: A11yNode[]
|
||||
}
|
||||
|
||||
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
|
||||
if (!node) return null
|
||||
const sel = selector.toLowerCase()
|
||||
if (
|
||||
node.name?.toLowerCase().includes(sel) ||
|
||||
node.role?.toLowerCase().includes(sel)
|
||||
) {
|
||||
return node
|
||||
}
|
||||
if (node.children) {
|
||||
for (const child of node.children) {
|
||||
const found = searchA11y(child, selector)
|
||||
if (found) return found
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function summarizeA11y(node: A11yNode | null): string {
|
||||
if (!node) return 'null'
|
||||
const parts = [`role=${node.role}`, `name="${node.name}"`]
|
||||
if (node.value !== undefined) parts.push(`value="${node.value}"`)
|
||||
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
|
||||
if (node.disabled) parts.push('disabled')
|
||||
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
|
||||
return `{${parts.join(', ')}}`
|
||||
}
|
||||
|
||||
// ── Subtitle overlay ──
|
||||
|
||||
async function showSubtitle(page: Page, text: string, step: number) {
|
||||
const encoded = encodeURIComponent(
|
||||
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
|
||||
)
|
||||
await page.addScriptTag({
|
||||
content: `(function(){
|
||||
var id='qa-subtitle';
|
||||
var el=document.getElementById(id);
|
||||
if(!el){
|
||||
el=document.createElement('div');
|
||||
el.id=id;
|
||||
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
|
||||
document.body.appendChild(el);
|
||||
}
|
||||
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
|
||||
})()`
|
||||
})
|
||||
}
|
||||
|
||||
// ── Gemini visual narration ──
|
||||
|
||||
async function geminiDescribe(
|
||||
page: Page,
|
||||
geminiApiKey: string,
|
||||
focus: string
|
||||
): Promise<string> {
|
||||
try {
|
||||
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
|
||||
const genAI = new GoogleGenerativeAI(geminiApiKey)
|
||||
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
|
||||
|
||||
const result = await model.generateContent([
|
||||
{
|
||||
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
|
||||
},
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: 'image/jpeg',
|
||||
data: screenshot.toString('base64')
|
||||
}
|
||||
}
|
||||
])
|
||||
return result.response.text().trim()
|
||||
} catch (e) {
|
||||
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
|
||||
}
|
||||
}
|
||||
|
||||
// ── Main reproduce function ──
|
||||
|
||||
export async function runReproducePhase(
|
||||
opts: ReproduceOptions
|
||||
): Promise<NarrationLog> {
|
||||
const { page, plan, geminiApiKey, outputDir } = opts
|
||||
const { executeAction } = await import('./qa-record.js')
|
||||
|
||||
const narrationDir = `${outputDir}/narration`
|
||||
mkdirSync(narrationDir, { recursive: true })
|
||||
|
||||
const entries: NarrationEntry[] = []
|
||||
const startMs = Date.now()
|
||||
|
||||
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
|
||||
|
||||
for (let i = 0; i < plan.length; i++) {
|
||||
const step = plan[i]
|
||||
const actionObj = step.action
|
||||
const elapsed = Date.now() - startMs
|
||||
|
||||
// Show subtitle
|
||||
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
|
||||
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
|
||||
|
||||
// Capture a11y BEFORE
|
||||
const a11yBefore = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 3000 })
|
||||
.catch(() => null)
|
||||
|
||||
// Execute action
|
||||
const result = await executeAction(
|
||||
page,
|
||||
actionObj as Parameters<typeof executeAction>[1],
|
||||
outputDir
|
||||
)
|
||||
await new Promise((r) => setTimeout(r, 500))
|
||||
|
||||
// Capture a11y AFTER
|
||||
const a11yAfter = await page
|
||||
.locator('body')
|
||||
.ariaSnapshot({ timeout: 3000 })
|
||||
.catch(() => null)
|
||||
|
||||
// Check assertion
|
||||
let assertionPassed = false
|
||||
let assertionActual = ''
|
||||
if (step.expectedAssertion) {
|
||||
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
|
||||
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
|
||||
const selectorName = parts[0]
|
||||
const expectedState = parts.slice(1).join(':').trim()
|
||||
|
||||
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
|
||||
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
|
||||
|
||||
if (expectedState === 'visible' || expectedState === 'exists') {
|
||||
assertionPassed = found !== null
|
||||
} else if (expectedState === 'hidden' || expectedState === 'gone') {
|
||||
assertionPassed = found === null
|
||||
} else {
|
||||
// Generic: check if the actual state contains the expected text
|
||||
assertionPassed = assertionActual
|
||||
.toLowerCase()
|
||||
.includes(expectedState.toLowerCase())
|
||||
}
|
||||
|
||||
console.warn(
|
||||
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
|
||||
)
|
||||
}
|
||||
|
||||
// Gemini narration (visual description for humans)
|
||||
const geminiNarration = await geminiDescribe(
|
||||
page,
|
||||
geminiApiKey,
|
||||
`What changed after ${actionObj.action}?`
|
||||
)
|
||||
|
||||
entries.push({
|
||||
step: i + 1,
|
||||
action: actionObj.action,
|
||||
params: actionObj,
|
||||
result,
|
||||
a11yBefore,
|
||||
a11yAfter,
|
||||
assertionExpected: step.expectedAssertion,
|
||||
assertionPassed,
|
||||
assertionActual,
|
||||
geminiNarration,
|
||||
timestampMs: elapsed
|
||||
})
|
||||
}
|
||||
|
||||
// Final screenshot
|
||||
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
|
||||
|
||||
const log: NarrationLog = {
|
||||
entries,
|
||||
allAssertionsPassed: entries.every((e) => e.assertionPassed)
|
||||
}
|
||||
|
||||
writeFileSync(
|
||||
`${narrationDir}/narration-log.json`,
|
||||
JSON.stringify(log, null, 2)
|
||||
)
|
||||
console.warn(
|
||||
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
|
||||
)
|
||||
|
||||
return log
|
||||
}
|
||||
@@ -1,150 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import {
|
||||
extractPlatformFromArtifactDirName,
|
||||
pickLatestVideosByPlatform,
|
||||
selectVideoCandidateByFile
|
||||
} from './qa-video-review'
|
||||
|
||||
describe('extractPlatformFromArtifactDirName', () => {
|
||||
it('extracts and normalizes known qa artifact directory names', () => {
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
|
||||
).toBe('windows')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
|
||||
).toBe('macos')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
|
||||
).toBe('linux')
|
||||
})
|
||||
|
||||
it('falls back to slugifying unknown directory names', () => {
|
||||
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
|
||||
'custom-platform-run'
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('pickLatestVideosByPlatform', () => {
|
||||
it('keeps only the latest candidate per platform', () => {
|
||||
const selected = pickLatestVideosByPlatform([
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-old.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
}
|
||||
])
|
||||
|
||||
expect(selected).toEqual([
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('selectVideoCandidateByFile', () => {
|
||||
it('selects a single candidate by artifacts-relative path', () => {
|
||||
const selected = selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-Linux-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
|
||||
expect(selected).toEqual({
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
})
|
||||
})
|
||||
|
||||
it('throws when basename matches multiple videos', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('matched 2 videos')
|
||||
})
|
||||
|
||||
it('throws when there is no matching video', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-macOS-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('No video matched')
|
||||
})
|
||||
|
||||
it('throws when video file is missing', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: ' '
|
||||
}
|
||||
)
|
||||
).toThrow('--video-file is required')
|
||||
})
|
||||
})
|
||||
@@ -1,771 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
|
||||
import { basename, dirname, extname, relative, resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
import { globSync } from 'glob'
|
||||
|
||||
interface CliOptions {
|
||||
artifactsDir: string
|
||||
videoFile: string
|
||||
beforeVideo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
requestTimeoutMs: number
|
||||
dryRun: boolean
|
||||
prContext: string
|
||||
targetUrl: string
|
||||
passLabel: string
|
||||
}
|
||||
|
||||
interface VideoCandidate {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
mtimeMs: number
|
||||
}
|
||||
|
||||
const DEFAULT_OPTIONS: CliOptions = {
|
||||
artifactsDir: './tmp/qa-artifacts',
|
||||
videoFile: '',
|
||||
beforeVideo: '',
|
||||
outputDir: './tmp',
|
||||
model: 'gemini-3-flash-preview',
|
||||
requestTimeoutMs: 300_000,
|
||||
dryRun: false,
|
||||
prContext: '',
|
||||
targetUrl: '',
|
||||
passLabel: ''
|
||||
}
|
||||
|
||||
const USAGE = `Usage:
|
||||
pnpm exec tsx scripts/qa-video-review.ts [options]
|
||||
|
||||
Options:
|
||||
--artifacts-dir <path> Artifacts root directory
|
||||
(default: ./tmp/qa-artifacts)
|
||||
--video-file <name-or-path> Video file to analyze (required)
|
||||
(supports basename or relative/absolute path)
|
||||
--before-video <path> Before video (main branch) for comparison
|
||||
When provided, sends both videos to Gemini
|
||||
for comparative before/after analysis
|
||||
--output-dir <path> Output directory for markdown reports
|
||||
(default: ./tmp)
|
||||
--model <name> Gemini model
|
||||
(default: gemini-3-flash-preview)
|
||||
--request-timeout-ms <n> Request timeout in milliseconds
|
||||
(default: 300000)
|
||||
--pr-context <file> File with PR context (title, body, diff)
|
||||
for PR-aware review
|
||||
--target-url <url> Issue or PR URL to include in the report
|
||||
--pass-label <label> Label for multi-pass reports (e.g. pass1)
|
||||
Output becomes {platform}-{label}-qa-video-report.md
|
||||
--dry-run Discover videos and output targets only
|
||||
--help Show this help text
|
||||
|
||||
Environment:
|
||||
GEMINI_API_KEY Required unless --dry-run
|
||||
`
|
||||
|
||||
function parsePositiveInteger(rawValue: string, flagName: string): number {
|
||||
const parsedValue = Number.parseInt(rawValue, 10)
|
||||
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
|
||||
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
|
||||
}
|
||||
return parsedValue
|
||||
}
|
||||
|
||||
function parseCliOptions(args: string[]): CliOptions {
|
||||
const options: CliOptions = { ...DEFAULT_OPTIONS }
|
||||
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const argument = args[index]
|
||||
const nextValue = args[index + 1]
|
||||
const requireValue = (flagName: string): string => {
|
||||
if (!nextValue || nextValue.startsWith('--')) {
|
||||
throw new Error(`Missing value for ${flagName}`)
|
||||
}
|
||||
index += 1
|
||||
return nextValue
|
||||
}
|
||||
|
||||
if (argument === '--help') {
|
||||
process.stdout.write(USAGE)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
if (argument === '--artifacts-dir') {
|
||||
options.artifactsDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--video-file') {
|
||||
options.videoFile = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--output-dir') {
|
||||
options.outputDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--model') {
|
||||
options.model = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--request-timeout-ms') {
|
||||
options.requestTimeoutMs = parsePositiveInteger(
|
||||
requireValue(argument),
|
||||
argument
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--before-video') {
|
||||
options.beforeVideo = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pr-context') {
|
||||
options.prContext = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--target-url') {
|
||||
options.targetUrl = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pass-label') {
|
||||
options.passLabel = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--dry-run') {
|
||||
options.dryRun = true
|
||||
continue
|
||||
}
|
||||
|
||||
throw new Error(`Unknown argument: ${argument}`)
|
||||
}
|
||||
|
||||
return options
|
||||
}
|
||||
|
||||
function normalizePlatformName(value: string): string {
|
||||
const slug = value
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
|
||||
return slug.length > 0 ? slug : 'unknown-platform'
|
||||
}
|
||||
|
||||
export function extractPlatformFromArtifactDirName(dirName: string): string {
|
||||
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
|
||||
return normalizePlatformName(matchedValue ?? dirName)
|
||||
}
|
||||
|
||||
function extractPlatformFromVideoPath(videoPath: string): string {
|
||||
const artifactDirName = basename(dirname(videoPath))
|
||||
return extractPlatformFromArtifactDirName(artifactDirName)
|
||||
}
|
||||
|
||||
export function pickLatestVideosByPlatform(
|
||||
candidates: VideoCandidate[]
|
||||
): VideoCandidate[] {
|
||||
const latestByPlatform = new Map<string, VideoCandidate>()
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const current = latestByPlatform.get(candidate.platformName)
|
||||
if (!current || candidate.mtimeMs > current.mtimeMs) {
|
||||
latestByPlatform.set(candidate.platformName, candidate)
|
||||
}
|
||||
}
|
||||
|
||||
return [...latestByPlatform.values()].sort((a, b) =>
|
||||
a.platformName.localeCompare(b.platformName)
|
||||
)
|
||||
}
|
||||
|
||||
function toProjectRelativePath(targetPath: string): string {
|
||||
const relativePath = relative(process.cwd(), targetPath)
|
||||
if (relativePath.startsWith('.')) {
|
||||
return relativePath
|
||||
}
|
||||
return `./${relativePath}`
|
||||
}
|
||||
|
||||
function errorToString(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
|
||||
function normalizePathForMatch(value: string): string {
|
||||
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
|
||||
}
|
||||
|
||||
export function selectVideoCandidateByFile(
|
||||
candidates: VideoCandidate[],
|
||||
options: { artifactsDir: string; videoFile: string }
|
||||
): VideoCandidate {
|
||||
const requestedValue = options.videoFile.trim()
|
||||
if (requestedValue.length === 0) {
|
||||
throw new Error('--video-file is required')
|
||||
}
|
||||
|
||||
const artifactsRoot = resolve(options.artifactsDir)
|
||||
const requestedAbsolutePath = resolve(requestedValue)
|
||||
const requestedPathKey = normalizePathForMatch(requestedValue)
|
||||
|
||||
const matches = candidates.filter((candidate) => {
|
||||
const candidateAbsolutePath = resolve(candidate.videoPath)
|
||||
if (candidateAbsolutePath === requestedAbsolutePath) {
|
||||
return true
|
||||
}
|
||||
|
||||
const candidateBaseName = basename(candidate.videoPath)
|
||||
if (candidateBaseName === requestedValue) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToCwd = normalizePathForMatch(
|
||||
relative(process.cwd(), candidateAbsolutePath)
|
||||
)
|
||||
if (relativeToCwd === requestedPathKey) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToArtifacts = normalizePathForMatch(
|
||||
relative(artifactsRoot, candidateAbsolutePath)
|
||||
)
|
||||
return relativeToArtifacts === requestedPathKey
|
||||
})
|
||||
|
||||
if (matches.length === 1) {
|
||||
return matches[0]
|
||||
}
|
||||
|
||||
if (matches.length === 0) {
|
||||
const availableVideos = candidates.map((candidate) =>
|
||||
toProjectRelativePath(candidate.videoPath)
|
||||
)
|
||||
throw new Error(
|
||||
[
|
||||
`No video matched --video-file "${options.videoFile}".`,
|
||||
'Available videos:',
|
||||
...availableVideos.map((videoPath) => `- ${videoPath}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
[
|
||||
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
|
||||
'Please pass a more specific path.',
|
||||
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
async function collectVideoCandidates(
|
||||
artifactsDir: string
|
||||
): Promise<VideoCandidate[]> {
|
||||
const absoluteArtifactsDir = resolve(artifactsDir)
|
||||
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
|
||||
cwd: absoluteArtifactsDir,
|
||||
absolute: true,
|
||||
nodir: true
|
||||
}).sort()
|
||||
|
||||
const candidates = await Promise.all(
|
||||
videoPaths.map(async (videoPath) => {
|
||||
const videoStat = await stat(videoPath)
|
||||
return {
|
||||
platformName: extractPlatformFromVideoPath(videoPath),
|
||||
videoPath,
|
||||
mtimeMs: videoStat.mtimeMs
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
return candidates
|
||||
}
|
||||
|
||||
function getMimeType(filePath: string): string {
|
||||
const ext = extname(filePath).toLowerCase()
|
||||
const mimeMap: Record<string, string> = {
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime',
|
||||
'.avi': 'video/x-msvideo',
|
||||
'.mkv': 'video/x-matroska',
|
||||
'.m4v': 'video/mp4'
|
||||
}
|
||||
return mimeMap[ext] || 'video/mp4'
|
||||
}
|
||||
|
||||
function buildReviewPrompt(options: {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
prContext: string
|
||||
isComparative: boolean
|
||||
}): string {
|
||||
const { platformName, videoPath, prContext, isComparative } = options
|
||||
|
||||
if (isComparative) {
|
||||
return buildComparativePrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
return buildSingleVideoPrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
function buildComparativePrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
|
||||
'',
|
||||
'You are given TWO videos:',
|
||||
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
|
||||
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
|
||||
'',
|
||||
'Both videos show the same test steps executed on different code versions.',
|
||||
''
|
||||
]
|
||||
|
||||
if (prContext) {
|
||||
lines.push('## PR Context', prContext, '')
|
||||
}
|
||||
|
||||
lines.push(
|
||||
'## Your Task',
|
||||
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'',
|
||||
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
|
||||
' Describe what you observe — this establishes the baseline.',
|
||||
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
|
||||
'3. **Comparison**: What specifically changed between before and after?',
|
||||
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
|
||||
' that were NOT present in the BEFORE video?',
|
||||
'',
|
||||
'Note: Brief black frames during page transitions are NORMAL.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible differences. Avoid speculation.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
|
||||
'',
|
||||
'## Behavior Changes',
|
||||
'Summarize ALL behavioral differences as a markdown TABLE:',
|
||||
'| Behavior | Before (main) | After (PR) | Verdict |',
|
||||
'',
|
||||
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
|
||||
'- **Before (main)**: how it works/looks in the BEFORE video',
|
||||
'- **After (PR)**: how it works/looks in the AFTER video',
|
||||
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
|
||||
'',
|
||||
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
|
||||
'that were tested, so reviewers can confirm nothing was missed.',
|
||||
'',
|
||||
'## Timeline Comparison',
|
||||
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
|
||||
'| Time | Type | Severity | Before (main) | After (PR) |',
|
||||
'',
|
||||
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
|
||||
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
|
||||
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
|
||||
'- **Before (main)**: what is observed in the BEFORE video at that time',
|
||||
'- **After (PR)**: what is observed in the AFTER video at that time',
|
||||
'',
|
||||
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
|
||||
'omit that row. Focus on meaningful differences, not narrating every frame.',
|
||||
'',
|
||||
'## Confirmed Issues',
|
||||
'For each issue, use this exact format:',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'',
|
||||
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
|
||||
'',
|
||||
'**Evidence:** [What you observed at the given timestamp in which video]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk',
|
||||
'(Assess whether the PR achieves its goal based on the before/after comparison)',
|
||||
'',
|
||||
'## Verdict',
|
||||
'End your report with this EXACT JSON block (no markdown fence):',
|
||||
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
|
||||
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
|
||||
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
|
||||
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
function buildSingleVideoPrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer reviewing a UI test session recording.',
|
||||
'',
|
||||
'## ANTI-HALLUCINATION RULES (READ FIRST)',
|
||||
'- Describe ONLY what you can directly observe in the video frames',
|
||||
'- NEVER infer or assume what "must have happened" between frames',
|
||||
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
|
||||
'- Your job is to be a CAMERA — report facts, not interpretations',
|
||||
''
|
||||
]
|
||||
|
||||
const isIssueContext =
|
||||
prContext &&
|
||||
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
|
||||
prContext
|
||||
)
|
||||
|
||||
if (prContext) {
|
||||
lines.push(
|
||||
'## Phase 1: Blind Observation (describe what you SEE and HEAR)',
|
||||
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
|
||||
'- What elements does the user click/hover/type?',
|
||||
'- What dialogs/menus open and close?',
|
||||
'- What keyboard indicators appear? (look for subtitle overlays)',
|
||||
'- What is the BEFORE state and AFTER state of each action?',
|
||||
'- **Audio**: Does the video have a TTS narration audio track? If yes, transcribe what the voice says. This narration describes the bug being reproduced.',
|
||||
'',
|
||||
'## Phase 2: Compare against expected behavior',
|
||||
'Now compare your observations against the context below.',
|
||||
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
|
||||
''
|
||||
)
|
||||
|
||||
if (isIssueContext) {
|
||||
lines.push(
|
||||
'## Issue Context',
|
||||
prContext,
|
||||
'',
|
||||
'## Comparison Questions',
|
||||
'1. Did the video perform the reproduction steps described in the issue?',
|
||||
'2. Did your Phase 1 observations show the reported bug behavior?',
|
||||
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
|
||||
''
|
||||
)
|
||||
} else {
|
||||
lines.push(
|
||||
'## PR Context',
|
||||
prContext,
|
||||
'',
|
||||
'## Comparison Questions',
|
||||
'1. Did the video test the specific behavior the PR changes?',
|
||||
'2. Did your Phase 1 observations show the expected before/after difference?',
|
||||
'3. If the test was incomplete or inconclusive, say so honestly.',
|
||||
''
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
lines.push(
|
||||
`Review this QA session video for platform "${platformName}".`,
|
||||
`Source video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'The video shows the full test session — analyze it chronologically.',
|
||||
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
|
||||
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible problems and avoid speculation.',
|
||||
'If confidence is low, mark it explicitly.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
isIssueContext
|
||||
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
|
||||
: prContext
|
||||
? '(Explain what the PR intended and whether the video confirms it works)'
|
||||
: '',
|
||||
'## Confirmed Issues',
|
||||
'For each confirmed issue, use this exact format (one block per issue):',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`HIGH` `01:03` `Confidence: High`',
|
||||
'',
|
||||
'[Description of the issue — what went wrong and what was expected]',
|
||||
'',
|
||||
'**Evidence:** [What you observed in the video at the given timestamp]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'Do NOT use a table for issues — use the block format above.',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk',
|
||||
'',
|
||||
'## Narration',
|
||||
'If the video contains a TTS audio narration track, transcribe it here.',
|
||||
'If there is no audio or the video is silent, write "No narration detected."',
|
||||
'',
|
||||
'## Verdict',
|
||||
'End your report with this EXACT JSON block (no markdown fence):',
|
||||
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low", "narrationDetected": true | false}',
|
||||
'- REPRODUCED: the bug/behavior is clearly visible in the video',
|
||||
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
|
||||
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient',
|
||||
'- narrationDetected: true if you heard TTS voice narration in the video, false if silent'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
|
||||
|
||||
async function readVideoFile(videoPath: string): Promise<Buffer> {
|
||||
const fileStat = await stat(videoPath)
|
||||
if (fileStat.size > MAX_VIDEO_BYTES) {
|
||||
throw new Error(
|
||||
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
|
||||
)
|
||||
}
|
||||
return readFile(videoPath)
|
||||
}
|
||||
|
||||
async function requestGeminiReview(options: {
|
||||
apiKey: string
|
||||
model: string
|
||||
platformName: string
|
||||
videoPath: string
|
||||
beforeVideoPath: string
|
||||
timeoutMs: number
|
||||
prContext: string
|
||||
}): Promise<string> {
|
||||
const genAI = new GoogleGenerativeAI(options.apiKey)
|
||||
const model = genAI.getGenerativeModel({ model: options.model })
|
||||
|
||||
const isComparative = options.beforeVideoPath.length > 0
|
||||
const prompt = buildReviewPrompt({
|
||||
platformName: options.platformName,
|
||||
videoPath: options.videoPath,
|
||||
prContext: options.prContext,
|
||||
isComparative
|
||||
})
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
if (isComparative) {
|
||||
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
|
||||
parts.push(
|
||||
{ text: 'Video 1 — BEFORE (main branch):' },
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.beforeVideoPath),
|
||||
data: beforeBuffer.toString('base64')
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
const afterBuffer = await readVideoFile(options.videoPath)
|
||||
if (isComparative) {
|
||||
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
|
||||
}
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.videoPath),
|
||||
data: afterBuffer.toString('base64')
|
||||
}
|
||||
})
|
||||
|
||||
const result = await model.generateContent(parts, {
|
||||
timeout: options.timeoutMs
|
||||
})
|
||||
const response = result.response
|
||||
const text = response.text()
|
||||
|
||||
if (!text || text.trim().length === 0) {
|
||||
throw new Error('Gemini API returned no output text')
|
||||
}
|
||||
|
||||
return text.trim()
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
function buildReportMarkdown(input: {
|
||||
platformName: string
|
||||
model: string
|
||||
videoPath: string
|
||||
videoSizeBytes: number
|
||||
beforeVideoPath?: string
|
||||
beforeVideoSizeBytes?: number
|
||||
reviewText: string
|
||||
targetUrl?: string
|
||||
}): string {
|
||||
const headerLines = [
|
||||
`# ${input.platformName} QA Video Report`,
|
||||
'',
|
||||
`- Generated at: ${new Date().toISOString()}`,
|
||||
`- Model: \`${input.model}\``
|
||||
]
|
||||
|
||||
if (input.targetUrl) {
|
||||
headerLines.push(`- Target: ${input.targetUrl}`)
|
||||
}
|
||||
|
||||
if (input.beforeVideoPath) {
|
||||
headerLines.push(
|
||||
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
|
||||
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
|
||||
'- Mode: **Comparative (before/after)**'
|
||||
)
|
||||
} else {
|
||||
headerLines.push(
|
||||
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
|
||||
`- Video size: ${formatBytes(input.videoSizeBytes)}`
|
||||
)
|
||||
}
|
||||
|
||||
headerLines.push('', '## AI Review', '')
|
||||
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
|
||||
}
|
||||
|
||||
async function reviewVideo(
|
||||
video: VideoCandidate,
|
||||
options: CliOptions,
|
||||
apiKey: string
|
||||
): Promise<void> {
|
||||
let prContext = ''
|
||||
if (options.prContext) {
|
||||
try {
|
||||
prContext = await readFile(options.prContext, 'utf-8')
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
|
||||
)
|
||||
} catch {
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const beforeVideoPath = options.beforeVideo
|
||||
? resolve(options.beforeVideo)
|
||||
: ''
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
|
||||
)
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
|
||||
)
|
||||
|
||||
const reviewText = await requestGeminiReview({
|
||||
apiKey,
|
||||
model: options.model,
|
||||
platformName: video.platformName,
|
||||
videoPath: video.videoPath,
|
||||
beforeVideoPath,
|
||||
timeoutMs: options.requestTimeoutMs,
|
||||
prContext
|
||||
})
|
||||
|
||||
const videoStat = await stat(video.videoPath)
|
||||
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
|
||||
const outputPath = resolve(
|
||||
options.outputDir,
|
||||
`${video.platformName}${passSegment}-qa-video-report.md`
|
||||
)
|
||||
|
||||
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
|
||||
platformName: video.platformName,
|
||||
model: options.model,
|
||||
videoPath: video.videoPath,
|
||||
videoSizeBytes: videoStat.size,
|
||||
reviewText,
|
||||
targetUrl: options.targetUrl || undefined
|
||||
}
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
reportInput.beforeVideoPath = beforeVideoPath
|
||||
reportInput.beforeVideoSizeBytes = beforeStat.size
|
||||
}
|
||||
|
||||
const reportMarkdown = buildReportMarkdown(reportInput)
|
||||
|
||||
await mkdir(dirname(outputPath), { recursive: true })
|
||||
await writeFile(outputPath, reportMarkdown, 'utf-8')
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
|
||||
)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const options = parseCliOptions(process.argv.slice(2))
|
||||
const candidates = await collectVideoCandidates(options.artifactsDir)
|
||||
|
||||
if (candidates.length === 0) {
|
||||
process.stdout.write(
|
||||
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
const selectedVideo = selectVideoCandidateByFile(candidates, {
|
||||
artifactsDir: options.artifactsDir,
|
||||
videoFile: options.videoFile
|
||||
})
|
||||
|
||||
process.stdout.write(
|
||||
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
|
||||
)
|
||||
|
||||
if (options.dryRun) {
|
||||
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
|
||||
return
|
||||
}
|
||||
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) {
|
||||
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
|
||||
}
|
||||
|
||||
await reviewVideo(selectedVideo, options, apiKey)
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
void main().catch((error: unknown) => {
|
||||
const message = errorToString(error)
|
||||
process.stderr.write(`qa-video-review failed: ${message}\n`)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
@@ -1,513 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA CLI — simplified entry point for local & CI QA runs
|
||||
*
|
||||
* Usage:
|
||||
* pnpm qa 10253 # auto-detects issue vs PR
|
||||
* pnpm qa https://github.com/.../pull/10270
|
||||
* pnpm qa 10270 -t base # test PR base (reproduce bug)
|
||||
* pnpm qa 10270 -t both # test base + head
|
||||
* pnpm qa --uncommitted # test local uncommitted changes
|
||||
*
|
||||
* Automatically loads .env.local / .env for GEMINI_API_KEY, ANTHROPIC_API_KEY.
|
||||
* Results are written to .comfy-qa/<number>/ by default.
|
||||
*/
|
||||
|
||||
import { parseArgs } from 'node:util'
|
||||
import { config } from 'dotenv'
|
||||
import { existsSync, mkdirSync, writeFileSync } from 'fs'
|
||||
import { dirname, resolve } from 'path'
|
||||
import { execSync, spawn, spawnSync } from 'child_process'
|
||||
import { fileURLToPath } from 'url'
|
||||
|
||||
// ── Constants ──
|
||||
|
||||
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
|
||||
const RECORD_SCRIPT = resolve(SCRIPT_DIR, 'qa-record.ts')
|
||||
const DEFAULT_REPO = 'Comfy-Org/ComfyUI_frontend'
|
||||
const VALID_TARGETS = ['head', 'base', 'both'] as const
|
||||
const CLOUD_FALLBACK_URL = 'https://testcloud.comfy.org/'
|
||||
type PrTarget = (typeof VALID_TARGETS)[number]
|
||||
type TargetType = 'issue' | 'pr'
|
||||
|
||||
// ── Load .env.local / .env ──
|
||||
|
||||
for (const f of ['.env.local', '.env']) {
|
||||
if (existsSync(f)) {
|
||||
config({ path: f })
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// ── Parse CLI ──
|
||||
|
||||
const { values, positionals } = tryParseArgs()
|
||||
|
||||
if (values.help) {
|
||||
printUsage()
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const serverUrl =
|
||||
values.url || process.env.DEV_SERVER_COMFYUI_URL || 'http://127.0.0.1:8188'
|
||||
|
||||
const prTarget = values.target as PrTarget
|
||||
if (!VALID_TARGETS.includes(prTarget)) {
|
||||
console.error(
|
||||
`Invalid --target "${prTarget}". Must be one of: ${VALID_TARGETS.join(', ')}`
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Ensure server is reachable (may fall back to cloud) ──
|
||||
|
||||
const resolvedServerUrl = await ensureServer(serverUrl)
|
||||
|
||||
// ── Dispatch by mode ──
|
||||
|
||||
if (values.uncommitted) {
|
||||
runUncommitted()
|
||||
} else {
|
||||
const input = positionals[0]
|
||||
if (!input) {
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
runTarget(input)
|
||||
}
|
||||
|
||||
// ── Mode: uncommitted changes ──
|
||||
|
||||
function runUncommitted(): never {
|
||||
const diff = shell('git diff && git diff --staged')
|
||||
if (!diff.trim()) {
|
||||
console.error('No uncommitted changes found')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const outputDir = resolveOutputDir('.comfy-qa/local')
|
||||
const diffFile = writeTmpFile(outputDir, 'uncommitted.diff', diff)
|
||||
|
||||
logHeader({ label: 'uncommitted changes', outputDir })
|
||||
const code = runQaRecord('after', diffFile, outputDir)
|
||||
exit(code, outputDir)
|
||||
}
|
||||
|
||||
// ── Mode: issue or PR by number/URL ──
|
||||
|
||||
function runTarget(input: string): never {
|
||||
const { targetType, number, repo } = resolveTarget(input)
|
||||
const outputDir = resolveOutputDir(`.comfy-qa/${number}`)
|
||||
|
||||
logHeader({
|
||||
label: `${targetType} #${number} (${repo})`,
|
||||
outputDir,
|
||||
extra: targetType === 'pr' ? `Target: ${prTarget}` : undefined
|
||||
})
|
||||
|
||||
const diffFile =
|
||||
targetType === 'issue'
|
||||
? fetchIssue(number, repo, outputDir)
|
||||
: fetchPR(number, repo, outputDir)
|
||||
|
||||
let exitCode: number
|
||||
if (targetType === 'issue') {
|
||||
exitCode = runQaRecord('reproduce', diffFile, outputDir)
|
||||
} else if (prTarget === 'both') {
|
||||
exitCode = runPrBoth(diffFile, outputDir)
|
||||
} else if (prTarget === 'base') {
|
||||
exitCode = runQaRecord('before', diffFile, outputDir)
|
||||
} else {
|
||||
exitCode = runQaRecord('after', diffFile, outputDir)
|
||||
}
|
||||
|
||||
exit(exitCode, outputDir)
|
||||
}
|
||||
|
||||
// ── PR both phases ──
|
||||
|
||||
function runPrBoth(diffFile: string, outputDir: string): number {
|
||||
console.warn('\n=== Phase 1: Reproduce bug on base ===')
|
||||
const baseDir = resolve(outputDir, 'base')
|
||||
mkdirSync(baseDir, { recursive: true })
|
||||
const baseCode = runQaRecord('before', diffFile, baseDir)
|
||||
if (baseCode !== 0) {
|
||||
console.warn('Base phase failed, continuing to head...')
|
||||
}
|
||||
|
||||
console.warn('\n=== Phase 2: Demonstrate fix on head ===')
|
||||
const headDir = resolve(outputDir, 'head')
|
||||
mkdirSync(headDir, { recursive: true })
|
||||
return runQaRecord('after', diffFile, headDir)
|
||||
}
|
||||
|
||||
// ── Target resolution ──
|
||||
|
||||
function resolveTarget(input: string): {
|
||||
targetType: TargetType
|
||||
number: string
|
||||
repo: string
|
||||
} {
|
||||
const urlMatch = input.match(
|
||||
/github\.com\/([^/]+\/[^/]+)\/(issues|pull)\/(\d+)/
|
||||
)
|
||||
|
||||
if (urlMatch) {
|
||||
return {
|
||||
repo: urlMatch[1],
|
||||
targetType: urlMatch[2] === 'pull' ? 'pr' : 'issue',
|
||||
number: urlMatch[3]
|
||||
}
|
||||
}
|
||||
|
||||
if (/^\d+$/.test(input)) {
|
||||
return {
|
||||
repo: DEFAULT_REPO,
|
||||
targetType: detectType(input, DEFAULT_REPO),
|
||||
number: input
|
||||
}
|
||||
}
|
||||
|
||||
console.error(`Cannot parse target: ${input}`)
|
||||
console.error('Expected a GitHub URL or issue/PR number')
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
function detectType(number: string, repo: string): TargetType {
|
||||
try {
|
||||
const result = execSync(
|
||||
`gh api repos/${repo}/issues/${number} --jq 'has("pull_request")'`,
|
||||
{ encoding: 'utf-8', timeout: 15000, stdio: ['pipe', 'pipe', 'pipe'] }
|
||||
)
|
||||
return result.trim() === 'true' ? 'pr' : 'issue'
|
||||
} catch {
|
||||
return 'issue'
|
||||
}
|
||||
}
|
||||
|
||||
// ── Data fetching ──
|
||||
|
||||
function fetchIssue(number: string, repo: string, outputDir: string): string {
|
||||
console.warn(`Fetching issue #${number}...`)
|
||||
const body = shell(
|
||||
`gh issue view ${number} --repo ${repo} --json title,body,labels --jq '"Title: " + .title + "\\n\\nLabels: " + ([.labels[].name] | join(", ")) + "\\n\\n" + .body'`
|
||||
)
|
||||
return writeTmpFile(outputDir, `issue-${number}.txt`, body)
|
||||
}
|
||||
|
||||
function fetchPR(number: string, repo: string, outputDir: string): string {
|
||||
console.warn(`Fetching PR #${number}...`)
|
||||
const prJson = shell(
|
||||
`gh pr view ${number} --repo ${repo} --json title,body,baseRefName,headRefName,baseRefOid,headRefOid`
|
||||
)
|
||||
const pr = JSON.parse(prJson) as {
|
||||
title: string
|
||||
body: string
|
||||
baseRefName: string
|
||||
headRefName: string
|
||||
baseRefOid: string
|
||||
headRefOid: string
|
||||
}
|
||||
|
||||
console.warn(` Base: ${pr.baseRefName} (${pr.baseRefOid.slice(0, 8)})`)
|
||||
console.warn(` Head: ${pr.headRefName} (${pr.headRefOid.slice(0, 8)})`)
|
||||
|
||||
let diff = ''
|
||||
try {
|
||||
diff = shell(`gh pr diff ${number} --repo ${repo}`)
|
||||
} catch {
|
||||
console.warn('Could not fetch PR diff')
|
||||
}
|
||||
|
||||
writeTmpFile(
|
||||
outputDir,
|
||||
'refs.json',
|
||||
JSON.stringify(
|
||||
{
|
||||
base: { ref: pr.baseRefName, sha: pr.baseRefOid },
|
||||
head: { ref: pr.headRefName, sha: pr.headRefOid }
|
||||
},
|
||||
null,
|
||||
2
|
||||
)
|
||||
)
|
||||
|
||||
return writeTmpFile(
|
||||
outputDir,
|
||||
`pr-${number}.txt`,
|
||||
`Title: ${pr.title}\n\n${pr.body}\n\n--- DIFF ---\n\n${diff}`
|
||||
)
|
||||
}
|
||||
|
||||
// ── QA record runner ──
|
||||
|
||||
function runQaRecord(
|
||||
mode: string,
|
||||
diffFile: string,
|
||||
outputDir: string
|
||||
): number {
|
||||
console.warn(`\nStarting QA ${mode} mode...\n`)
|
||||
const r = spawnSync(
|
||||
'pnpm',
|
||||
[
|
||||
'exec',
|
||||
'tsx',
|
||||
RECORD_SCRIPT,
|
||||
'--mode',
|
||||
mode,
|
||||
'--diff',
|
||||
diffFile,
|
||||
'--output-dir',
|
||||
outputDir,
|
||||
'--url',
|
||||
resolvedServerUrl
|
||||
],
|
||||
{ stdio: 'inherit', env: process.env }
|
||||
)
|
||||
return r.status ?? 1
|
||||
}
|
||||
|
||||
// ── Server management ──
|
||||
|
||||
async function ensureServer(url: string): Promise<string> {
|
||||
if (await isReachable(url)) {
|
||||
console.warn(`Server OK: ${url}`)
|
||||
return url
|
||||
}
|
||||
|
||||
console.warn(`Server not reachable at ${url}, attempting auto-start...`)
|
||||
const port = new URL(url).port || '8188'
|
||||
|
||||
// Strategy 1: comfy-cli (pip install comfy-cli)
|
||||
try {
|
||||
execSync('which comfy', { stdio: 'pipe' })
|
||||
console.warn('Starting ComfyUI via comfy-cli...')
|
||||
const proc = spawn(
|
||||
'comfy',
|
||||
['launch', '--background', '--', '--cpu', '--port', port],
|
||||
{
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
}
|
||||
)
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
} catch {
|
||||
// comfy-cli not available
|
||||
}
|
||||
|
||||
// Strategy 2: python main.py from TEST_COMFYUI_DIR or .comfy-qa/ComfyUI
|
||||
const comfyDir = findComfyUIDir()
|
||||
if (comfyDir) {
|
||||
console.warn(`Starting ComfyUI from ${comfyDir}...`)
|
||||
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
|
||||
cwd: comfyDir,
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
})
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
}
|
||||
|
||||
// Strategy 3: clone ComfyUI and start
|
||||
const cloneDir = resolve('.comfy-qa/ComfyUI')
|
||||
if (!existsSync(resolve(cloneDir, 'main.py'))) {
|
||||
console.warn('No ComfyUI installation found, cloning...')
|
||||
try {
|
||||
execSync(
|
||||
`git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git "${cloneDir}"`,
|
||||
{ stdio: 'inherit', timeout: 120000 }
|
||||
)
|
||||
console.warn('Installing ComfyUI dependencies...')
|
||||
execSync('pip install -r requirements.txt', {
|
||||
cwd: cloneDir,
|
||||
stdio: 'inherit',
|
||||
timeout: 300000
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Clone/install failed: ${err instanceof Error ? err.message : err}`
|
||||
)
|
||||
}
|
||||
}
|
||||
if (existsSync(resolve(cloneDir, 'main.py'))) {
|
||||
console.warn(`Starting ComfyUI from ${cloneDir}...`)
|
||||
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
|
||||
cwd: cloneDir,
|
||||
stdio: 'ignore',
|
||||
detached: true
|
||||
})
|
||||
proc.unref()
|
||||
await waitForServer(url, 120000)
|
||||
return url
|
||||
}
|
||||
|
||||
// Strategy 4: fallback to testcloud
|
||||
console.warn(`Local server failed. Falling back to ${CLOUD_FALLBACK_URL}`)
|
||||
if (await isReachable(CLOUD_FALLBACK_URL)) {
|
||||
console.warn(`Cloud server OK: ${CLOUD_FALLBACK_URL}`)
|
||||
return CLOUD_FALLBACK_URL
|
||||
}
|
||||
|
||||
console.error(`
|
||||
No ComfyUI server available. Tried:
|
||||
1. ${url} (not reachable)
|
||||
2. comfy-cli (not installed)
|
||||
3. Local ComfyUI installation (not found)
|
||||
4. ${CLOUD_FALLBACK_URL} (not reachable)
|
||||
|
||||
Install: pip install comfy-cli && comfy install && comfy launch --cpu
|
||||
`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
function findComfyUIDir(): string | undefined {
|
||||
const candidates = [
|
||||
process.env.TEST_COMFYUI_DIR,
|
||||
resolve('.comfy-qa/ComfyUI'),
|
||||
'/home/ComfyUI'
|
||||
]
|
||||
return candidates.find((d) => d && existsSync(resolve(d, 'main.py')))
|
||||
}
|
||||
|
||||
async function isReachable(url: string): Promise<boolean> {
|
||||
try {
|
||||
const controller = new AbortController()
|
||||
const timeout = setTimeout(() => controller.abort(), 5000)
|
||||
const res = await fetch(url, { signal: controller.signal })
|
||||
clearTimeout(timeout)
|
||||
return res.ok || res.status === 200 || res.status === 304
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function waitForServer(url: string, timeoutMs: number): Promise<void> {
|
||||
const start = Date.now()
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
if (await isReachable(url)) {
|
||||
console.warn('Server is ready')
|
||||
return
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 2000))
|
||||
}
|
||||
console.error(`Server did not start within ${timeoutMs / 1000}s`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Utilities ──
|
||||
|
||||
function shell(cmd: string): string {
|
||||
return execSync(cmd, { encoding: 'utf-8', timeout: 30000 })
|
||||
}
|
||||
|
||||
function writeTmpFile(
|
||||
outputDir: string,
|
||||
filename: string,
|
||||
content: string
|
||||
): string {
|
||||
const tmpDir = resolve(outputDir, '.tmp')
|
||||
mkdirSync(tmpDir, { recursive: true })
|
||||
const filePath = resolve(tmpDir, filename)
|
||||
writeFileSync(filePath, content)
|
||||
return filePath
|
||||
}
|
||||
|
||||
function resolveOutputDir(defaultPath: string): string {
|
||||
const dir = values.output ? resolve(values.output) : resolve(defaultPath)
|
||||
mkdirSync(dir, { recursive: true })
|
||||
return dir
|
||||
}
|
||||
|
||||
function logHeader(opts: { label: string; outputDir: string; extra?: string }) {
|
||||
console.warn(`QA target: ${opts.label}`)
|
||||
console.warn(`Output: ${opts.outputDir}`)
|
||||
console.warn(`Server: ${resolvedServerUrl}`)
|
||||
if (values.ref) console.warn(`Ref: ${values.ref}`)
|
||||
if (opts.extra) console.warn(opts.extra)
|
||||
}
|
||||
|
||||
function exit(code: number, outputDir: string): never {
|
||||
console.warn('\n=== QA Complete ===')
|
||||
console.warn(`Results: ${outputDir}`)
|
||||
try {
|
||||
console.warn(shell(`ls -la "${outputDir}"`))
|
||||
} catch {
|
||||
// not critical
|
||||
}
|
||||
process.exit(code)
|
||||
}
|
||||
|
||||
function tryParseArgs() {
|
||||
try {
|
||||
const parsed = parseArgs({
|
||||
args: process.argv.slice(2),
|
||||
options: {
|
||||
target: { type: 'string', short: 't', default: 'head' },
|
||||
uncommitted: { type: 'boolean', default: false },
|
||||
url: { type: 'string', default: '' },
|
||||
ref: { type: 'string', default: '' },
|
||||
output: { type: 'string', short: 'o', default: '' },
|
||||
help: { type: 'boolean', short: 'h', default: false }
|
||||
},
|
||||
allowPositionals: true,
|
||||
strict: true
|
||||
})
|
||||
return {
|
||||
values: parsed.values as {
|
||||
target: string
|
||||
uncommitted: boolean
|
||||
url: string
|
||||
ref: string
|
||||
output: string
|
||||
help: boolean
|
||||
},
|
||||
positionals: parsed.positionals
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`Error: ${err instanceof Error ? err.message : err}\n`)
|
||||
printUsage()
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
function printUsage() {
|
||||
console.warn(`
|
||||
QA CLI — Reproduce issues & test PRs for ComfyUI frontend
|
||||
|
||||
Usage:
|
||||
pnpm qa <number|url> [options]
|
||||
pnpm qa --uncommitted
|
||||
|
||||
Targets:
|
||||
10253 Number (auto-detects issue vs PR via gh CLI)
|
||||
https://github.com/Comfy-Org/ComfyUI_frontend/issues/10253
|
||||
https://github.com/Comfy-Org/ComfyUI_frontend/pull/10270
|
||||
|
||||
Options:
|
||||
-t, --target <head|base|both>
|
||||
For PRs: which ref to test (default: head)
|
||||
head — test the fix (PR head)
|
||||
base — reproduce the bug (PR base)
|
||||
both — base then head
|
||||
--uncommitted Test local uncommitted changes
|
||||
--url <url> ComfyUI server URL (default: from .env or http://127.0.0.1:8188)
|
||||
--ref <ref> Git ref to test against
|
||||
-o, --output <dir> Override output directory (default: .comfy-qa/<number>)
|
||||
-h, --help Show this help
|
||||
|
||||
Environment (auto-loaded from .env.local or .env):
|
||||
GEMINI_API_KEY Required — used for PR analysis, video review, TTS
|
||||
ANTHROPIC_API_KEY Optional locally — Claude Agent SDK auto-detects Claude Code session
|
||||
|
||||
Examples:
|
||||
pnpm qa 10253 # reproduce an issue
|
||||
pnpm qa 10270 # test PR head (the fix)
|
||||
pnpm qa 10270 -t base # reproduce bug on PR base
|
||||
pnpm qa 10270 -t both # test base + head
|
||||
pnpm qa --uncommitted # test local changes
|
||||
`)
|
||||
}
|
||||
@@ -1,246 +0,0 @@
|
||||
---
|
||||
name: hardening-flaky-e2e-tests
|
||||
description: 'Diagnoses and fixes flaky Playwright e2e tests by replacing race-prone patterns with retry-safe alternatives. Use when triaging CI flakes, hardening spec files, fixing timing races, or asked to stabilize browser tests. Triggers on: flaky, flake, harden, stabilize, race condition in e2e, intermittent failure.'
|
||||
---
|
||||
|
||||
# Hardening Flaky E2E Tests
|
||||
|
||||
Fix flaky Playwright specs by identifying race-prone patterns and replacing them with retry-safe alternatives. This skill covers diagnosis, pattern matching, and mechanical transforms — not writing new tests (see `writing-playwright-tests` for that).
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Gather CI Evidence
|
||||
|
||||
```bash
|
||||
gh run list --workflow=ci-test.yaml --limit=5
|
||||
gh run download <run-id> -n playwright-report
|
||||
```
|
||||
|
||||
- Open `report.json` and search for `"status": "flaky"` entries.
|
||||
- Collect file paths, test titles, and error messages.
|
||||
- Do NOT trust green checks alone — flaky tests that passed on retry still need fixing.
|
||||
- Use `error-context.md`, traces, and page snapshots before editing code.
|
||||
- Pull the newest run after each push instead of assuming the flaky set is unchanged.
|
||||
|
||||
### 2. Classify the Flake
|
||||
|
||||
Read the failing assertion and match it against the pattern table. Most flakes fall into one of these categories:
|
||||
|
||||
| # | Pattern | Signature in Code | Fix |
|
||||
| --- | ------------------------------------- | --------------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | **Snapshot-then-assert** | `expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
|
||||
| 2 | **Immediate count** | `const n = await loc.count(); expect(n).toBe(3)` | `await expect(loc).toHaveCount(3)` |
|
||||
| 3 | **nextFrame after menu click** | `clickMenuItem(x); nextFrame()` | `clickMenuItem(x); contextMenu.waitForHidden()` |
|
||||
| 4 | **Tight poll timeout** | `expect.poll(..., { timeout: 250 })` | ≥2000 ms; prefer default 5000 ms |
|
||||
| 5 | **Immediate evaluate after mutation** | `setSetting(k, v); expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
|
||||
| 6 | **Screenshot without readiness** | `loadWorkflow(); nextFrame(); toHaveScreenshot()` | `waitForNodes()` or poll state first |
|
||||
| 7 | **Non-deterministic node order** | `getNodeRefsByType('X')[0]` with >1 match | `getNodeRefById(id)` or guard `toHaveLength(1)` |
|
||||
| 8 | **Fake readiness helper** | Helper clicks but doesn't assert state | Remove; poll the actual value |
|
||||
| 9 | **Immediate graph state after drop** | `expect(await getLinkCount()).toBe(1)` | `await expect.poll(() => getLinkCount()).toBe(1)` |
|
||||
| 10 | **Immediate boundingBox/layout read** | `const box = await loc.boundingBox(); expect(box!.width)` | `await expect.poll(() => loc.boundingBox().then(b => b?.width))` |
|
||||
|
||||
### 3. Apply the Transform
|
||||
|
||||
#### Rule: Choose the Smallest Correct Assertion
|
||||
|
||||
- **Locator state** → use built-in retrying assertions: `toBeVisible()`, `toHaveText()`, `toHaveCount()`, `toHaveClass()`
|
||||
- **Single async value** → `expect.poll(() => asyncFn()).toBe(expected)`
|
||||
- **Multiple assertions that must settle together** → `expect(async () => { ... }).toPass()`
|
||||
- **Never** use `waitForTimeout()` to hide a race.
|
||||
|
||||
```typescript
|
||||
// ✅ Single value — use expect.poll
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.graph.links.length))
|
||||
.toBe(3)
|
||||
|
||||
// ✅ Locator count — use toHaveCount
|
||||
await expect(comfyPage.page.locator('.dom-widget')).toHaveCount(2)
|
||||
|
||||
// ✅ Multiple conditions — use toPass
|
||||
await expect(async () => {
|
||||
expect(await node1.getValue()).toBe('foo')
|
||||
expect(await node2.getValue()).toBe('bar')
|
||||
}).toPass({ timeout: 5000 })
|
||||
```
|
||||
|
||||
#### Rule: Wait for the Real Readiness Boundary
|
||||
|
||||
Visible is not always ready. Prefer user-facing assertions when possible; poll internal state only when there is no UI surface to assert on.
|
||||
|
||||
Common readiness boundaries:
|
||||
|
||||
| After this action... | Wait for... |
|
||||
| -------------------------------------- | ------------------------------------------------------------ |
|
||||
| Canvas interaction (drag, click node) | `await comfyPage.nextFrame()` |
|
||||
| Menu item click | `await contextMenu.waitForHidden()` |
|
||||
| Workflow load | `await comfyPage.workflow.loadWorkflow(...)` (built-in wait) |
|
||||
| Settings write | Poll the setting value with `expect.poll()` |
|
||||
| Node pin/bypass/collapse toggle | `await expect.poll(() => nodeRef.isPinned()).toBe(true)` |
|
||||
| Graph mutation (add/remove node, link) | Poll link/node count |
|
||||
| Clipboard write | Poll pasted value |
|
||||
| Screenshot | Ensure nodes are rendered: `waitForNodes()` or poll state |
|
||||
|
||||
#### Rule: Expose Locators for Retrying Assertions
|
||||
|
||||
When a helper returns a count via `await loc.count()`, callers can't use `toHaveCount()`. Expose the underlying `Locator` as a getter so callers choose between:
|
||||
|
||||
```typescript
|
||||
// Helper exposes locator
|
||||
get domWidgets(): Locator {
|
||||
return this.page.locator('.dom-widget')
|
||||
}
|
||||
|
||||
// Caller uses retrying assertion
|
||||
await expect(comfyPage.domWidgets).toHaveCount(2)
|
||||
```
|
||||
|
||||
Replace count methods with locator getters so callers can use retrying assertions directly.
|
||||
|
||||
#### Rule: Fix Check-then-Act Races in Helpers
|
||||
|
||||
```typescript
|
||||
// ❌ Race: count can change between check and waitFor
|
||||
const count = await locator.count()
|
||||
if (count > 0) {
|
||||
await locator.waitFor({ state: 'hidden' })
|
||||
}
|
||||
|
||||
// ✅ Direct: waitFor handles both cases
|
||||
await locator.waitFor({ state: 'hidden' })
|
||||
```
|
||||
|
||||
#### Rule: Remove force:true from Clicks
|
||||
|
||||
`force: true` bypasses actionability checks, hiding real animation/visibility races. Remove it and fix the underlying timing issue.
|
||||
|
||||
```typescript
|
||||
// ❌ Hides the race
|
||||
await closeButton.click({ force: true })
|
||||
|
||||
// ✅ Surfaces the real issue — fix with proper wait
|
||||
await closeButton.click()
|
||||
await dialog.waitForHidden()
|
||||
```
|
||||
|
||||
#### Rule: Handle Non-deterministic Element Order
|
||||
|
||||
When `getNodeRefsByType` returns multiple nodes, the order is not guaranteed. Don't use index `[0]` blindly.
|
||||
|
||||
```typescript
|
||||
// ❌ Assumes order
|
||||
const node = (await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode'))[0]
|
||||
|
||||
// ✅ Find by ID or proximity
|
||||
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
let target = nodes[0]
|
||||
for (const n of nodes) {
|
||||
const pos = await n.getPosition()
|
||||
if (Math.abs(pos.y - expectedY) < minDist) target = n
|
||||
}
|
||||
```
|
||||
|
||||
Or guard the assumption:
|
||||
|
||||
```typescript
|
||||
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
expect(nodes).toHaveLength(1)
|
||||
const node = nodes[0]
|
||||
```
|
||||
|
||||
#### Rule: Use toPass for Timing-sensitive Dismiss Guards
|
||||
|
||||
Some UI elements (e.g. LiteGraph's graphdialog) have built-in dismiss delays. Retry the entire dismiss action:
|
||||
|
||||
```typescript
|
||||
// ✅ Retry click+assert together
|
||||
await expect(async () => {
|
||||
await comfyPage.canvas.click({ position: { x: 10, y: 10 } })
|
||||
await expect(dialog).toBeHidden({ timeout: 500 })
|
||||
}).toPass({ timeout: 5000 })
|
||||
```
|
||||
|
||||
### 4. Keep Changes Narrow
|
||||
|
||||
- Shared helpers should drive setup to a stable boundary.
|
||||
- Do not encode one-spec timing assumptions into generic helpers.
|
||||
- If a race only matters to one spec, prefer a local wait in that spec.
|
||||
- If a helper fails before the real test begins, remove or relax the brittle precondition and let downstream UI interaction prove readiness.
|
||||
|
||||
### 5. Verify Narrowly
|
||||
|
||||
```bash
|
||||
# Targeted rerun with repetition
|
||||
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts --repeat-each 10
|
||||
|
||||
# Single test by line number (avoids grep quoting issues on Windows)
|
||||
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts:42
|
||||
```
|
||||
|
||||
- Use `--repeat-each 10` for targeted flake verification (use 20 for single test cases).
|
||||
- Verify with the smallest command that exercises the flaky path.
|
||||
|
||||
### 6. Watch CI E2E Runs
|
||||
|
||||
After pushing, use `gh` to monitor the E2E workflow:
|
||||
|
||||
```bash
|
||||
# Find the run for the current branch
|
||||
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1
|
||||
|
||||
# Watch it live (blocks until complete, streams logs)
|
||||
gh run watch <run-id>
|
||||
|
||||
# One-liner: find and watch the latest E2E run for the current branch
|
||||
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId" | xargs gh run watch
|
||||
```
|
||||
|
||||
On Windows (PowerShell):
|
||||
|
||||
```powershell
|
||||
# One-liner equivalent
|
||||
gh run watch (gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId")
|
||||
```
|
||||
|
||||
After the run completes:
|
||||
|
||||
```bash
|
||||
# Download the Playwright report artifact
|
||||
gh run download <run-id> -n playwright-report
|
||||
|
||||
# View the run summary in browser
|
||||
gh run view <run-id> --web
|
||||
```
|
||||
|
||||
Also watch the unit test workflow in parallel if you changed helpers:
|
||||
|
||||
```bash
|
||||
gh run list --workflow="CI: Tests Unit" --branch=$(git branch --show-current) --limit=1
|
||||
```
|
||||
|
||||
### 7. Pre-merge Checklist
|
||||
|
||||
Before merging a flaky-test fix, confirm:
|
||||
|
||||
- [ ] The latest CI artifact was inspected directly
|
||||
- [ ] The root cause is stated as a race or readiness mismatch
|
||||
- [ ] The fix waits on the real readiness boundary
|
||||
- [ ] The assertion primitive matches the job (poll vs toHaveCount vs toPass)
|
||||
- [ ] The fix stays local unless a shared helper truly owns the race
|
||||
- [ ] Local verification uses a targeted rerun
|
||||
- [ ] No behavioral changes to the test — only timing/retry strategy updated
|
||||
|
||||
## Local Noise — Do Not Fix
|
||||
|
||||
These are local distractions, not CI root causes:
|
||||
|
||||
- Missing local input fixture files required by the test path
|
||||
- Missing local models directory
|
||||
- Teardown `EPERM` while restoring the local browser-test user data directory
|
||||
- Local screenshot baseline differences on Windows
|
||||
|
||||
Rules:
|
||||
|
||||
- First confirm whether it blocks the exact flaky path under investigation.
|
||||
- Do not commit temporary local assets used only for verification.
|
||||
- Do not commit local screenshot baselines.
|
||||
@@ -46,11 +46,3 @@ ALGOLIA_API_KEY=684d998c36b67a9a9fce8fc2d8860579
|
||||
# SENTRY_ORG=comfy-org
|
||||
# SENTRY_PROJECT=cloud-frontend-staging
|
||||
# SENTRY_PROJECT_PROD= # prod project slug for sourcemap uploads
|
||||
|
||||
# ── QA Skill (scripts/qa-record.ts) ──
|
||||
# Required for automated bug reproduction via `pnpm exec tsx scripts/qa-record.ts`
|
||||
# GEMINI_API_KEY is required — used for PR analysis, video review, and TTS narration
|
||||
GEMINI_API_KEY=
|
||||
# ANTHROPIC_API_KEY is optional locally — Claude Agent SDK auto-detects Claude Code session
|
||||
# Required in CI (set as GitHub Actions secret)
|
||||
# ANTHROPIC_API_KEY=
|
||||
|
||||
1094
.github/workflows/pr-qa.yaml
vendored
1094
.github/workflows/pr-qa.yaml
vendored
File diff suppressed because it is too large
Load Diff
3
.gitignore
vendored
3
.gitignore
vendored
@@ -67,9 +67,6 @@ dist.zip
|
||||
/temp/
|
||||
/tmp/
|
||||
|
||||
# QA local output
|
||||
/.comfy-qa/
|
||||
|
||||
# Generated JSON Schemas
|
||||
/schemas/
|
||||
|
||||
|
||||
@@ -83,21 +83,6 @@ await expect
|
||||
a different reproduction pattern.
|
||||
- Verify with the smallest command that exercises the flaky path.
|
||||
|
||||
## 7. Common Flake Patterns
|
||||
|
||||
| Pattern | Bad | Fix |
|
||||
| ------------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| **Snapshot-then-assert** | `expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
|
||||
| **Immediate boundingBox/layout read** | `const box = await loc.boundingBox(); expect(box!.width).toBe(w)` | `await expect.poll(() => loc.boundingBox().then(b => b?.width)).toBe(w)` |
|
||||
| **Immediate graph state after drop** | `expect(await getLinkCount()).toBe(1)` | `await expect.poll(() => getLinkCount()).toBe(1)` |
|
||||
| **Fake readiness helper** | Helper that clicks but doesn't assert state | Remove; poll the actual value |
|
||||
| **nextFrame after menu click** | `clickMenuItem(x); nextFrame()` | `clickMenuItem(x); contextMenu.waitForHidden()` |
|
||||
| **Tight poll timeout** | `expect.poll(..., { timeout: 250 })` | ≥2000ms; prefer default (5000ms) |
|
||||
| **Immediate count()** | `const n = await loc.count(); expect(n).toBe(3)` | `await expect(loc).toHaveCount(3)` |
|
||||
| **Immediate evaluate after mutation** | `setSetting(); expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
|
||||
| **Screenshot without readiness** | `loadWorkflow(); nextFrame(); toHaveScreenshot()` | `waitForNodes()` or poll state first |
|
||||
| **Non-deterministic node order** | `getNodeRefsByType('X')[0]` with >1 match | `getNodeRefById(id)` or guard `toHaveLength(1)` |
|
||||
|
||||
## Current Local Noise
|
||||
|
||||
These are local distractions, not automatic CI root causes:
|
||||
|
||||
@@ -210,8 +210,8 @@ Most common testing needs are already addressed by these helpers, which will mak
|
||||
|
||||
```typescript
|
||||
// Prefer this:
|
||||
await expect.poll(() => node.isPinned()).toBe(true)
|
||||
await expect.poll(() => node.getProperty('title')).toBe('Expected Title')
|
||||
expect(await node.isPinned()).toBe(true)
|
||||
expect(await node.getProperty('title')).toBe('Expected Title')
|
||||
|
||||
// Over this - only use when needed:
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('state.png')
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
# Blender 5.2.0 Alpha
|
||||
# www.blender.org
|
||||
mtllib Untitled.mtl
|
||||
o Cube
|
||||
v 2.857396 2.486626 -0.081892
|
||||
v 2.857396 0.486626 -0.081892
|
||||
v 2.857396 2.486626 1.918108
|
||||
v 2.857396 0.486626 1.918108
|
||||
v 0.857396 2.486626 -0.081892
|
||||
v 0.857396 0.486626 -0.081892
|
||||
v 0.857396 2.486626 1.918108
|
||||
v 0.857396 0.486626 1.918108
|
||||
vn -0.0000 1.0000 -0.0000
|
||||
vn -0.0000 -0.0000 1.0000
|
||||
vn -1.0000 -0.0000 -0.0000
|
||||
vn -0.0000 -1.0000 -0.0000
|
||||
vn 1.0000 -0.0000 -0.0000
|
||||
vn -0.0000 -0.0000 -1.0000
|
||||
vt 0.625000 0.500000
|
||||
vt 0.875000 0.500000
|
||||
vt 0.875000 0.750000
|
||||
vt 0.625000 0.750000
|
||||
vt 0.375000 0.750000
|
||||
vt 0.625000 1.000000
|
||||
vt 0.375000 1.000000
|
||||
vt 0.375000 0.000000
|
||||
vt 0.625000 0.000000
|
||||
vt 0.625000 0.250000
|
||||
vt 0.375000 0.250000
|
||||
vt 0.125000 0.500000
|
||||
vt 0.375000 0.500000
|
||||
vt 0.125000 0.750000
|
||||
s 0
|
||||
usemtl Material
|
||||
f 1/1/1 5/2/1 7/3/1 3/4/1
|
||||
f 4/5/2 3/4/2 7/6/2 8/7/2
|
||||
f 8/8/3 7/9/3 5/10/3 6/11/3
|
||||
f 6/12/4 2/13/4 4/5/4 8/14/4
|
||||
f 2/13/5 1/1/5 3/4/5 4/5/5
|
||||
f 6/11/6 5/10/6 1/1/6 2/13/6
|
||||
@@ -392,8 +392,9 @@ export class ComfyPage {
|
||||
await modal.waitFor({ state: 'hidden' })
|
||||
}
|
||||
|
||||
get domWidgets(): Locator {
|
||||
return this.page.locator('.dom-widget')
|
||||
/** Get number of DOM widgets on the canvas. */
|
||||
async getDOMWidgetCount() {
|
||||
return await this.page.locator('.dom-widget').count()
|
||||
}
|
||||
|
||||
async setFocusMode(focusMode: boolean) {
|
||||
|
||||
@@ -48,6 +48,13 @@ export class VueNodeHelpers {
|
||||
return await this.nodes.count()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get count of selected Vue nodes
|
||||
*/
|
||||
async getSelectedNodeCount(): Promise<number> {
|
||||
return await this.selectedNodes.count()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all Vue node IDs currently in the DOM
|
||||
*/
|
||||
|
||||
@@ -25,7 +25,7 @@ export class BaseDialog {
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.closeButton.click()
|
||||
await this.closeButton.click({ force: true })
|
||||
await this.waitForHidden()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -65,9 +65,21 @@ export class ContextMenu {
|
||||
}
|
||||
|
||||
async waitForHidden(): Promise<void> {
|
||||
const waitIfExists = async (locator: Locator, menuName: string) => {
|
||||
const count = await locator.count()
|
||||
if (count > 0) {
|
||||
await locator.waitFor({ state: 'hidden' }).catch((error: Error) => {
|
||||
console.warn(
|
||||
`[waitForHidden] ${menuName} waitFor failed:`,
|
||||
error.message
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
await Promise.all([
|
||||
this.primeVueMenu.waitFor({ state: 'hidden' }),
|
||||
this.litegraphMenu.waitFor({ state: 'hidden' })
|
||||
waitIfExists(this.primeVueMenu, 'primeVueMenu'),
|
||||
waitIfExists(this.litegraphMenu, 'litegraphMenu')
|
||||
])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -168,14 +168,10 @@ export class WorkflowsSidebarTab extends SidebarTab {
|
||||
.allInnerTexts()
|
||||
}
|
||||
|
||||
get activeWorkflowLabel(): Locator {
|
||||
return this.root.locator(
|
||||
'.comfyui-workflows-open .p-tree-node-selected .node-label'
|
||||
)
|
||||
}
|
||||
|
||||
async getActiveWorkflowName() {
|
||||
return await this.activeWorkflowLabel.innerText()
|
||||
return await this.root
|
||||
.locator('.comfyui-workflows-open .p-tree-node-selected .node-label')
|
||||
.innerText()
|
||||
}
|
||||
|
||||
async getTopLevelSavedWorkflowNames() {
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
import { comfyPageFixture } from '@e2e/fixtures/ComfyPage'
|
||||
import { Load3DHelper } from '@e2e/tests/load3d/Load3DHelper'
|
||||
import { Load3DViewerHelper } from '@e2e/tests/load3d/Load3DViewerHelper'
|
||||
|
||||
export const load3dTest = comfyPageFixture.extend<{
|
||||
load3d: Load3DHelper
|
||||
}>({
|
||||
load3d: async ({ comfyPage }, use) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
await comfyPage.workflow.loadWorkflow('3d/load3d_node')
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
await use(new Load3DHelper(node))
|
||||
}
|
||||
})
|
||||
|
||||
export const load3dViewerTest = load3dTest.extend<{
|
||||
viewer: Load3DViewerHelper
|
||||
}>({
|
||||
viewer: async ({ comfyPage }, use) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Load3D.3DViewerEnable', true)
|
||||
await use(new Load3DViewerHelper(comfyPage.page))
|
||||
}
|
||||
})
|
||||
@@ -445,7 +445,7 @@ export class SubgraphHelper {
|
||||
await this.rightClickOutputSlot(slotName)
|
||||
}
|
||||
await this.comfyPage.contextMenu.clickLitegraphMenuItem('Remove Slot')
|
||||
await this.comfyPage.contextMenu.waitForHidden()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
async findSubgraphNodeId(): Promise<string> {
|
||||
|
||||
@@ -8,8 +8,14 @@ export class ToastHelper {
|
||||
return this.page.locator('.p-toast-message:visible')
|
||||
}
|
||||
|
||||
get toastErrors(): Locator {
|
||||
return this.page.locator('.p-toast-message.p-toast-message-error')
|
||||
async getToastErrorCount(): Promise<number> {
|
||||
return await this.page
|
||||
.locator('.p-toast-message.p-toast-message-error')
|
||||
.count()
|
||||
}
|
||||
|
||||
async getVisibleToastCount(): Promise<number> {
|
||||
return await this.visibleToasts.count()
|
||||
}
|
||||
|
||||
async closeToasts(requireCount = 0): Promise<void> {
|
||||
|
||||
@@ -112,7 +112,9 @@ export const TestIds = {
|
||||
decrement: 'decrement',
|
||||
increment: 'increment',
|
||||
domWidgetTextarea: 'dom-widget-textarea',
|
||||
subgraphEnterButton: 'subgraph-enter-button'
|
||||
subgraphEnterButton: 'subgraph-enter-button',
|
||||
formDropdownMenu: 'form-dropdown-menu',
|
||||
formDropdownTrigger: 'form-dropdown-trigger'
|
||||
},
|
||||
builder: {
|
||||
footerNav: 'builder-footer-nav',
|
||||
@@ -155,12 +157,6 @@ export const TestIds = {
|
||||
errors: {
|
||||
imageLoadError: 'error-loading-image',
|
||||
videoLoadError: 'error-loading-video'
|
||||
},
|
||||
loading: {
|
||||
overlay: 'loading-overlay'
|
||||
},
|
||||
load3dViewer: {
|
||||
sidebar: 'load3d-viewer-sidebar'
|
||||
}
|
||||
} as const
|
||||
|
||||
@@ -191,5 +187,3 @@ export type TestIdValue =
|
||||
| (typeof TestIds.subgraphEditor)[keyof typeof TestIds.subgraphEditor]
|
||||
| (typeof TestIds.queue)[keyof typeof TestIds.queue]
|
||||
| (typeof TestIds.errors)[keyof typeof TestIds.errors]
|
||||
| (typeof TestIds.loading)[keyof typeof TestIds.loading]
|
||||
| (typeof TestIds.load3dViewer)[keyof typeof TestIds.load3dViewer]
|
||||
|
||||
@@ -26,12 +26,9 @@ export class ManageGroupNode {
|
||||
await this.footer.getByText('Close').click()
|
||||
}
|
||||
|
||||
get selectedNodeTypeSelect(): Locator {
|
||||
return this.header.locator('select').first()
|
||||
}
|
||||
|
||||
async getSelectedNodeType() {
|
||||
return await this.selectedNodeTypeSelect.inputValue()
|
||||
const select = this.header.locator('select').first()
|
||||
return await select.inputValue()
|
||||
}
|
||||
|
||||
async selectNode(name: string) {
|
||||
|
||||
@@ -22,24 +22,7 @@ export async function getPromotedWidgets(
|
||||
): Promise<PromotedWidgetEntry[]> {
|
||||
const raw = await comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.canvas.graph!.getNodeById(id)
|
||||
const widgets = node?.widgets ?? []
|
||||
|
||||
// Read the live promoted widget views from the host node instead of the
|
||||
// serialized proxyWidgets snapshot, which can lag behind the current graph
|
||||
// state during promotion and cleanup flows.
|
||||
return widgets.flatMap((widget) => {
|
||||
if (
|
||||
widget &&
|
||||
typeof widget === 'object' &&
|
||||
'sourceNodeId' in widget &&
|
||||
typeof widget.sourceNodeId === 'string' &&
|
||||
'sourceWidgetName' in widget &&
|
||||
typeof widget.sourceWidgetName === 'string'
|
||||
) {
|
||||
return [[widget.sourceNodeId, widget.sourceWidgetName]]
|
||||
}
|
||||
return []
|
||||
})
|
||||
return node?.properties?.proxyWidgets ?? []
|
||||
}, nodeId)
|
||||
|
||||
return normalizePromotedWidgets(raw)
|
||||
|
||||
@@ -22,10 +22,10 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
|
||||
}) => {
|
||||
// Enable change auto-queue mode
|
||||
const queueOpts = await comfyPage.actionbar.queueButton.toggleOptions()
|
||||
await expect.poll(() => queueOpts.getMode()).toBe('disabled')
|
||||
expect(await queueOpts.getMode()).toBe('disabled')
|
||||
await queueOpts.setMode('change')
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => queueOpts.getMode()).toBe('change')
|
||||
expect(await queueOpts.getMode()).toBe('change')
|
||||
await comfyPage.actionbar.queueButton.toggleOptions()
|
||||
|
||||
// Intercept the prompt queue endpoint
|
||||
@@ -124,8 +124,6 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
|
||||
force: true
|
||||
}
|
||||
)
|
||||
await expect(comfyPage.actionbar.root.locator('.actionbar')).toHaveClass(
|
||||
/static/
|
||||
)
|
||||
expect(await comfyPage.actionbar.isDocked()).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '@e2e/fixtures/ComfyPage'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
/**
|
||||
* Default workflow widget inputs as [nodeId, widgetName] tuples.
|
||||
@@ -92,23 +93,19 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
const overlay = comfyPage.page.locator('.p-select-overlay').first()
|
||||
await expect(overlay).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
overlay.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
})
|
||||
const isInViewport = await overlay.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
.toBe(true)
|
||||
})
|
||||
expect(isInViewport).toBe(true)
|
||||
|
||||
await expect
|
||||
.poll(() => overlay.evaluate(isClippedByAnyAncestor))
|
||||
.toBe(false)
|
||||
const isClipped = await overlay.evaluate(isClippedByAnyAncestor)
|
||||
expect(isClipped).toBe(false)
|
||||
})
|
||||
|
||||
test('FormDropdown popup is not clipped in app mode panel', async ({
|
||||
@@ -141,27 +138,23 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
const dropdownButton = imageRow.locator('button:has(> span)').first()
|
||||
await dropdownButton.click()
|
||||
|
||||
// The unstyled PrimeVue Popover renders with role="dialog".
|
||||
// Locate the one containing the image grid (filter buttons like "All", "Inputs").
|
||||
const popover = comfyPage.appMode.imagePickerPopover
|
||||
await expect(popover).toBeVisible({ timeout: 5000 })
|
||||
const menu = comfyPage.page
|
||||
.getByTestId(TestIds.widgets.formDropdownMenu)
|
||||
.first()
|
||||
await expect(menu).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
popover.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
})
|
||||
const isInViewport = await menu.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
.toBe(true)
|
||||
})
|
||||
expect(isInViewport).toBe(true)
|
||||
|
||||
await expect
|
||||
.poll(() => popover.evaluate(isClippedByAnyAncestor))
|
||||
.toBe(false)
|
||||
const isClipped = await menu.evaluate(isClippedByAnyAncestor)
|
||||
expect(isClipped).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '@e2e/fixtures/ComfyPage'
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
test.describe('App mode welcome states', { tag: '@ui' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
|
||||
@@ -103,15 +103,14 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
|
||||
const keyBadges = bottomPanel.shortcuts.keyBadges
|
||||
await keyBadges.first().waitFor({ state: 'visible' })
|
||||
await expect.poll(() => keyBadges.count()).toBeGreaterThanOrEqual(1)
|
||||
const count = await keyBadges.count()
|
||||
expect(count).toBeGreaterThanOrEqual(1)
|
||||
|
||||
await expect
|
||||
.poll(() => keyBadges.allTextContents())
|
||||
.toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.stringMatching(/^(Ctrl|Cmd|Shift|Alt)$/)
|
||||
])
|
||||
)
|
||||
const badgeText = await keyBadges.allTextContents()
|
||||
const hasModifiers = badgeText.some((text) =>
|
||||
['Ctrl', 'Cmd', 'Shift', 'Alt'].includes(text)
|
||||
)
|
||||
expect(hasModifiers).toBeTruthy()
|
||||
})
|
||||
|
||||
test('should maintain panel state when switching between panels', async ({
|
||||
@@ -197,7 +196,8 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
|
||||
).toBeVisible()
|
||||
|
||||
const subcategoryTitles = bottomPanel.shortcuts.subcategoryTitles
|
||||
await expect.poll(() => subcategoryTitles.count()).toBeGreaterThanOrEqual(2)
|
||||
const titleCount = await subcategoryTitles.count()
|
||||
expect(titleCount).toBeGreaterThanOrEqual(2)
|
||||
})
|
||||
|
||||
test('should open shortcuts panel with Ctrl+Shift+K', async ({
|
||||
|
||||
@@ -21,7 +21,7 @@ async function saveCloseAndReopenAsApp(
|
||||
await appMode.steps.goToPreview()
|
||||
await builderSaveAs(appMode, workflowName)
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await expect(appMode.saveAs.successDialog).not.toBeVisible()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await appMode.footer.exitBuilder()
|
||||
await openWorkflowFromSidebar(comfyPage, workflowName)
|
||||
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
comfyExpect as expect
|
||||
} from '@e2e/fixtures/ComfyPage'
|
||||
import type { AppModeHelper } from '@e2e/fixtures/helpers/AppModeHelper'
|
||||
import type { BuilderSaveAsHelper } from '@e2e/fixtures/helpers/BuilderSaveAsHelper'
|
||||
import {
|
||||
builderSaveAs,
|
||||
openWorkflowFromSidebar,
|
||||
@@ -25,15 +24,6 @@ async function reSaveAs(
|
||||
await appMode.saveAs.fillAndSave(workflowName, viewType)
|
||||
}
|
||||
|
||||
async function dismissSuccessDialog(
|
||||
saveAs: BuilderSaveAsHelper,
|
||||
button: 'close' | 'dismiss' = 'close'
|
||||
) {
|
||||
const btn = button === 'close' ? saveAs.closeButton : saveAs.dismissButton
|
||||
await btn.click()
|
||||
await expect(saveAs.successDialog).not.toBeVisible()
|
||||
}
|
||||
|
||||
test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.appMode.enableLinearMode()
|
||||
@@ -131,7 +121,8 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
|
||||
await builderSaveAs(comfyPage.appMode, `${Date.now()} direct-save`, 'App')
|
||||
await dismissSuccessDialog(saveAs)
|
||||
await saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Modify the workflow so the save button becomes enabled
|
||||
await comfyPage.appMode.steps.goToInputs()
|
||||
@@ -152,7 +143,8 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
|
||||
await builderSaveAs(comfyPage.appMode, `${Date.now()} split-btn`, 'App')
|
||||
await dismissSuccessDialog(saveAs)
|
||||
await saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await footer.openSaveAsFromChevron()
|
||||
|
||||
@@ -169,11 +161,8 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await appMode.enterBuilder()
|
||||
|
||||
// State 1: Disabled "Save as" (no outputs selected)
|
||||
await expect(appMode.footer.saveAsButton).toBeVisible()
|
||||
const disabledBox = await appMode.footer.saveAsButton.boundingBox()
|
||||
if (!disabledBox)
|
||||
throw new Error('saveAsButton boundingBox returned null while visible')
|
||||
const disabledWidth = disabledBox.width
|
||||
expect(disabledBox).toBeTruthy()
|
||||
|
||||
// Select I/O to enable the button
|
||||
await appMode.steps.goToInputs()
|
||||
@@ -182,20 +171,19 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await appMode.select.selectOutputNode('Save Image')
|
||||
|
||||
// State 2: Enabled "Save as" (unsaved, has outputs)
|
||||
await expect
|
||||
.poll(
|
||||
async () => (await appMode.footer.saveAsButton.boundingBox())?.width
|
||||
)
|
||||
.toBe(disabledWidth)
|
||||
const enabledBox = await appMode.footer.saveAsButton.boundingBox()
|
||||
expect(enabledBox).toBeTruthy()
|
||||
expect(enabledBox!.width).toBe(disabledBox!.width)
|
||||
|
||||
// Save the workflow to transition to the Save + chevron state
|
||||
await builderSaveAs(appMode, `${Date.now()} width-test`, 'App')
|
||||
await dismissSuccessDialog(appMode.saveAs)
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// State 3: Save + chevron button group (saved workflow)
|
||||
await expect
|
||||
.poll(async () => (await appMode.footer.saveGroup.boundingBox())?.width)
|
||||
.toBe(disabledWidth)
|
||||
const saveButtonGroupBox = await appMode.footer.saveGroup.boundingBox()
|
||||
expect(saveButtonGroupBox).toBeTruthy()
|
||||
expect(saveButtonGroupBox!.width).toBe(disabledBox!.width)
|
||||
})
|
||||
|
||||
test('Connect output popover appears when no outputs selected', async ({
|
||||
@@ -218,13 +206,11 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
await builderSaveAs(comfyPage.appMode, `${Date.now()} app-ext`, 'App')
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toContain('.app.json')
|
||||
const path = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(path).toContain('.app.json')
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getLinearModeFromGraph())
|
||||
.toBe(true)
|
||||
const linearMode = await comfyPage.workflow.getLinearModeFromGraph()
|
||||
expect(linearMode).toBe(true)
|
||||
})
|
||||
|
||||
test('save as node graph produces correct extension and linearMode', async ({
|
||||
@@ -237,15 +223,12 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
'Node graph'
|
||||
)
|
||||
|
||||
await expect(async () => {
|
||||
const path = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(path).toMatch(/\.json$/)
|
||||
expect(path).not.toContain('.app.json')
|
||||
}).toPass({ timeout: 5000 })
|
||||
const path = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(path).toMatch(/\.json$/)
|
||||
expect(path).not.toContain('.app.json')
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getLinearModeFromGraph())
|
||||
.toBe(false)
|
||||
const linearMode = await comfyPage.workflow.getLinearModeFromGraph()
|
||||
expect(linearMode).toBe(false)
|
||||
})
|
||||
|
||||
test('save as app View App button enters app mode', async ({ comfyPage }) => {
|
||||
@@ -253,11 +236,11 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await builderSaveAs(comfyPage.appMode, `${Date.now()} app-view`, 'App')
|
||||
|
||||
await comfyPage.appMode.saveAs.viewAppButton.click()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowActiveAppMode())
|
||||
.toBe('app')
|
||||
expect(await comfyPage.workflow.getActiveWorkflowActiveAppMode()).toBe(
|
||||
'app'
|
||||
)
|
||||
})
|
||||
|
||||
test('save as node graph Exit builder exits builder mode', async ({
|
||||
@@ -271,7 +254,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
)
|
||||
|
||||
await comfyPage.appMode.saveAs.exitBuilderButton.click()
|
||||
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
|
||||
})
|
||||
@@ -284,27 +267,27 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
|
||||
const originalName = `${Date.now()} original`
|
||||
await builderSaveAs(appMode, originalName, 'App')
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toContain('.app.json')
|
||||
await dismissSuccessDialog(appMode.saveAs)
|
||||
const originalPath = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(originalPath).toContain('.app.json')
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Re-save as node graph — creates a copy
|
||||
await reSaveAs(appMode, `${Date.now()} copy`, 'Node graph')
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.not.toContain('.app.json')
|
||||
const newPath = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(newPath).not.toBe(originalPath)
|
||||
expect(newPath).not.toContain('.app.json')
|
||||
|
||||
// Dismiss success dialog, exit app mode, reopen the original
|
||||
await dismissSuccessDialog(appMode.saveAs, 'dismiss')
|
||||
await appMode.saveAs.dismissButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
await appMode.toggleAppMode()
|
||||
await openWorkflowFromSidebar(comfyPage, originalName)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getLinearModeFromGraph())
|
||||
.toBe(true)
|
||||
const linearMode = await comfyPage.workflow.getLinearModeFromGraph()
|
||||
expect(linearMode).toBe(true)
|
||||
})
|
||||
|
||||
test('save as with same name and same mode overwrites in place', async ({
|
||||
@@ -315,25 +298,20 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
|
||||
await builderSaveAs(appMode, name, 'App')
|
||||
await dismissSuccessDialog(appMode.saveAs)
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toContain('.app.json')
|
||||
const pathAfterFirst = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
|
||||
await reSaveAs(appMode, name, 'App')
|
||||
|
||||
await expect(appMode.saveAs.overwriteDialog).toBeVisible({ timeout: 5000 })
|
||||
await appMode.saveAs.overwriteButton.click()
|
||||
await expect(appMode.saveAs.overwriteDialog).not.toBeVisible()
|
||||
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toBe(pathAfterFirst)
|
||||
const pathAfterSecond = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(pathAfterSecond).toBe(pathAfterFirst)
|
||||
})
|
||||
|
||||
test('save as with same name but different mode creates a new file', async ({
|
||||
@@ -344,38 +322,32 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
await setupBuilder(comfyPage)
|
||||
|
||||
await builderSaveAs(appMode, name, 'App')
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toContain('.app.json')
|
||||
const pathAfterFirst = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
await dismissSuccessDialog(appMode.saveAs)
|
||||
expect(pathAfterFirst).toContain('.app.json')
|
||||
await appMode.saveAs.closeButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await reSaveAs(appMode, name, 'Node graph')
|
||||
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.not.toBe(pathAfterFirst)
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.toMatch(/\.json$/)
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
|
||||
.not.toContain('.app.json')
|
||||
const pathAfterSecond = await comfyPage.workflow.getActiveWorkflowPath()
|
||||
expect(pathAfterSecond).not.toBe(pathAfterFirst)
|
||||
expect(pathAfterSecond).toMatch(/\.json$/)
|
||||
expect(pathAfterSecond).not.toContain('.app.json')
|
||||
})
|
||||
|
||||
test('save as app workflow reloads in app mode', async ({ comfyPage }) => {
|
||||
const name = `${Date.now()} reload-app`
|
||||
await setupBuilder(comfyPage)
|
||||
await builderSaveAs(comfyPage.appMode, name, 'App')
|
||||
await dismissSuccessDialog(comfyPage.appMode.saveAs, 'dismiss')
|
||||
await comfyPage.appMode.saveAs.dismissButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.appMode.footer.exitBuilder()
|
||||
|
||||
await openWorkflowFromSidebar(comfyPage, name)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowInitialMode())
|
||||
.toBe('app')
|
||||
const mode = await comfyPage.workflow.getActiveWorkflowInitialMode()
|
||||
expect(mode).toBe('app')
|
||||
})
|
||||
|
||||
test('save as node graph workflow reloads in node graph mode', async ({
|
||||
@@ -384,13 +356,13 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
|
||||
const name = `${Date.now()} reload-graph`
|
||||
await setupBuilder(comfyPage)
|
||||
await builderSaveAs(comfyPage.appMode, name, 'Node graph')
|
||||
await dismissSuccessDialog(comfyPage.appMode.saveAs, 'dismiss')
|
||||
await comfyPage.appMode.saveAs.dismissButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.appMode.toggleAppMode()
|
||||
|
||||
await openWorkflowFromSidebar(comfyPage, name)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.getActiveWorkflowInitialMode())
|
||||
.toBe('graph')
|
||||
const mode = await comfyPage.workflow.getActiveWorkflowInitialMode()
|
||||
expect(mode).toBe('graph')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,275 +0,0 @@
|
||||
import type { Page } from '@playwright/test'
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
const getLocators = (page: Page) => ({
|
||||
trigger: page.getByRole('button', { name: 'Canvas Mode' }),
|
||||
menu: page.getByRole('menu', { name: 'Canvas Mode' }),
|
||||
selectItem: page.getByRole('menuitemradio', { name: 'Select' }),
|
||||
handItem: page.getByRole('menuitemradio', { name: 'Hand' })
|
||||
})
|
||||
|
||||
const MODES = [
|
||||
{
|
||||
label: 'Select',
|
||||
activateCommand: 'Comfy.Canvas.Unlock',
|
||||
isReadOnly: false,
|
||||
iconPattern: /lucide--mouse-pointer-2/
|
||||
},
|
||||
{
|
||||
label: 'Hand',
|
||||
activateCommand: 'Comfy.Canvas.Lock',
|
||||
isReadOnly: true,
|
||||
iconPattern: /lucide--hand/
|
||||
}
|
||||
]
|
||||
|
||||
test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.CanvasMenu', true)
|
||||
await comfyPage.command.executeCommand('Comfy.Canvas.Unlock')
|
||||
await comfyPage.nextFrame()
|
||||
})
|
||||
|
||||
test.describe('Trigger button', () => {
|
||||
test('visible in canvas toolbar with ARIA markup', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { trigger } = getLocators(comfyPage.page)
|
||||
await expect(trigger).toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
|
||||
})
|
||||
|
||||
for (const mode of MODES) {
|
||||
test(`shows ${mode.label}-mode icon on trigger button`, async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.command.executeCommand(mode.activateCommand)
|
||||
await comfyPage.nextFrame()
|
||||
const { trigger } = getLocators(comfyPage.page)
|
||||
const modeIcon = trigger.locator('i[aria-hidden="true"]').first()
|
||||
await expect(modeIcon).toHaveClass(mode.iconPattern)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
test.describe('Popover lifecycle', () => {
|
||||
test('opens when trigger is clicked', async ({ comfyPage }) => {
|
||||
const { trigger, menu } = getLocators(comfyPage.page)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'true')
|
||||
})
|
||||
|
||||
test('closes when trigger is clicked again', async ({ comfyPage }) => {
|
||||
const { trigger, menu } = getLocators(comfyPage.page)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
|
||||
})
|
||||
|
||||
test('closes after a mode item is selected', async ({ comfyPage }) => {
|
||||
const { trigger, menu, handItem } = getLocators(comfyPage.page)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await handItem.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('closes when Escape is pressed', async ({ comfyPage }) => {
|
||||
const { trigger, menu, selectItem } = getLocators(comfyPage.page)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await selectItem.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Mode switching', () => {
|
||||
for (const mode of MODES) {
|
||||
test(`clicking "${mode.label}" sets canvas readOnly=${mode.isReadOnly}`, async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
if (!mode.isReadOnly) {
|
||||
await comfyPage.command.executeCommand('Comfy.Canvas.Lock')
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
const item = mode.isReadOnly ? handItem : selectItem
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await item.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() => comfyPage.canvasOps.isReadOnly())
|
||||
.toBe(mode.isReadOnly)
|
||||
})
|
||||
}
|
||||
|
||||
test('clicking the currently active item is a no-op', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
expect(
|
||||
await comfyPage.canvasOps.isReadOnly(),
|
||||
'Precondition: canvas starts in Select mode'
|
||||
).toBe(false)
|
||||
const { trigger, menu, selectItem } = getLocators(comfyPage.page)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await selectItem.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('ARIA state', () => {
|
||||
test('aria-checked marks Select active on default load', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(selectItem).toHaveAttribute('aria-checked', 'true')
|
||||
await expect(handItem).toHaveAttribute('aria-checked', 'false')
|
||||
})
|
||||
|
||||
for (const mode of MODES) {
|
||||
test(`tabindex=0 is on the active "${mode.label}" item`, async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.command.executeCommand(mode.activateCommand)
|
||||
await comfyPage.nextFrame()
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
|
||||
const activeItem = mode.isReadOnly ? handItem : selectItem
|
||||
const inactiveItem = mode.isReadOnly ? selectItem : handItem
|
||||
|
||||
await expect(activeItem).toHaveAttribute('tabindex', '0')
|
||||
await expect(inactiveItem).toHaveAttribute('tabindex', '-1')
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
test.describe('Keyboard navigation', () => {
|
||||
test('ArrowDown moves focus from Select to Hand', async ({ comfyPage }) => {
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await selectItem.press('ArrowDown')
|
||||
await expect(handItem).toBeFocused()
|
||||
})
|
||||
|
||||
test('Escape closes popover and restores focus to trigger', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await selectItem.press('ArrowDown')
|
||||
await handItem.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).not.toBeVisible()
|
||||
await expect(trigger).toBeFocused()
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Focus management on open', () => {
|
||||
for (const mode of MODES) {
|
||||
test(`auto-focuses the checked "${mode.label}" item on open`, async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.command.executeCommand(mode.activateCommand)
|
||||
await comfyPage.nextFrame()
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
const item = mode.isReadOnly ? handItem : selectItem
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(item).toBeFocused()
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
test.describe('Keybinding integration', { tag: '@keyboard' }, () => {
|
||||
test("'H' locks canvas and updates trigger icon to Hand", async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
expect(
|
||||
await comfyPage.canvasOps.isReadOnly(),
|
||||
'Precondition: canvas starts unlocked'
|
||||
).toBe(false)
|
||||
await comfyPage.canvas.press('KeyH')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
const { trigger } = getLocators(comfyPage.page)
|
||||
const modeIcon = trigger.locator('i[aria-hidden="true"]').first()
|
||||
await expect(modeIcon).toHaveClass(/lucide--hand/)
|
||||
})
|
||||
|
||||
test("'V' unlocks canvas and updates trigger icon to Select", async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.command.executeCommand('Comfy.Canvas.Lock')
|
||||
await comfyPage.nextFrame()
|
||||
expect(
|
||||
await comfyPage.canvasOps.isReadOnly(),
|
||||
'Precondition: canvas starts locked'
|
||||
).toBe(true)
|
||||
await comfyPage.canvas.press('KeyV')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
const { trigger } = getLocators(comfyPage.page)
|
||||
const modeIcon = trigger.locator('i[aria-hidden="true"]').first()
|
||||
await expect(modeIcon).toHaveClass(/lucide--mouse-pointer-2/)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Shortcut hint display', () => {
|
||||
test('menu items show non-empty keyboard shortcut text', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { trigger, menu, selectItem, handItem } = getLocators(
|
||||
comfyPage.page
|
||||
)
|
||||
await trigger.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(menu).toBeVisible()
|
||||
const selectHint = selectItem.getByTestId('shortcut-hint')
|
||||
const handHint = handItem.getByTestId('shortcut-hint')
|
||||
|
||||
await expect(selectHint).not.toBeEmpty()
|
||||
await expect(handHint).not.toBeEmpty()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,94 +1,14 @@
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import type { WorkspaceStore } from '@e2e/types/globals'
|
||||
import {
|
||||
comfyExpect as expect,
|
||||
comfyPageFixture as test
|
||||
} from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
type ChangeTrackerDebugState = {
|
||||
changeCount: number
|
||||
graphMatchesActiveState: boolean
|
||||
isLoadingGraph: boolean
|
||||
isModified: boolean | undefined
|
||||
redoQueueSize: number
|
||||
restoringState: boolean
|
||||
undoQueueSize: number
|
||||
}
|
||||
|
||||
async function getChangeTrackerDebugState(comfyPage: ComfyPage) {
|
||||
return await comfyPage.page.evaluate(() => {
|
||||
type ChangeTrackerClassLike = {
|
||||
graphEqual: (left: unknown, right: unknown) => boolean
|
||||
isLoadingGraph: boolean
|
||||
}
|
||||
|
||||
type ChangeTrackerLike = {
|
||||
_restoringState: boolean
|
||||
activeState: unknown
|
||||
changeCount: number
|
||||
constructor: ChangeTrackerClassLike
|
||||
redoQueue: unknown[]
|
||||
undoQueue: unknown[]
|
||||
}
|
||||
|
||||
type ActiveWorkflowLike = {
|
||||
changeTracker?: ChangeTrackerLike
|
||||
isModified?: boolean
|
||||
}
|
||||
|
||||
const workflowStore = window.app!.extensionManager as WorkspaceStore
|
||||
const workflow = workflowStore.workflow
|
||||
.activeWorkflow as ActiveWorkflowLike | null
|
||||
const tracker = workflow?.changeTracker
|
||||
if (!workflow || !tracker) {
|
||||
throw new Error('Active workflow change tracker is not available')
|
||||
}
|
||||
|
||||
const currentState = JSON.parse(
|
||||
JSON.stringify(window.app!.rootGraph.serialize())
|
||||
)
|
||||
return {
|
||||
changeCount: tracker.changeCount,
|
||||
graphMatchesActiveState: tracker.constructor.graphEqual(
|
||||
tracker.activeState,
|
||||
currentState
|
||||
),
|
||||
isLoadingGraph: tracker.constructor.isLoadingGraph,
|
||||
isModified: workflow.isModified,
|
||||
redoQueueSize: tracker.redoQueue.length,
|
||||
restoringState: tracker._restoringState,
|
||||
undoQueueSize: tracker.undoQueue.length
|
||||
} satisfies ChangeTrackerDebugState
|
||||
})
|
||||
}
|
||||
|
||||
async function waitForChangeTrackerSettled(
|
||||
comfyPage: ComfyPage,
|
||||
expected: Pick<
|
||||
ChangeTrackerDebugState,
|
||||
'isModified' | 'redoQueueSize' | 'undoQueueSize'
|
||||
>
|
||||
) {
|
||||
// Visible node flags can flip before undo finishes loadGraphData() and
|
||||
// updates the tracker. Poll the tracker's own settled state so we do not
|
||||
// start the next transaction while checkState() is still gated.
|
||||
await expect
|
||||
.poll(() => getChangeTrackerDebugState(comfyPage))
|
||||
.toMatchObject({
|
||||
changeCount: 0,
|
||||
graphMatchesActiveState: true,
|
||||
isLoadingGraph: false,
|
||||
restoringState: false,
|
||||
...expected
|
||||
})
|
||||
}
|
||||
|
||||
async function beforeChange(comfyPage: ComfyPage) {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.canvas!.emitBeforeChange()
|
||||
})
|
||||
}
|
||||
|
||||
async function afterChange(comfyPage: ComfyPage) {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.canvas!.emitAfterChange()
|
||||
@@ -112,7 +32,7 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
|
||||
|
||||
// Save, confirm no errors & workflow modified flag removed
|
||||
await comfyPage.menu.topbar.saveWorkflow('undo-redo-test')
|
||||
await expect(comfyPage.toast.toastErrors).toHaveCount(0)
|
||||
await expect.poll(() => comfyPage.toast.getToastErrorCount()).toBe(0)
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.isCurrentWorkflowModified())
|
||||
.toBe(false)
|
||||
@@ -139,19 +59,19 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
|
||||
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(node).not.toBeBypassed()
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: true,
|
||||
redoQueueSize: 1,
|
||||
undoQueueSize: 1
|
||||
})
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.isCurrentWorkflowModified())
|
||||
.toBe(true)
|
||||
await expect.poll(() => comfyPage.workflow.getUndoQueueSize()).toBe(1)
|
||||
await expect.poll(() => comfyPage.workflow.getRedoQueueSize()).toBe(1)
|
||||
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(node).not.toBeCollapsed()
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: false,
|
||||
redoQueueSize: 2,
|
||||
undoQueueSize: 0
|
||||
})
|
||||
await expect
|
||||
.poll(() => comfyPage.workflow.isCurrentWorkflowModified())
|
||||
.toBe(false)
|
||||
await expect.poll(() => comfyPage.workflow.getUndoQueueSize()).toBe(0)
|
||||
await expect.poll(() => comfyPage.workflow.getRedoQueueSize()).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -178,11 +98,6 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(node).not.toBeBypassed()
|
||||
await expect(node).not.toBeCollapsed()
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: false,
|
||||
redoQueueSize: 2,
|
||||
undoQueueSize: 0
|
||||
})
|
||||
|
||||
// Prevent clicks registering a double-click
|
||||
await comfyPage.canvasOps.clickEmptySpace()
|
||||
@@ -198,21 +113,11 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
|
||||
|
||||
// End transaction
|
||||
await afterChange(comfyPage)
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: true,
|
||||
redoQueueSize: 0,
|
||||
undoQueueSize: 1
|
||||
})
|
||||
|
||||
// Ensure undo reverts both changes
|
||||
await comfyPage.keyboard.undo()
|
||||
await expect(node).not.toBeBypassed({ timeout: 5000 })
|
||||
await expect(node).not.toBeCollapsed({ timeout: 5000 })
|
||||
await waitForChangeTrackerSettled(comfyPage, {
|
||||
isModified: false,
|
||||
redoQueueSize: 1,
|
||||
undoQueueSize: 0
|
||||
})
|
||||
await expect(node).not.toBeBypassed()
|
||||
await expect(node).not.toBeCollapsed()
|
||||
})
|
||||
|
||||
test('Can nest multiple change transactions without adding undo steps', async ({
|
||||
|
||||
@@ -16,7 +16,7 @@ test.describe(
|
||||
comfyPage
|
||||
}) => {
|
||||
// Tab 0: default workflow (7 nodes)
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
|
||||
// Save tab 0 so it has a unique name for tab switching
|
||||
await comfyPage.menu.topbar.saveWorkflow('workflow-a')
|
||||
@@ -42,21 +42,25 @@ test.describe(
|
||||
|
||||
// Create tab 1: blank workflow (0 nodes)
|
||||
await comfyPage.menu.topbar.triggerTopbarCommand(['New'])
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
|
||||
// Switch back to tab 0 (workflow-a).
|
||||
const tab0 = comfyPage.menu.topbar.getWorkflowTab('workflow-a')
|
||||
await tab0.click()
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
|
||||
// switch to blank tab and back to verify no corruption
|
||||
const tab1 = comfyPage.menu.topbar.getWorkflowTab('Unsaved Workflow')
|
||||
await tab1.click()
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
|
||||
// switch again and verify no corruption
|
||||
await tab0.click()
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(7)
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
@@ -65,25 +65,18 @@ test.describe('Asset-supported node default value', { tag: '@cloud' }, () => {
|
||||
|
||||
// Add a new CheckpointLoaderSimple — should use first cloud asset,
|
||||
// not the server's object_info default.
|
||||
const widgetValue = await comfyPage.page.evaluate(() => {
|
||||
const node = window.LiteGraph!.createNode('CheckpointLoaderSimple')
|
||||
window.app!.graph.add(node!)
|
||||
const widget = node!.widgets?.find(
|
||||
(w: { name: string }) => w.name === 'ckpt_name'
|
||||
)
|
||||
return String(widget?.value ?? '')
|
||||
})
|
||||
|
||||
// Production resolves via getAssetFilename (user_metadata.filename →
|
||||
// metadata.filename → asset.name). Test fixtures have no metadata
|
||||
// filename, so asset.name is the resolved value.
|
||||
const nodeId = await comfyPage.page.evaluate(() => {
|
||||
const node = window.LiteGraph!.createNode('CheckpointLoaderSimple')
|
||||
window.app!.graph.add(node!)
|
||||
return node!.id
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
return await comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.graph.getNodeById(id)
|
||||
const widget = node?.widgets?.find(
|
||||
(w: { name: string }) => w.name === 'ckpt_name'
|
||||
)
|
||||
return String(widget?.value ?? '')
|
||||
}, nodeId)
|
||||
})
|
||||
.toBe(CLOUD_ASSETS[0].name)
|
||||
expect(widgetValue).toBe(CLOUD_ASSETS[0].name)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -35,8 +35,8 @@ test.describe(
|
||||
await node.toggleCollapse()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(async () => await node.boundingBox()).not.toBeNull()
|
||||
const box = await node.boundingBox()
|
||||
expect(box).not.toBeNull()
|
||||
await comfyPage.page.mouse.move(
|
||||
box!.x + box!.width / 2,
|
||||
box!.y + box!.height / 2
|
||||
|
||||
@@ -157,7 +157,6 @@ test.describe('Color Palette', { tag: ['@screenshot', '@settings'] }, () => {
|
||||
|
||||
await comfyPage.workflow.loadWorkflow('nodes/every_node_color')
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'obsidian_dark')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'custom-color-palette-obsidian-dark-all-colors.png'
|
||||
)
|
||||
@@ -178,7 +177,7 @@ test.describe('Color Palette', { tag: ['@screenshot', '@settings'] }, () => {
|
||||
window.app!.extensionManager as WorkspaceStore
|
||||
).colorPalette.addCustomColorPalette(p)
|
||||
}, customColorPalettes.obsidian_dark)
|
||||
await expect(comfyPage.toast.toastErrors).toHaveCount(0)
|
||||
expect(await comfyPage.toast.getToastErrorCount()).toBe(0)
|
||||
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'obsidian_dark')
|
||||
await comfyPage.nextFrame()
|
||||
@@ -212,14 +211,12 @@ test.describe(
|
||||
|
||||
// Drag mouse to force canvas to redraw
|
||||
await comfyPage.page.mouse.move(0, 0)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('node-opacity-0.5.png')
|
||||
|
||||
await comfyPage.settings.setSetting('Comfy.Node.Opacity', 1.0)
|
||||
|
||||
await comfyPage.page.mouse.move(8, 8)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('node-opacity-1.png')
|
||||
})
|
||||
|
||||
@@ -228,8 +225,8 @@ test.describe(
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Node.Opacity', 0.2)
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'arc')
|
||||
await comfyPage.page.mouse.move(0, 0)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.mouse.move(0, 0)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'node-opacity-0.2-arc-theme.png'
|
||||
)
|
||||
@@ -241,38 +238,22 @@ test.describe(
|
||||
await comfyPage.settings.setSetting('Comfy.Node.Opacity', 0.5)
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'light')
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
if (typeof graph.serialize !== 'function') return undefined
|
||||
const parsed = graph.serialize() as {
|
||||
nodes: Array<{ bgcolor?: string; color?: string }>
|
||||
}
|
||||
return parsed.nodes
|
||||
})
|
||||
)
|
||||
.toBeDefined()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const nodes = await comfyPage.page.evaluate(() => {
|
||||
return (
|
||||
window.app!.graph!.serialize() as {
|
||||
nodes: Array<{ bgcolor?: string; color?: string }>
|
||||
}
|
||||
).nodes
|
||||
})
|
||||
if (!Array.isArray(nodes)) return 'not an array'
|
||||
for (const node of nodes) {
|
||||
if (node.bgcolor && /hsla/.test(node.bgcolor))
|
||||
return `bgcolor contains hsla: ${node.bgcolor}`
|
||||
if (node.color && /hsla/.test(node.color))
|
||||
return `color contains hsla: ${node.color}`
|
||||
}
|
||||
return 'ok'
|
||||
})
|
||||
.toBe('ok')
|
||||
const parsed = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
if (typeof graph.serialize !== 'function') {
|
||||
throw new Error('app.graph.serialize is not available')
|
||||
}
|
||||
return graph.serialize() as {
|
||||
nodes: Array<{ bgcolor?: string; color?: string }>
|
||||
}
|
||||
})
|
||||
expect(parsed.nodes).toBeDefined()
|
||||
expect(Array.isArray(parsed.nodes)).toBe(true)
|
||||
const nodes = parsed.nodes
|
||||
for (const node of nodes) {
|
||||
if (node.bgcolor) expect(node.bgcolor).not.toMatch(/hsla/)
|
||||
if (node.color) expect(node.color).not.toMatch(/hsla/)
|
||||
}
|
||||
})
|
||||
|
||||
test('should lighten node colors when switching to light theme', async ({
|
||||
|
||||
@@ -13,9 +13,7 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
})
|
||||
|
||||
await comfyPage.command.executeCommand('TestCommand')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.foo))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.foo)).toBe(true)
|
||||
})
|
||||
|
||||
test('Should execute async command', async ({ comfyPage }) => {
|
||||
@@ -29,9 +27,7 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
})
|
||||
|
||||
await comfyPage.command.executeCommand('TestCommand')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.foo))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.foo)).toBe(true)
|
||||
})
|
||||
|
||||
test('Should handle command errors', async ({ comfyPage }) => {
|
||||
@@ -40,7 +36,7 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
})
|
||||
|
||||
await comfyPage.command.executeCommand('TestCommand')
|
||||
await expect(comfyPage.toast.toastErrors).toHaveCount(1)
|
||||
expect(await comfyPage.toast.getToastErrorCount()).toBe(1)
|
||||
})
|
||||
|
||||
test('Should handle async command errors', async ({ comfyPage }) => {
|
||||
@@ -53,6 +49,6 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
})
|
||||
|
||||
await comfyPage.command.executeCommand('TestCommand')
|
||||
await expect(comfyPage.toast.toastErrors).toHaveCount(1)
|
||||
expect(await comfyPage.toast.getToastErrorCount()).toBe(1)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -4,19 +4,22 @@ import type { Locator } from '@playwright/test'
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
async function verifyCustomIconSvg(iconElement: Locator) {
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const svgVariable = await iconElement.evaluate((element) =>
|
||||
getComputedStyle(element).getPropertyValue('--svg')
|
||||
)
|
||||
if (!svgVariable) return null
|
||||
const dataUrlMatch = svgVariable.match(
|
||||
/url\("data:image\/svg\+xml,([^"]+)"\)/
|
||||
)
|
||||
if (!dataUrlMatch) return null
|
||||
return decodeURIComponent(dataUrlMatch[1])
|
||||
})
|
||||
.toContain("<svg xmlns='http://www.w3.org/2000/svg'")
|
||||
const svgVariable = await iconElement.evaluate((element) => {
|
||||
const styles = getComputedStyle(element)
|
||||
return styles.getPropertyValue('--svg')
|
||||
})
|
||||
|
||||
expect(svgVariable).toBeTruthy()
|
||||
const dataUrlMatch = svgVariable.match(
|
||||
/url\("data:image\/svg\+xml,([^"]+)"\)/
|
||||
)
|
||||
expect(dataUrlMatch).toBeTruthy()
|
||||
|
||||
const encodedSvg = dataUrlMatch![1]
|
||||
const decodedSvg = decodeURIComponent(encodedSvg)
|
||||
|
||||
// Check for SVG header to confirm it's a valid SVG
|
||||
expect(decodedSvg).toContain("<svg xmlns='http://www.w3.org/2000/svg'")
|
||||
}
|
||||
|
||||
test.describe('Custom Icons', { tag: '@settings' }, () => {
|
||||
|
||||
@@ -41,9 +41,11 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(selectedButton).toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
})
|
||||
}
|
||||
@@ -56,9 +58,8 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await comfyPage.canvas.press('Alt+Equal')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.canvasOps.getScale())
|
||||
.toBeGreaterThan(initialScale)
|
||||
const newScale = await comfyPage.canvasOps.getScale()
|
||||
expect(newScale).toBeGreaterThan(initialScale)
|
||||
})
|
||||
|
||||
test("'Alt+-' zooms out", async ({ comfyPage }) => {
|
||||
@@ -67,17 +68,15 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await comfyPage.canvas.press('Alt+Minus')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.canvasOps.getScale())
|
||||
.toBeLessThan(initialScale)
|
||||
const newScale = await comfyPage.canvasOps.getScale()
|
||||
expect(newScale).toBeLessThan(initialScale)
|
||||
})
|
||||
|
||||
test("'.' fits view to nodes", async ({ comfyPage }) => {
|
||||
// Set scale very small so fit-view will zoom back to fit nodes
|
||||
await comfyPage.canvasOps.setScale(0.1)
|
||||
await expect
|
||||
.poll(() => comfyPage.canvasOps.getScale())
|
||||
.toBeCloseTo(0.1, 1)
|
||||
const scaleBefore = await comfyPage.canvasOps.getScale()
|
||||
expect(scaleBefore).toBeCloseTo(0.1, 1)
|
||||
|
||||
// Click canvas to ensure focus is within graph-canvas-container
|
||||
await comfyPage.canvas.click({ position: { x: 400, y: 400 } })
|
||||
@@ -86,30 +85,29 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await comfyPage.canvas.press('Period')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.canvasOps.getScale())
|
||||
.toBeGreaterThan(0.1)
|
||||
const scaleAfter = await comfyPage.canvasOps.getScale()
|
||||
expect(scaleAfter).toBeGreaterThan(scaleBefore)
|
||||
})
|
||||
|
||||
test("'h' locks canvas", async ({ comfyPage }) => {
|
||||
await expect.poll(() => comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
|
||||
await comfyPage.canvas.press('KeyH')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
})
|
||||
|
||||
test("'v' unlocks canvas", async ({ comfyPage }) => {
|
||||
// Lock first
|
||||
await comfyPage.command.executeCommand('Comfy.Canvas.Lock')
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
|
||||
await comfyPage.canvas.press('KeyV')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -124,15 +122,15 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await node.click('title')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => node.isCollapsed()).toBe(false)
|
||||
expect(await node.isCollapsed()).toBe(false)
|
||||
|
||||
await comfyPage.canvas.press('Alt+KeyC')
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => node.isCollapsed()).toBe(true)
|
||||
expect(await node.isCollapsed()).toBe(true)
|
||||
|
||||
await comfyPage.canvas.press('Alt+KeyC')
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => node.isCollapsed()).toBe(false)
|
||||
expect(await node.isCollapsed()).toBe(false)
|
||||
})
|
||||
|
||||
test("'Ctrl+m' mutes and unmutes selected nodes", async ({ comfyPage }) => {
|
||||
@@ -149,16 +147,16 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
return window.app!.canvas.graph!.getNodeById(nodeId)!.mode
|
||||
}, node.id)
|
||||
|
||||
await expect.poll(() => getMode()).toBe(0)
|
||||
expect(await getMode()).toBe(0)
|
||||
|
||||
await comfyPage.canvas.press('Control+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
// NEVER (2) = muted
|
||||
await expect.poll(() => getMode()).toBe(2)
|
||||
expect(await getMode()).toBe(2)
|
||||
|
||||
await comfyPage.canvas.press('Control+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => getMode()).toBe(0)
|
||||
expect(await getMode()).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -172,10 +170,12 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
|
||||
// Toggle off with Alt+m
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.appMode.linearWidgets).not.toBeVisible()
|
||||
|
||||
// Toggle on again
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -189,9 +189,11 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await expect(minimap).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(minimap).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(minimap).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -201,9 +203,11 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.bottomPanel.root).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
@@ -274,9 +278,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
await comfyPage.page.keyboard.press('Control+o')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.TestCommand))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -284,14 +286,8 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
test("'Ctrl+Shift+e' converts selection to subgraph", async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await expect
|
||||
.poll(
|
||||
() => comfyPage.nodeOps.getGraphNodesCount(),
|
||||
'Default workflow should have multiple nodes'
|
||||
)
|
||||
.toBeGreaterThan(1)
|
||||
|
||||
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(initialCount).toBeGreaterThan(1)
|
||||
|
||||
// Select all nodes
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
|
||||
@@ -17,9 +17,10 @@ test.describe('Settings', () => {
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
const contentArea = settingsDialog.locator('main')
|
||||
await expect(contentArea).toBeVisible()
|
||||
await expect
|
||||
.poll(() => contentArea.evaluate((el) => el.clientHeight))
|
||||
.toBeGreaterThan(30)
|
||||
const isUsableHeight = await contentArea.evaluate(
|
||||
(el) => el.clientHeight > 30
|
||||
)
|
||||
expect(isUsableHeight).toBeTruthy()
|
||||
})
|
||||
|
||||
test('Can open settings with hotkey', async ({ comfyPage }) => {
|
||||
@@ -38,27 +39,27 @@ test.describe('Settings', () => {
|
||||
const maxSpeed = 2.5
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.ZoomSpeed', maxSpeed)
|
||||
await test.step('Setting should persist', async () => {
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.Graph.ZoomSpeed'))
|
||||
.toBe(maxSpeed)
|
||||
expect(await comfyPage.settings.getSetting('Comfy.Graph.ZoomSpeed')).toBe(
|
||||
maxSpeed
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
test('Should persist keybinding setting', async ({ comfyPage }) => {
|
||||
// Open the settings dialog
|
||||
await comfyPage.page.keyboard.press('Control+,')
|
||||
await comfyPage.page.waitForSelector('[data-testid="settings-dialog"]')
|
||||
|
||||
// Open the keybinding tab
|
||||
const settingsDialog = comfyPage.page.locator(
|
||||
'[data-testid="settings-dialog"]'
|
||||
)
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
await settingsDialog
|
||||
.locator('nav [role="button"]', { hasText: 'Keybinding' })
|
||||
.click()
|
||||
await expect(
|
||||
comfyPage.page.getByPlaceholder('Search Keybindings...')
|
||||
).toBeVisible()
|
||||
await comfyPage.page.waitForSelector(
|
||||
'[placeholder="Search Keybindings..."]'
|
||||
)
|
||||
|
||||
// Focus the 'New Blank Workflow' row
|
||||
const newBlankWorkflowRow = comfyPage.page.locator('tr', {
|
||||
@@ -155,6 +156,6 @@ test.describe('Signin dialog', () => {
|
||||
await input.press('Control+v')
|
||||
await expect(input).toHaveValue('test_password')
|
||||
|
||||
await expect.poll(() => comfyPage.nodeOps.getNodeCount()).toBe(nodeNum)
|
||||
expect(await comfyPage.nodeOps.getNodeCount()).toBe(nodeNum)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,417 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { AlgoliaNodePack } from '@/types/algoliaTypes'
|
||||
import type { components as ManagerComponents } from '@/workbench/extensions/manager/types/generatedManagerTypes'
|
||||
import type { components as RegistryComponents } from '@comfyorg/registry-types'
|
||||
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
import { mockSystemStats } from '@e2e/fixtures/data/systemStats'
|
||||
|
||||
type InstalledPacksResponse =
|
||||
ManagerComponents['schemas']['InstalledPacksResponse']
|
||||
type RegistryNodePack = RegistryComponents['schemas']['Node']
|
||||
|
||||
interface AlgoliaSearchResult {
|
||||
hits: Partial<AlgoliaNodePack>[]
|
||||
nbHits: number
|
||||
page: number
|
||||
nbPages: number
|
||||
hitsPerPage: number
|
||||
}
|
||||
|
||||
interface AlgoliaSearchResponse {
|
||||
results: AlgoliaSearchResult[]
|
||||
}
|
||||
|
||||
const MOCK_PACK_A: RegistryNodePack = {
|
||||
id: 'test-pack-a',
|
||||
name: 'Test Pack A',
|
||||
description: 'A test custom node pack',
|
||||
downloads: 5000,
|
||||
status: 'NodeStatusActive',
|
||||
publisher: { id: 'test-publisher', name: 'Test Publisher' },
|
||||
latest_version: { version: '1.0.0', status: 'NodeVersionStatusActive' },
|
||||
repository: 'https://github.com/test/pack-a',
|
||||
tags: ['image', 'processing']
|
||||
}
|
||||
|
||||
const MOCK_PACK_B: RegistryNodePack = {
|
||||
id: 'test-pack-b',
|
||||
name: 'Test Pack B',
|
||||
description: 'Another test custom node pack for testing search',
|
||||
downloads: 3000,
|
||||
status: 'NodeStatusActive',
|
||||
publisher: { id: 'another-publisher', name: 'Another Publisher' },
|
||||
latest_version: { version: '2.1.0', status: 'NodeVersionStatusActive' },
|
||||
repository: 'https://github.com/test/pack-b',
|
||||
tags: ['video', 'generation']
|
||||
}
|
||||
|
||||
const MOCK_PACK_C: RegistryNodePack = {
|
||||
id: 'test-pack-c',
|
||||
name: 'Test Pack C',
|
||||
description: 'Third test pack',
|
||||
downloads: 100,
|
||||
status: 'NodeStatusActive',
|
||||
publisher: { id: 'test-publisher', name: 'Test Publisher' },
|
||||
latest_version: { version: '0.5.0', status: 'NodeVersionStatusActive' },
|
||||
repository: 'https://github.com/test/pack-c'
|
||||
}
|
||||
|
||||
const MOCK_INSTALLED_PACKS: InstalledPacksResponse = {
|
||||
'test-pack-a': {
|
||||
ver: '1.0.0',
|
||||
cnr_id: 'test-pack-a',
|
||||
enabled: true
|
||||
},
|
||||
'test-pack-c': {
|
||||
ver: '0.5.0',
|
||||
cnr_id: 'test-pack-c',
|
||||
enabled: false
|
||||
}
|
||||
}
|
||||
|
||||
const MOCK_HIT_A: Partial<AlgoliaNodePack> = {
|
||||
objectID: 'test-pack-a',
|
||||
id: 'test-pack-a',
|
||||
name: 'Test Pack A',
|
||||
description: 'A test custom node pack',
|
||||
total_install: 5000,
|
||||
status: 'NodeStatusActive',
|
||||
publisher_id: 'test-publisher',
|
||||
latest_version: '1.0.0',
|
||||
latest_version_status: 'NodeVersionStatusActive',
|
||||
repository_url: 'https://github.com/test/pack-a',
|
||||
comfy_nodes: ['TestNodeA1', 'TestNodeA2'],
|
||||
create_time: '2024-01-01T00:00:00Z',
|
||||
update_time: '2024-06-01T00:00:00Z',
|
||||
license: 'MIT',
|
||||
tags: ['image', 'processing']
|
||||
}
|
||||
|
||||
const MOCK_HIT_B: Partial<AlgoliaNodePack> = {
|
||||
objectID: 'test-pack-b',
|
||||
id: 'test-pack-b',
|
||||
name: 'Test Pack B',
|
||||
description: 'Another test custom node pack for testing search',
|
||||
total_install: 3000,
|
||||
status: 'NodeStatusActive',
|
||||
publisher_id: 'another-publisher',
|
||||
latest_version: '2.1.0',
|
||||
latest_version_status: 'NodeVersionStatusActive',
|
||||
repository_url: 'https://github.com/test/pack-b',
|
||||
comfy_nodes: ['TestNodeB1'],
|
||||
create_time: '2024-02-01T00:00:00Z',
|
||||
update_time: '2024-07-01T00:00:00Z',
|
||||
license: 'Apache-2.0',
|
||||
tags: ['video', 'generation']
|
||||
}
|
||||
|
||||
const MOCK_HIT_C: Partial<AlgoliaNodePack> = {
|
||||
objectID: 'test-pack-c',
|
||||
id: 'test-pack-c',
|
||||
name: 'Test Pack C',
|
||||
description: 'Third test pack',
|
||||
total_install: 100,
|
||||
status: 'NodeStatusActive',
|
||||
publisher_id: 'test-publisher',
|
||||
latest_version: '0.5.0',
|
||||
latest_version_status: 'NodeVersionStatusActive',
|
||||
repository_url: 'https://github.com/test/pack-c',
|
||||
comfy_nodes: ['TestNodeC1'],
|
||||
create_time: '2024-03-01T00:00:00Z',
|
||||
update_time: '2024-05-01T00:00:00Z',
|
||||
license: 'MIT'
|
||||
}
|
||||
|
||||
const MOCK_ALGOLIA_RESPONSE: AlgoliaSearchResponse = {
|
||||
results: [
|
||||
{
|
||||
hits: [MOCK_HIT_A, MOCK_HIT_B, MOCK_HIT_C],
|
||||
nbHits: 3,
|
||||
page: 0,
|
||||
nbPages: 1,
|
||||
hitsPerPage: 20
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
const MOCK_ALGOLIA_PACK_B_ONLY: AlgoliaSearchResponse = {
|
||||
results: [
|
||||
{
|
||||
hits: [MOCK_HIT_B],
|
||||
nbHits: 1,
|
||||
page: 0,
|
||||
nbPages: 1,
|
||||
hitsPerPage: 20
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
const MOCK_ALGOLIA_EMPTY: AlgoliaSearchResponse = {
|
||||
results: [
|
||||
{
|
||||
hits: [],
|
||||
nbHits: 0,
|
||||
page: 0,
|
||||
nbPages: 0,
|
||||
hitsPerPage: 20
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
test.describe('ManagerDialog', { tag: '@ui' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
const statsWithManager = {
|
||||
...mockSystemStats,
|
||||
system: {
|
||||
...mockSystemStats.system,
|
||||
argv: ['main.py', '--listen', '0.0.0.0', '--enable-manager']
|
||||
}
|
||||
}
|
||||
await comfyPage.page.route('**/system_stats**', async (route) => {
|
||||
await route.fulfill({ json: statsWithManager })
|
||||
})
|
||||
|
||||
await comfyPage.featureFlags.mockServerFeatures({
|
||||
'extension.manager.supports_v4': true
|
||||
})
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/v2/customnode/installed**',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: MOCK_INSTALLED_PACKS })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/v2/manager/queue/status**',
|
||||
async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
history: {},
|
||||
running_queue: [],
|
||||
pending_queue: [],
|
||||
installed_packs: {}
|
||||
}
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/v2/manager/queue/history**',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: {} })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route('**/*.algolia.net/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_RESPONSE })
|
||||
})
|
||||
|
||||
await comfyPage.page.route('**/*.algolianet.com/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_RESPONSE })
|
||||
})
|
||||
|
||||
// Mock Comfy Registry API (fallback when Algolia credentials are unavailable)
|
||||
const registryListResponse = {
|
||||
total: 3,
|
||||
nodes: [MOCK_PACK_A, MOCK_PACK_B, MOCK_PACK_C],
|
||||
page: 1,
|
||||
limit: 64,
|
||||
totalPages: 1
|
||||
}
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/api.comfy.org/nodes/search**',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: registryListResponse })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route(
|
||||
(url) => url.hostname === 'api.comfy.org' && url.pathname === '/nodes',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: registryListResponse })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/v2/customnode/getmappings**',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: {} })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.page.route(
|
||||
'**/v2/customnode/import_fail_info**',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: {} })
|
||||
}
|
||||
)
|
||||
|
||||
await comfyPage.setup()
|
||||
})
|
||||
|
||||
async function openManagerDialog(comfyPage: ComfyPage) {
|
||||
await comfyPage.command.executeCommand('Comfy.OpenManagerDialog')
|
||||
}
|
||||
|
||||
test('Opens the manager dialog via command', async ({ comfyPage }) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
})
|
||||
|
||||
test('Displays pack cards from search results', async ({ comfyPage }) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
await expect(dialog.getByText('Test Pack A')).toBeVisible()
|
||||
await expect(dialog.getByText('Test Pack B')).toBeVisible()
|
||||
await expect(dialog.getByText('Test Pack C')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Search filters displayed packs', async ({ comfyPage }) => {
|
||||
await comfyPage.page.route('**/*.algolia.net/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_PACK_B_ONLY })
|
||||
})
|
||||
await comfyPage.page.route('**/*.algolianet.com/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_PACK_B_ONLY })
|
||||
})
|
||||
await comfyPage.page.route(
|
||||
'**/api.comfy.org/nodes/search**',
|
||||
async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
total: 1,
|
||||
nodes: [MOCK_PACK_B],
|
||||
page: 1,
|
||||
limit: 64,
|
||||
totalPages: 1
|
||||
}
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
const searchInput = dialog.getByPlaceholder(/search/i)
|
||||
await searchInput.fill('Test Pack B')
|
||||
|
||||
await expect(dialog.getByText('Test Pack B')).toBeVisible()
|
||||
await expect(dialog.getByText('Test Pack A')).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Clicking a pack card opens the info panel', async ({ comfyPage }) => {
|
||||
await comfyPage.page.route(
|
||||
'**/api.comfy.org/nodes/test-pack-a',
|
||||
async (route) => {
|
||||
await route.fulfill({ json: MOCK_PACK_A })
|
||||
}
|
||||
)
|
||||
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
await dialog.getByText('Test Pack A').first().click()
|
||||
|
||||
await expect(dialog.getByText('Test Publisher').first()).toBeVisible()
|
||||
})
|
||||
|
||||
test('Left side panel navigation tabs exist', async ({ comfyPage }) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
const nav = dialog.locator('nav')
|
||||
await expect(nav.getByText('All Extensions')).toBeVisible()
|
||||
await expect(nav.getByText('Not Installed')).toBeVisible()
|
||||
await expect(nav.getByText('All Installed')).toBeVisible()
|
||||
await expect(nav.getByText('Updates Available')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Switching tabs changes the content view', async ({ comfyPage }) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
const nav = dialog.locator('nav')
|
||||
await nav.getByText('All Installed').click()
|
||||
|
||||
await expect(dialog.getByText('Test Pack A')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Closes via Escape key', async ({ comfyPage }) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await expect(dialog).not.toBeVisible()
|
||||
})
|
||||
|
||||
test('Empty search shows no results message', async ({ comfyPage }) => {
|
||||
await comfyPage.page.route('**/*.algolia.net/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_EMPTY })
|
||||
})
|
||||
await comfyPage.page.route('**/*.algolianet.com/**', async (route) => {
|
||||
await route.fulfill({ json: MOCK_ALGOLIA_EMPTY })
|
||||
})
|
||||
await comfyPage.page.route(
|
||||
'**/api.comfy.org/nodes/search**',
|
||||
async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
total: 0,
|
||||
nodes: [],
|
||||
page: 1,
|
||||
limit: 64,
|
||||
totalPages: 0
|
||||
}
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
const searchInput = dialog.getByPlaceholder(/search/i)
|
||||
await searchInput.fill('nonexistent-pack-xyz-999')
|
||||
|
||||
await expect(
|
||||
dialog.getByText(/no results found|try a different search/i).first()
|
||||
).toBeVisible()
|
||||
})
|
||||
|
||||
test('Search mode can be switched between packs and nodes', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await openManagerDialog(comfyPage)
|
||||
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
await expect(dialog).toBeVisible()
|
||||
|
||||
const modeSelector = dialog.getByText('Node Pack').first()
|
||||
await expect(modeSelector).toBeVisible()
|
||||
|
||||
await modeSelector.click()
|
||||
const nodesOption = comfyPage.page.getByRole('option', { name: 'Nodes' })
|
||||
await expect(nodesOption).toBeVisible()
|
||||
await nodesOption.click()
|
||||
})
|
||||
})
|
||||
@@ -1,7 +1,7 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
import { mockSystemStats } from '@e2e/fixtures/data/systemStats'
|
||||
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
|
||||
import { mockSystemStats } from '../../fixtures/data/systemStats'
|
||||
|
||||
const MOCK_COMFYUI_VERSION = '9.99.0-e2e-test'
|
||||
|
||||
@@ -145,20 +145,15 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
|
||||
const settingRow = dialog.root.locator(`[data-setting-id="${settingId}"]`)
|
||||
await expect(settingRow).toBeVisible()
|
||||
|
||||
// Open the dropdown via its combobox role and verify it expanded.
|
||||
// Retry because the PrimeVue Select may re-render during search
|
||||
// filtering, causing the first click to land on a stale element.
|
||||
const select = settingRow.getByRole('combobox')
|
||||
await expect(async () => {
|
||||
const expanded = await select.getAttribute('aria-expanded')
|
||||
if (expanded !== 'true') await select.click()
|
||||
await expect(select).toHaveAttribute('aria-expanded', 'true')
|
||||
}).toPass({ timeout: 3000 })
|
||||
// Click the PrimeVue Select to open the dropdown
|
||||
await settingRow.locator('.p-select').click()
|
||||
const overlay = comfyPage.page.locator('.p-select-overlay')
|
||||
await expect(overlay).toBeVisible()
|
||||
|
||||
// Pick the option that is not the current value
|
||||
const targetValue = initialValue === 'Top' ? 'Disabled' : 'Top'
|
||||
await comfyPage.page
|
||||
.getByRole('option', { name: targetValue, exact: true })
|
||||
await overlay
|
||||
.locator(`.p-select-option-label:text-is("${targetValue}")`)
|
||||
.click()
|
||||
|
||||
await expect
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import type { Page } from '@playwright/test'
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { AssetInfo } from '@/schemas/apiSchema'
|
||||
import { comfyPageFixture } from '@e2e/fixtures/ComfyPage'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
import type { AssetInfo } from '../../../src/schemas/apiSchema'
|
||||
import { comfyPageFixture } from '../../fixtures/ComfyPage'
|
||||
import { TestIds } from '../../fixtures/selectors'
|
||||
|
||||
interface PublishRecord {
|
||||
workflow_id: string
|
||||
|
||||
@@ -35,14 +35,14 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('focus-mode-on.png')
|
||||
}
|
||||
)
|
||||
|
||||
// No DOM widget should be created by creation of interim LGraphNode objects.
|
||||
test('Copy node with DOM widget by dragging + alt', async ({ comfyPage }) => {
|
||||
const initialCount = await comfyPage.domWidgets.count()
|
||||
const initialCount = await comfyPage.getDOMWidgetCount()
|
||||
|
||||
// TextEncodeNode1
|
||||
await comfyPage.page.mouse.move(618, 191)
|
||||
@@ -52,6 +52,7 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
|
||||
await comfyPage.page.mouse.up()
|
||||
await comfyPage.page.keyboard.up('Alt')
|
||||
|
||||
await expect(comfyPage.domWidgets).toHaveCount(initialCount + 1)
|
||||
const finalCount = await comfyPage.getDOMWidgetCount()
|
||||
expect(finalCount).toBe(initialCount + 1)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -100,9 +100,8 @@ test.describe('Error dialog', () => {
|
||||
await errorDialog.getByTestId(TestIds.dialogs.errorDialogCopyReport).click()
|
||||
|
||||
const reportText = await errorDialog.locator('pre').textContent()
|
||||
await expect
|
||||
.poll(async () => await getClipboardText(comfyPage.page))
|
||||
.toBe(reportText)
|
||||
const copiedText = await getClipboardText(comfyPage.page)
|
||||
expect(copiedText).toBe(reportText)
|
||||
})
|
||||
|
||||
test('Should open GitHub issues search when "Find Issues" is clicked', async ({
|
||||
|
||||
@@ -47,16 +47,11 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
test('Should display "Show missing models" button for missing model errors', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(async (url: string) => {
|
||||
const response = await fetch(
|
||||
`${url}/api/devtools/cleanup_fake_model`
|
||||
)
|
||||
return response.ok
|
||||
}, comfyPage.url)
|
||||
)
|
||||
.toBeTruthy()
|
||||
const cleanupOk = await comfyPage.page.evaluate(async (url: string) => {
|
||||
const response = await fetch(`${url}/api/devtools/cleanup_fake_model`)
|
||||
return response.ok
|
||||
}, comfyPage.url)
|
||||
expect(cleanupOk).toBeTruthy()
|
||||
|
||||
await comfyPage.workflow.loadWorkflow('missing/missing_models')
|
||||
|
||||
@@ -156,7 +151,6 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
|
||||
|
||||
await overlay.getByTestId(TestIds.dialogs.errorOverlaySeeErrors).click()
|
||||
|
||||
await expect(overlay).not.toBeVisible()
|
||||
await expect(comfyPage.page.getByTestId('properties-panel')).toBeVisible()
|
||||
})
|
||||
|
||||
|
||||
@@ -49,34 +49,18 @@ test.describe(
|
||||
const input = await comfyPage.nodeOps.getNodeRefById(3)
|
||||
const output1 = await comfyPage.nodeOps.getNodeRefById(1)
|
||||
const output2 = await comfyPage.nodeOps.getNodeRefById(4)
|
||||
await expect
|
||||
.poll(async () => (await input.getWidget(0)).getValue())
|
||||
.toBe('foo')
|
||||
await expect
|
||||
.poll(async () => (await output1.getWidget(0)).getValue())
|
||||
.toBe('')
|
||||
await expect
|
||||
.poll(async () => (await output2.getWidget(0)).getValue())
|
||||
.toBe('')
|
||||
expect(await (await input.getWidget(0)).getValue()).toBe('foo')
|
||||
expect(await (await output1.getWidget(0)).getValue()).toBe('')
|
||||
expect(await (await output2.getWidget(0)).getValue()).toBe('')
|
||||
|
||||
await output1.click('title')
|
||||
|
||||
await comfyPage.command.executeCommand('Comfy.QueueSelectedOutputNodes')
|
||||
await expect
|
||||
.poll(async () => (await input.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('foo')
|
||||
await expect
|
||||
.poll(async () => (await output1.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('foo')
|
||||
await expect
|
||||
.poll(async () => (await output2.getWidget(0)).getValue(), {
|
||||
timeout: 2_000
|
||||
})
|
||||
.toBe('')
|
||||
await expect(async () => {
|
||||
expect(await (await input.getWidget(0)).getValue()).toBe('foo')
|
||||
expect(await (await output1.getWidget(0)).getValue()).toBe('foo')
|
||||
expect(await (await output2.getWidget(0)).getValue()).toBe('')
|
||||
}).toPass({ timeout: 2_000 })
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
@@ -40,9 +40,7 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
await comfyPage.menu.topbar.triggerTopbarCommand(['ext', 'foo-command'])
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.foo))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.foo)).toBe(true)
|
||||
})
|
||||
|
||||
test('Should not allow register command defined in other extension', async ({
|
||||
@@ -62,7 +60,7 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
const menuItem = comfyPage.menu.topbar.getMenuItem('ext')
|
||||
await expect(menuItem).toHaveCount(0)
|
||||
expect(await menuItem.count()).toBe(0)
|
||||
})
|
||||
|
||||
test('Should allow registering keybindings', async ({ comfyPage }) => {
|
||||
@@ -88,9 +86,7 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
await comfyPage.page.keyboard.press('k')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.TestCommand))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(true)
|
||||
})
|
||||
|
||||
test.describe('Settings', () => {
|
||||
@@ -113,20 +109,16 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
})
|
||||
// onChange is called when the setting is first added
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.changeCount))
|
||||
.toBe(1)
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('TestSetting'))
|
||||
.toBe('Hello, world!')
|
||||
expect(await comfyPage.page.evaluate(() => window.changeCount)).toBe(1)
|
||||
expect(await comfyPage.settings.getSetting('TestSetting')).toBe(
|
||||
'Hello, world!'
|
||||
)
|
||||
|
||||
await comfyPage.settings.setSetting('TestSetting', 'Hello, universe!')
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('TestSetting'))
|
||||
.toBe('Hello, universe!')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.changeCount))
|
||||
.toBe(2)
|
||||
expect(await comfyPage.settings.getSetting('TestSetting')).toBe(
|
||||
'Hello, universe!'
|
||||
)
|
||||
expect(await comfyPage.page.evaluate(() => window.changeCount)).toBe(2)
|
||||
})
|
||||
|
||||
test('Should allow setting boolean settings', async ({ comfyPage }) => {
|
||||
@@ -148,21 +140,17 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.TestSetting'))
|
||||
.toBe(false)
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.changeCount))
|
||||
.toBe(1)
|
||||
expect(await comfyPage.settings.getSetting('Comfy.TestSetting')).toBe(
|
||||
false
|
||||
)
|
||||
expect(await comfyPage.page.evaluate(() => window.changeCount)).toBe(1)
|
||||
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.toggleBooleanSetting('Comfy.TestSetting')
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.TestSetting'))
|
||||
.toBe(true)
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.changeCount))
|
||||
.toBe(2)
|
||||
expect(await comfyPage.settings.getSetting('Comfy.TestSetting')).toBe(
|
||||
true
|
||||
)
|
||||
expect(await comfyPage.page.evaluate(() => window.changeCount)).toBe(2)
|
||||
})
|
||||
|
||||
test.describe('Passing through attrs to setting components', () => {
|
||||
@@ -240,15 +228,12 @@ test.describe('Topbar commands', () => {
|
||||
.getByText('TestSetting Test')
|
||||
.locator(selector)
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
component.evaluate((el) =>
|
||||
el.tagName === 'INPUT'
|
||||
? (el as HTMLInputElement).disabled
|
||||
: el.classList.contains('p-disabled')
|
||||
)
|
||||
)
|
||||
.toBe(true)
|
||||
const isDisabled = await component.evaluate((el) =>
|
||||
el.tagName === 'INPUT'
|
||||
? (el as HTMLInputElement).disabled
|
||||
: el.classList.contains('p-disabled')
|
||||
)
|
||||
expect(isDisabled).toBe(true)
|
||||
})
|
||||
}
|
||||
})
|
||||
@@ -273,7 +258,7 @@ test.describe('Topbar commands', () => {
|
||||
await comfyPage.settingDialog.goToAboutPanel()
|
||||
const badge = comfyPage.page.locator('.about-badge').last()
|
||||
expect(badge).toBeDefined()
|
||||
await expect(badge).toContainText('Test Badge')
|
||||
expect(await badge.textContent()).toContain('Test Badge')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -291,13 +276,11 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
await comfyPage.nodeOps.fillPromptDialog('Hello, world!')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
expect(
|
||||
await comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
.toBe('Hello, world!')
|
||||
).toBe('Hello, world!')
|
||||
})
|
||||
|
||||
test('Should allow showing a confirmation dialog', async ({
|
||||
@@ -315,13 +298,11 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
await comfyPage.confirmDialog.click('confirm')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
expect(
|
||||
await comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
.toBe(true)
|
||||
).toBe(true)
|
||||
})
|
||||
|
||||
test('Should allow dismissing a dialog', async ({ comfyPage }) => {
|
||||
@@ -338,13 +319,11 @@ test.describe('Topbar commands', () => {
|
||||
})
|
||||
|
||||
await comfyPage.confirmDialog.click('reject')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
expect(
|
||||
await comfyPage.page.evaluate(
|
||||
() => (window as unknown as Record<string, unknown>)['value']
|
||||
)
|
||||
.toBeNull()
|
||||
).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -384,16 +363,14 @@ test.describe('Topbar commands', () => {
|
||||
)
|
||||
await toolboxButton.click()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() =>
|
||||
(window as unknown as Record<string, unknown>)[
|
||||
'selectionCommandExecuted'
|
||||
]
|
||||
)
|
||||
expect(
|
||||
await comfyPage.page.evaluate(
|
||||
() =>
|
||||
(window as unknown as Record<string, unknown>)[
|
||||
'selectionCommandExecuted'
|
||||
]
|
||||
)
|
||||
.toBe(true)
|
||||
).toBe(true)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -59,30 +59,31 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
{ timeout: 10000 }
|
||||
)
|
||||
|
||||
// Get the captured messages
|
||||
const messages = await newPage.evaluate(() => window.__capturedMessages)
|
||||
|
||||
// Verify client sent feature flags
|
||||
await expect(async () => {
|
||||
const flags = await newPage.evaluate(
|
||||
() => window.__capturedMessages?.clientFeatureFlags
|
||||
)
|
||||
expect(flags).not.toBeNull()
|
||||
expect(flags?.type).toBe('feature_flags')
|
||||
expect(flags?.data).not.toBeNull()
|
||||
expect(flags?.data).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags?.data?.supports_preview_metadata).toBe('boolean')
|
||||
}).toPass()
|
||||
expect(messages!.clientFeatureFlags).toBeTruthy()
|
||||
expect(messages!.clientFeatureFlags).toHaveProperty('type', 'feature_flags')
|
||||
expect(messages!.clientFeatureFlags).toHaveProperty('data')
|
||||
expect(messages!.clientFeatureFlags!.data).toHaveProperty(
|
||||
'supports_preview_metadata'
|
||||
)
|
||||
expect(
|
||||
typeof messages!.clientFeatureFlags!.data.supports_preview_metadata
|
||||
).toBe('boolean')
|
||||
|
||||
// Verify server sent feature flags back
|
||||
await expect(async () => {
|
||||
const flags = await newPage.evaluate(
|
||||
() => window.__capturedMessages?.serverFeatureFlags
|
||||
)
|
||||
expect(flags).not.toBeNull()
|
||||
expect(flags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags?.supports_preview_metadata).toBe('boolean')
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
expect(typeof flags?.max_upload_size).toBe('number')
|
||||
expect(Object.keys(flags ?? {}).length).toBeGreaterThan(0)
|
||||
}).toPass()
|
||||
expect(messages!.serverFeatureFlags).toBeTruthy()
|
||||
expect(messages!.serverFeatureFlags).toHaveProperty(
|
||||
'supports_preview_metadata'
|
||||
)
|
||||
expect(typeof messages!.serverFeatureFlags!.supports_preview_metadata).toBe(
|
||||
'boolean'
|
||||
)
|
||||
expect(messages!.serverFeatureFlags).toHaveProperty('max_upload_size')
|
||||
expect(typeof messages!.serverFeatureFlags!.max_upload_size).toBe('number')
|
||||
expect(Object.keys(messages!.serverFeatureFlags!).length).toBeGreaterThan(0)
|
||||
|
||||
await newPage.close()
|
||||
})
|
||||
@@ -90,44 +91,37 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
test('Server feature flags are received and accessible', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
// Get the actual server feature flags from the backend
|
||||
const serverFlags = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.serverFeatureFlags.value
|
||||
})
|
||||
|
||||
// Verify we received real feature flags from the backend
|
||||
await expect(async () => {
|
||||
const flags = await comfyPage.page.evaluate(
|
||||
() => window.app!.api.serverFeatureFlags.value
|
||||
)
|
||||
expect(flags).not.toBeNull()
|
||||
expect(Object.keys(flags).length).toBeGreaterThan(0)
|
||||
// The backend should send feature flags
|
||||
expect(flags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags.supports_preview_metadata).toBe('boolean')
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
expect(typeof flags.max_upload_size).toBe('number')
|
||||
}).toPass()
|
||||
expect(serverFlags).toBeTruthy()
|
||||
expect(Object.keys(serverFlags).length).toBeGreaterThan(0)
|
||||
|
||||
// The backend should send feature flags
|
||||
expect(serverFlags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof serverFlags.supports_preview_metadata).toBe('boolean')
|
||||
expect(serverFlags).toHaveProperty('max_upload_size')
|
||||
expect(typeof serverFlags.max_upload_size).toBe('number')
|
||||
})
|
||||
|
||||
test('serverSupportsFeature method works with real backend flags', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
// Test serverSupportsFeature with real backend flags
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() =>
|
||||
typeof window.app!.api.serverSupportsFeature(
|
||||
'supports_preview_metadata'
|
||||
)
|
||||
)
|
||||
)
|
||||
.toBe('boolean')
|
||||
const supportsPreviewMetadata = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.serverSupportsFeature('supports_preview_metadata')
|
||||
})
|
||||
// The method should return a boolean based on the backend's value
|
||||
expect(typeof supportsPreviewMetadata).toBe('boolean')
|
||||
|
||||
// Test non-existent feature - should always return false
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() =>
|
||||
window.app!.api.serverSupportsFeature('non_existent_feature_xyz')
|
||||
)
|
||||
)
|
||||
.toBe(false)
|
||||
const supportsNonExistent = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.serverSupportsFeature('non_existent_feature_xyz')
|
||||
})
|
||||
expect(supportsNonExistent).toBe(false)
|
||||
|
||||
// Test that the method only returns true for boolean true values
|
||||
const testResults = await comfyPage.page.evaluate(() => {
|
||||
@@ -166,51 +160,41 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
comfyPage
|
||||
}) => {
|
||||
// Test getServerFeature method
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(
|
||||
() =>
|
||||
typeof window.app!.api.getServerFeature('supports_preview_metadata')
|
||||
)
|
||||
)
|
||||
.toBe('boolean')
|
||||
const previewMetadataValue = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.getServerFeature('supports_preview_metadata')
|
||||
})
|
||||
expect(typeof previewMetadataValue).toBe('boolean')
|
||||
|
||||
// Test getting max_upload_size
|
||||
await expect(async () => {
|
||||
const maxUpload = await comfyPage.page.evaluate(() =>
|
||||
window.app!.api.getServerFeature('max_upload_size')
|
||||
)
|
||||
expect(typeof maxUpload).toBe('number')
|
||||
expect(maxUpload as number).toBeGreaterThan(0)
|
||||
}).toPass()
|
||||
const maxUploadSize = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.getServerFeature('max_upload_size')
|
||||
})
|
||||
expect(typeof maxUploadSize).toBe('number')
|
||||
expect(maxUploadSize).toBeGreaterThan(0)
|
||||
|
||||
// Test getServerFeature with default value for non-existent feature
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() =>
|
||||
window.app!.api.getServerFeature(
|
||||
'non_existent_feature_xyz',
|
||||
'default'
|
||||
)
|
||||
)
|
||||
const defaultValue = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.getServerFeature(
|
||||
'non_existent_feature_xyz',
|
||||
'default'
|
||||
)
|
||||
.toBe('default')
|
||||
})
|
||||
expect(defaultValue).toBe('default')
|
||||
})
|
||||
|
||||
test('getServerFeatures returns all backend feature flags', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
// Test getServerFeatures returns all flags
|
||||
await expect(async () => {
|
||||
const features = await comfyPage.page.evaluate(() =>
|
||||
window.app!.api.getServerFeatures()
|
||||
)
|
||||
expect(features).not.toBeNull()
|
||||
expect(features).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof features.supports_preview_metadata).toBe('boolean')
|
||||
expect(features).toHaveProperty('max_upload_size')
|
||||
expect(Object.keys(features).length).toBeGreaterThan(0)
|
||||
}).toPass()
|
||||
const allFeatures = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.api.getServerFeatures()
|
||||
})
|
||||
|
||||
expect(allFeatures).toBeTruthy()
|
||||
expect(allFeatures).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof allFeatures.supports_preview_metadata).toBe('boolean')
|
||||
expect(allFeatures).toHaveProperty('max_upload_size')
|
||||
expect(Object.keys(allFeatures).length).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
test('Client feature flags are immutable', async ({ comfyPage }) => {
|
||||
@@ -340,22 +324,26 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
|
||||
}
|
||||
)
|
||||
|
||||
// Verify feature flags are available
|
||||
await expect(async () => {
|
||||
const flags = await newPage.evaluate(
|
||||
() => window.app!.api.serverFeatureFlags.value
|
||||
)
|
||||
expect(flags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof flags?.supports_preview_metadata).toBe('boolean')
|
||||
expect(flags).toHaveProperty('max_upload_size')
|
||||
}).toPass()
|
||||
// Get readiness state
|
||||
const readiness = await newPage.evaluate(() => {
|
||||
return {
|
||||
...window.__appReadiness,
|
||||
currentFlags: window.app!.api.serverFeatureFlags.value
|
||||
}
|
||||
})
|
||||
|
||||
// Verify feature flags were received and API was initialized
|
||||
await expect(async () => {
|
||||
const readiness = await newPage.evaluate(() => window.__appReadiness)
|
||||
expect(readiness?.featureFlagsReceived).toBe(true)
|
||||
expect(readiness?.apiInitialized).toBe(true)
|
||||
}).toPass()
|
||||
// Verify feature flags are available
|
||||
expect(readiness.currentFlags).toHaveProperty('supports_preview_metadata')
|
||||
expect(typeof readiness.currentFlags.supports_preview_metadata).toBe(
|
||||
'boolean'
|
||||
)
|
||||
expect(readiness.currentFlags).toHaveProperty('max_upload_size')
|
||||
|
||||
// Verify feature flags were received (we detected them via polling)
|
||||
expect(readiness.featureFlagsReceived).toBe(true)
|
||||
|
||||
// Verify API was initialized (feature flags require API)
|
||||
expect(readiness.apiInitialized).toBe(true)
|
||||
|
||||
await newPage.close()
|
||||
})
|
||||
|
||||
@@ -29,9 +29,11 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
|
||||
|
||||
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.menu.sideToolbar).toBeVisible()
|
||||
})
|
||||
|
||||
|
||||
@@ -11,19 +11,17 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
|
||||
// Ref: https://github.com/Comfy-Org/ComfyUI_frontend/issues/3348
|
||||
test('Fix link input slots', async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('inputs/input_order_swap')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
return window.app!.graph!.links.get(1)?.target_slot
|
||||
})
|
||||
)
|
||||
.toBe(1)
|
||||
expect(
|
||||
await comfyPage.page.evaluate(() => {
|
||||
return window.app!.graph!.links.get(1)?.target_slot
|
||||
})
|
||||
).toBe(1)
|
||||
})
|
||||
|
||||
test('Validate workflow links', async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Validation.Workflows', true)
|
||||
await comfyPage.workflow.loadWorkflow('links/bad_link')
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(2)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(2)
|
||||
})
|
||||
|
||||
// Regression: duplicate links with shifted target_slot (widget-to-input
|
||||
@@ -38,77 +36,72 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('links/duplicate_links_slot_drift')
|
||||
|
||||
function evaluateGraph() {
|
||||
return comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
const result = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
|
||||
const subgraph = graph.subgraphs.values().next().value
|
||||
if (!subgraph) return { error: 'No subgraph found' }
|
||||
const subgraph = graph.subgraphs.values().next().value
|
||||
if (!subgraph) return { error: 'No subgraph found' }
|
||||
|
||||
// Node 120 = Switch (CFG), connects to both KSamplerAdvanced 85 and 86
|
||||
const switchCfg = subgraph.getNodeById(120)
|
||||
const ksampler85 = subgraph.getNodeById(85)
|
||||
const ksampler86 = subgraph.getNodeById(86)
|
||||
if (!switchCfg || !ksampler85 || !ksampler86)
|
||||
return { error: 'Required nodes not found' }
|
||||
// Node 120 = Switch (CFG), connects to both KSamplerAdvanced 85 and 86
|
||||
const switchCfg = subgraph.getNodeById(120)
|
||||
const ksampler85 = subgraph.getNodeById(85)
|
||||
const ksampler86 = subgraph.getNodeById(86)
|
||||
if (!switchCfg || !ksampler85 || !ksampler86)
|
||||
return { error: 'Required nodes not found' }
|
||||
|
||||
// Find cfg inputs by name (slot indices shift due to widget-to-input)
|
||||
const cfgInput85 = ksampler85.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
const cfgInput86 = ksampler86.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
const cfg85Linked = cfgInput85?.link != null
|
||||
const cfg86Linked = cfgInput86?.link != null
|
||||
|
||||
// Verify the surviving links exist in the subgraph link map
|
||||
const cfg85LinkValid =
|
||||
cfg85Linked && subgraph.links.has(cfgInput85!.link!)
|
||||
const cfg86LinkValid =
|
||||
cfg86Linked && subgraph.links.has(cfgInput86!.link!)
|
||||
|
||||
// Switch(CFG) output should have exactly 2 links (one to each KSampler)
|
||||
const switchOutputLinkCount = switchCfg.outputs[0]?.links?.length ?? 0
|
||||
|
||||
// Count links from Switch(CFG) to node 85 cfg (should be 1, not 2)
|
||||
let cfgLinkToNode85Count = 0
|
||||
for (const link of subgraph.links.values()) {
|
||||
if (link.origin_id === 120 && link.target_id === 85)
|
||||
cfgLinkToNode85Count++
|
||||
}
|
||||
|
||||
return {
|
||||
cfg85Linked,
|
||||
cfg86Linked,
|
||||
cfg85LinkValid,
|
||||
cfg86LinkValid,
|
||||
cfg85LinkId: cfgInput85?.link ?? null,
|
||||
cfg86LinkId: cfgInput86?.link ?? null,
|
||||
switchOutputLinkIds: [...(switchCfg.outputs[0]?.links ?? [])],
|
||||
switchOutputLinkCount,
|
||||
cfgLinkToNode85Count
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Poll graph state once, then assert all properties
|
||||
await expect(async () => {
|
||||
const r = await evaluateGraph()
|
||||
// Both KSamplerAdvanced nodes must have their cfg input connected
|
||||
expect(r.cfg85Linked).toBe(true)
|
||||
expect(r.cfg86Linked).toBe(true)
|
||||
// Links must exist in the subgraph link map
|
||||
expect(r.cfg85LinkValid).toBe(true)
|
||||
expect(r.cfg86LinkValid).toBe(true)
|
||||
// Switch(CFG) output has exactly 2 links (one per KSamplerAdvanced)
|
||||
expect(r.switchOutputLinkCount).toBe(2)
|
||||
// Only 1 link from Switch(CFG) to node 85 (duplicate removed)
|
||||
expect(r.cfgLinkToNode85Count).toBe(1)
|
||||
// Output link IDs must match the input link IDs (source/target integrity)
|
||||
expect(r.switchOutputLinkIds).toEqual(
|
||||
expect.arrayContaining([r.cfg85LinkId, r.cfg86LinkId])
|
||||
// Find cfg inputs by name (slot indices shift due to widget-to-input)
|
||||
const cfgInput85 = ksampler85.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
}).toPass()
|
||||
const cfgInput86 = ksampler86.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
const cfg85Linked = cfgInput85?.link != null
|
||||
const cfg86Linked = cfgInput86?.link != null
|
||||
|
||||
// Verify the surviving links exist in the subgraph link map
|
||||
const cfg85LinkValid =
|
||||
cfg85Linked && subgraph.links.has(cfgInput85!.link!)
|
||||
const cfg86LinkValid =
|
||||
cfg86Linked && subgraph.links.has(cfgInput86!.link!)
|
||||
|
||||
// Switch(CFG) output should have exactly 2 links (one to each KSampler)
|
||||
const switchOutputLinkCount = switchCfg.outputs[0]?.links?.length ?? 0
|
||||
|
||||
// Count links from Switch(CFG) to node 85 cfg (should be 1, not 2)
|
||||
let cfgLinkToNode85Count = 0
|
||||
for (const link of subgraph.links.values()) {
|
||||
if (link.origin_id === 120 && link.target_id === 85)
|
||||
cfgLinkToNode85Count++
|
||||
}
|
||||
|
||||
return {
|
||||
cfg85Linked,
|
||||
cfg86Linked,
|
||||
cfg85LinkValid,
|
||||
cfg86LinkValid,
|
||||
cfg85LinkId: cfgInput85?.link ?? null,
|
||||
cfg86LinkId: cfgInput86?.link ?? null,
|
||||
switchOutputLinkIds: [...(switchCfg.outputs[0]?.links ?? [])],
|
||||
switchOutputLinkCount,
|
||||
cfgLinkToNode85Count
|
||||
}
|
||||
})
|
||||
|
||||
expect(result).not.toHaveProperty('error')
|
||||
// Both KSamplerAdvanced nodes must have their cfg input connected
|
||||
expect(result.cfg85Linked).toBe(true)
|
||||
expect(result.cfg86Linked).toBe(true)
|
||||
// Links must exist in the subgraph link map
|
||||
expect(result.cfg85LinkValid).toBe(true)
|
||||
expect(result.cfg86LinkValid).toBe(true)
|
||||
// Switch(CFG) output has exactly 2 links (one per KSamplerAdvanced)
|
||||
expect(result.switchOutputLinkCount).toBe(2)
|
||||
// Only 1 link from Switch(CFG) to node 85 (duplicate removed)
|
||||
expect(result.cfgLinkToNode85Count).toBe(1)
|
||||
// Output link IDs must match the input link IDs (source/target integrity)
|
||||
expect(result.switchOutputLinkIds).toEqual(
|
||||
expect.arrayContaining([result.cfg85LinkId, result.cfg86LinkId])
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -31,18 +31,18 @@ test.describe('Graph Canvas Menu', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
const hiddenLinkRenderMode = await comfyPage.page.evaluate(() => {
|
||||
return window.LiteGraph!.HIDDEN_LINK
|
||||
})
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.LinkRenderMode'))
|
||||
.toBe(hiddenLinkRenderMode)
|
||||
expect(await comfyPage.settings.getSetting('Comfy.LinkRenderMode')).toBe(
|
||||
hiddenLinkRenderMode
|
||||
)
|
||||
|
||||
await button.click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'canvas-with-visible-links.png'
|
||||
)
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.LinkRenderMode'))
|
||||
.not.toBe(hiddenLinkRenderMode)
|
||||
expect(
|
||||
await comfyPage.settings.getSetting('Comfy.LinkRenderMode')
|
||||
).not.toBe(hiddenLinkRenderMode)
|
||||
}
|
||||
)
|
||||
|
||||
@@ -92,6 +92,7 @@ test.describe('Graph Canvas Menu', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
// Click backdrop to close
|
||||
const backdrop = comfyPage.page.locator('.fixed.inset-0').first()
|
||||
await backdrop.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Modal should be hidden
|
||||
await expect(zoomModal).not.toBeVisible()
|
||||
|
||||
@@ -28,20 +28,17 @@ test.describe('Group Copy Paste', { tag: ['@canvas'] }, () => {
|
||||
await comfyPage.clipboard.paste()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const getGroupPositions = () =>
|
||||
comfyPage.page.evaluate(() =>
|
||||
window.app!.graph.groups.map((g: { pos: number[] }) => ({
|
||||
x: g.pos[0],
|
||||
y: g.pos[1]
|
||||
}))
|
||||
)
|
||||
const positions = await comfyPage.page.evaluate(() =>
|
||||
window.app!.graph.groups.map((g: { pos: number[] }) => ({
|
||||
x: g.pos[0],
|
||||
y: g.pos[1]
|
||||
}))
|
||||
)
|
||||
|
||||
await expect.poll(getGroupPositions).toHaveLength(2)
|
||||
|
||||
await expect(async () => {
|
||||
const positions = await getGroupPositions()
|
||||
expect(Math.abs(positions[0].x - positions[1].x)).toBeCloseTo(50, 0)
|
||||
expect(Math.abs(positions[0].y - positions[1].y)).toBeCloseTo(15, 0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
expect(positions).toHaveLength(2)
|
||||
const dx = Math.abs(positions[0].x - positions[1].x)
|
||||
const dy = Math.abs(positions[0].y - positions[1].y)
|
||||
expect(dx).toBeCloseTo(50, 0)
|
||||
expect(dy).toBeCloseTo(15, 0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
|
||||
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import {
|
||||
comfyExpect as expect,
|
||||
comfyPageFixture as test
|
||||
} from '@e2e/fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
import type { NodeLibrarySidebarTab } from '@e2e/fixtures/components/SidebarTab'
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
import { DefaultGraphPositions } from '@e2e/fixtures/constants/defaultGraphPositions'
|
||||
@@ -33,7 +32,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
test('Is added to node library sidebar', async ({
|
||||
comfyPage: _comfyPage
|
||||
}) => {
|
||||
await expect(libraryTab.getFolder(groupNodeCategory)).toHaveCount(1)
|
||||
expect(await libraryTab.getFolder(groupNodeCategory).count()).toBe(1)
|
||||
})
|
||||
|
||||
test('Can be added to canvas using node library sidebar', async ({
|
||||
@@ -46,9 +45,9 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
await libraryTab.getNode(groupNodeName).click()
|
||||
|
||||
// Verify the node is added to the canvas
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialNodeCount + 1)
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
|
||||
initialNodeCount + 1
|
||||
)
|
||||
})
|
||||
|
||||
test('Can be bookmarked and unbookmarked', async ({ comfyPage }) => {
|
||||
@@ -59,13 +58,11 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
.click()
|
||||
|
||||
// Verify the node is added to the bookmarks tab
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.settings.getSetting('Comfy.NodeLibrary.Bookmarks.V2')
|
||||
)
|
||||
.toEqual([groupNodeBookmarkName])
|
||||
expect(
|
||||
await comfyPage.settings.getSetting('Comfy.NodeLibrary.Bookmarks.V2')
|
||||
).toEqual([groupNodeBookmarkName])
|
||||
// Verify the bookmark node with the same name is added to the tree
|
||||
await expect(libraryTab.getNode(groupNodeName)).not.toHaveCount(0)
|
||||
expect(await libraryTab.getNode(groupNodeName).count()).not.toBe(0)
|
||||
|
||||
// Unbookmark the node
|
||||
await libraryTab
|
||||
@@ -75,11 +72,9 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
.click()
|
||||
|
||||
// Verify the node is removed from the bookmarks tab
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.settings.getSetting('Comfy.NodeLibrary.Bookmarks.V2')
|
||||
)
|
||||
.toHaveLength(0)
|
||||
expect(
|
||||
await comfyPage.settings.getSetting('Comfy.NodeLibrary.Bookmarks.V2')
|
||||
).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('Displays preview on bookmark hover', async ({ comfyPage }) => {
|
||||
@@ -89,9 +84,9 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
.locator('.bookmark-button')
|
||||
.click()
|
||||
await comfyPage.page.hover('.p-tree-node-label.tree-explorer-node-label')
|
||||
await expect(
|
||||
comfyPage.page.locator('.node-lib-node-preview')
|
||||
).toBeVisible()
|
||||
expect(await comfyPage.page.isVisible('.node-lib-node-preview')).toBe(
|
||||
true
|
||||
)
|
||||
await libraryTab
|
||||
.getNode(groupNodeName)
|
||||
.locator('.bookmark-button')
|
||||
@@ -152,12 +147,12 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
|
||||
const manage1 = await group1.manageGroupNode()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(manage1.selectedNodeTypeSelect).toHaveValue('g1')
|
||||
expect(await manage1.getSelectedNodeType()).toBe('g1')
|
||||
await manage1.close()
|
||||
await expect(manage1.root).not.toBeVisible()
|
||||
|
||||
const manage2 = await group2.manageGroupNode()
|
||||
await expect(manage2.selectedNodeTypeSelect).toHaveValue('g2')
|
||||
expect(await manage2.getSelectedNodeType()).toBe('g2')
|
||||
})
|
||||
|
||||
test('Preserves hidden input configuration when containing duplicate node types', async ({
|
||||
@@ -171,31 +166,24 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
const groupNodeId = 19
|
||||
const groupNodeName = 'two_VAE_decode'
|
||||
|
||||
const totalInputCount = await comfyPage.page.evaluate((nodeName) => {
|
||||
const {
|
||||
extra: { groupNodes }
|
||||
} = window.app!.graph!
|
||||
const { nodes } = groupNodes![nodeName]
|
||||
return nodes.reduce((acc, node) => acc + (node.inputs?.length ?? 0), 0)
|
||||
}, groupNodeName)
|
||||
|
||||
const visibleInputCount = await comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.graph!.getNodeById(id)
|
||||
return node!.inputs.length
|
||||
}, groupNodeId)
|
||||
|
||||
// Verify there are 4 total inputs (2 VAE decode nodes with 2 inputs each)
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate((nodeName) => {
|
||||
const {
|
||||
extra: { groupNodes }
|
||||
} = window.app!.graph!
|
||||
const { nodes } = groupNodes![nodeName]
|
||||
return nodes.reduce(
|
||||
(acc, node) => acc + (node.inputs?.length ?? 0),
|
||||
0
|
||||
)
|
||||
}, groupNodeName)
|
||||
)
|
||||
.toBe(4)
|
||||
expect(totalInputCount).toBe(4)
|
||||
|
||||
// Verify there are 2 visible inputs (2 have been hidden in config)
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.graph!.getNodeById(id)
|
||||
return node!.inputs.length
|
||||
}, groupNodeId)
|
||||
)
|
||||
.toBe(2)
|
||||
expect(visibleInputCount).toBe(2)
|
||||
})
|
||||
|
||||
test('Reconnects inputs after configuration changed via manage dialog save', async ({
|
||||
@@ -222,7 +210,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
// Connect node to group
|
||||
const ckpt = await expectSingleNode('CheckpointLoaderSimple')
|
||||
const input = await ckpt.connectOutput(0, groupNode, 0)
|
||||
await expect.poll(() => input.getLinkCount()).toBe(1)
|
||||
expect(await input.getLinkCount()).toBe(1)
|
||||
// Modify the group node via manage dialog
|
||||
const manage = await groupNode.manageGroupNode()
|
||||
await manage.selectNode('KSampler')
|
||||
@@ -231,14 +219,14 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
await manage.save()
|
||||
await manage.close()
|
||||
// Ensure the link is still present
|
||||
await expect.poll(() => input.getLinkCount()).toBe(1)
|
||||
expect(await input.getLinkCount()).toBe(1)
|
||||
})
|
||||
|
||||
test('Loads from a workflow using the legacy path separator ("/")', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('groupnodes/legacy_group_node')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
await expect(
|
||||
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
|
||||
).not.toBeVisible()
|
||||
@@ -273,8 +261,8 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
expect(
|
||||
await comfyPage.nodeOps.getNodeRefsByType(GROUP_NODE_TYPE)
|
||||
).toHaveLength(expectedCount)
|
||||
await expect.poll(() => isRegisteredLitegraph(comfyPage)).toBe(true)
|
||||
await expect.poll(() => isRegisteredNodeDefStore(comfyPage)).toBe(true)
|
||||
expect(await isRegisteredLitegraph(comfyPage)).toBe(true)
|
||||
expect(await isRegisteredNodeDefStore(comfyPage)).toBe(true)
|
||||
}
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
@@ -345,18 +333,18 @@ test.describe('Group Node', { tag: '@node' }, () => {
|
||||
|
||||
test.describe('Keybindings', () => {
|
||||
test('Convert to group node, no selection', async ({ comfyPage }) => {
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(0)
|
||||
await comfyPage.page.keyboard.press('Alt+g')
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(1)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(1)
|
||||
})
|
||||
test('Convert to group node, selected 1 node', async ({ comfyPage }) => {
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(0)
|
||||
await comfyPage.canvas.click({
|
||||
position: DefaultGraphPositions.textEncodeNode1
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.keyboard.press('Alt+g')
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(1)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(1)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -66,14 +66,11 @@ test.describe('Group Select Children', { tag: ['@canvas', '@node'] }, () => {
|
||||
await comfyPage.canvas.click({ position: outerPos })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const counts = await getSelectionCounts(comfyPage)
|
||||
// Outer Group + Inner Group + 1 node = 3 items
|
||||
await expect
|
||||
.poll(() => getSelectionCounts(comfyPage))
|
||||
.toMatchObject({
|
||||
selectedItemCount: 3,
|
||||
selectedGroupCount: 2,
|
||||
selectedNodeCount: 1
|
||||
})
|
||||
expect(counts.selectedItemCount).toBe(3)
|
||||
expect(counts.selectedGroupCount).toBe(2)
|
||||
expect(counts.selectedNodeCount).toBe(1)
|
||||
})
|
||||
|
||||
test('Setting disabled: clicking outer group selects only the group', async ({
|
||||
@@ -90,13 +87,10 @@ test.describe('Group Select Children', { tag: ['@canvas', '@node'] }, () => {
|
||||
await comfyPage.canvas.click({ position: outerPos })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => getSelectionCounts(comfyPage))
|
||||
.toMatchObject({
|
||||
selectedItemCount: 1,
|
||||
selectedGroupCount: 1,
|
||||
selectedNodeCount: 0
|
||||
})
|
||||
const counts = await getSelectionCounts(comfyPage)
|
||||
expect(counts.selectedItemCount).toBe(1)
|
||||
expect(counts.selectedGroupCount).toBe(1)
|
||||
expect(counts.selectedNodeCount).toBe(0)
|
||||
})
|
||||
|
||||
test('Deselecting outer group deselects all children', async ({
|
||||
@@ -114,9 +108,8 @@ test.describe('Group Select Children', { tag: ['@canvas', '@node'] }, () => {
|
||||
await comfyPage.canvas.click({ position: outerPos })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => getSelectionCounts(comfyPage))
|
||||
.toMatchObject({ selectedItemCount: 3 })
|
||||
let counts = await getSelectionCounts(comfyPage)
|
||||
expect(counts.selectedItemCount).toBe(3)
|
||||
|
||||
// Deselect all via page.evaluate to avoid UI overlay interception
|
||||
await comfyPage.page.evaluate(() => {
|
||||
@@ -124,8 +117,7 @@ test.describe('Group Select Children', { tag: ['@canvas', '@node'] }, () => {
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => getSelectionCounts(comfyPage))
|
||||
.toMatchObject({ selectedItemCount: 0 })
|
||||
counts = await getSelectionCounts(comfyPage)
|
||||
expect(counts.selectedItemCount).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -11,8 +11,8 @@ test.describe(
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('nodes/load_image_with_ksampler')
|
||||
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(2)
|
||||
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(initialCount).toBe(2)
|
||||
|
||||
// Copy the KSampler node (puts data-metadata in clipboard)
|
||||
const ksamplerNodes =
|
||||
@@ -51,9 +51,8 @@ test.describe(
|
||||
|
||||
// Node count should remain the same — stale node metadata should NOT
|
||||
// be deserialized when a media node is selected.
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialCount)
|
||||
const finalCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(finalCount).toBe(initialCount)
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
@@ -62,9 +62,9 @@ test.describe('Node Interaction', () => {
|
||||
for (const node of clipNodes) {
|
||||
await node.click('title', { modifiers: [modifier] })
|
||||
}
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(clipNodes.length)
|
||||
const selectedNodeCount =
|
||||
await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedNodeCount).toBe(clipNodes.length)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -111,9 +111,9 @@ test.describe('Node Interaction', () => {
|
||||
const clipNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
await dragSelectNodes(comfyPage, clipNodes)
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(clipNodes.length)
|
||||
expect(await comfyPage.nodeOps.getSelectedGraphNodesCount()).toBe(
|
||||
clipNodes.length
|
||||
)
|
||||
})
|
||||
|
||||
test('Can move selected nodes using the Comfy.Canvas.MoveSelectedNodes.{Up|Down|Left|Right} commands', async ({
|
||||
@@ -243,7 +243,6 @@ test.describe('Node Interaction', () => {
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Node.AutoSnapLinkToSlot', true)
|
||||
await comfyPage.settings.setSetting('Comfy.Node.SnapHighlightsNode', true)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyMouse.move(DefaultGraphPositions.clipTextEncodeNode1InputSlot)
|
||||
await comfyMouse.drag(DefaultGraphPositions.clipTextEncodeNode2InputSlot)
|
||||
@@ -328,42 +327,18 @@ test.describe('Node Interaction', () => {
|
||||
'Can toggle dom widget node open/closed',
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
// Find the node whose collapse toggler matches the hardcoded position.
|
||||
// getNodeRefsByType order is non-deterministic, so identify by proximity.
|
||||
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
const togglerPos = DefaultGraphPositions.textEncodeNodeToggler
|
||||
let targetNode = nodes[0]
|
||||
let minDist = Infinity
|
||||
for (const n of nodes) {
|
||||
const pos = await n.getPosition()
|
||||
const dist = Math.hypot(pos.x - togglerPos.x, pos.y - togglerPos.y)
|
||||
if (dist < minDist) {
|
||||
minDist = dist
|
||||
targetNode = n
|
||||
}
|
||||
}
|
||||
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('default.png')
|
||||
await comfyPage.canvas.click({
|
||||
position: togglerPos
|
||||
position: DefaultGraphPositions.textEncodeNodeToggler
|
||||
})
|
||||
await expect.poll(() => targetNode.isCollapsed()).toBe(true)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'text-encode-toggled-off.png'
|
||||
)
|
||||
// Re-expand: clicking the canvas toggler on a collapsed node is
|
||||
// unreliable because DOM widget overlays may intercept the pointer
|
||||
// event. Use programmatic collapse() for the expand step.
|
||||
// TODO(#11006): Restore click-to-expand once DOM widget overlay pointer interception is fixed
|
||||
await comfyPage.page.evaluate((nodeId) => {
|
||||
const node = window.app!.graph.getNodeById(nodeId)!
|
||||
node.collapse()
|
||||
window.app!.canvas.setDirty(true, true)
|
||||
}, targetNode.id)
|
||||
await comfyPage.nextFrame()
|
||||
await expect.poll(() => targetNode.isCollapsed()).toBe(false)
|
||||
// Move mouse away to avoid hover highlight differences.
|
||||
await comfyPage.canvasOps.moveMouseToEmptyArea()
|
||||
await comfyPage.delay(1000)
|
||||
await comfyPage.canvas.click({
|
||||
position: DefaultGraphPositions.textEncodeNodeToggler
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'text-encode-toggled-back-open.png'
|
||||
@@ -383,17 +358,14 @@ test.describe('Node Interaction', () => {
|
||||
})
|
||||
const legacyPrompt = comfyPage.page.locator('.graphdialog')
|
||||
await expect(legacyPrompt).toBeVisible()
|
||||
// LiteGraph's graphdialog has a 256ms dismiss guard (Date.now() - clickTime > 256).
|
||||
// Retry the canvas click until the dialog actually closes.
|
||||
await expect(async () => {
|
||||
await comfyPage.canvas.click({
|
||||
position: {
|
||||
x: 10,
|
||||
y: 10
|
||||
}
|
||||
})
|
||||
await expect(legacyPrompt).toBeHidden({ timeout: 500 })
|
||||
}).toPass({ timeout: 5000 })
|
||||
await comfyPage.delay(300)
|
||||
await comfyPage.canvas.click({
|
||||
position: {
|
||||
x: 10,
|
||||
y: 10
|
||||
}
|
||||
})
|
||||
await expect(legacyPrompt).toBeHidden()
|
||||
})
|
||||
|
||||
test('Can close prompt dialog with canvas click (text widget)', async ({
|
||||
@@ -409,17 +381,14 @@ test.describe('Node Interaction', () => {
|
||||
})
|
||||
const legacyPrompt = comfyPage.page.locator('.graphdialog')
|
||||
await expect(legacyPrompt).toBeVisible()
|
||||
// LiteGraph's graphdialog has a 256ms dismiss guard (Date.now() - clickTime > 256).
|
||||
// Retry the canvas click until the dialog actually closes.
|
||||
await expect(async () => {
|
||||
await comfyPage.canvas.click({
|
||||
position: {
|
||||
x: 10,
|
||||
y: 10
|
||||
}
|
||||
})
|
||||
await expect(legacyPrompt).toBeHidden({ timeout: 500 })
|
||||
}).toPass({ timeout: 5000 })
|
||||
await comfyPage.delay(300)
|
||||
await comfyPage.canvas.click({
|
||||
position: {
|
||||
x: 10,
|
||||
y: 10
|
||||
}
|
||||
})
|
||||
await expect(legacyPrompt).toBeHidden()
|
||||
})
|
||||
|
||||
test(
|
||||
@@ -451,7 +420,7 @@ test.describe('Node Interaction', () => {
|
||||
},
|
||||
delay: 5
|
||||
})
|
||||
await expect(comfyPage.page.locator('.node-title-editor')).toHaveCount(0)
|
||||
expect(await comfyPage.page.locator('.node-title-editor').count()).toBe(0)
|
||||
})
|
||||
|
||||
test(
|
||||
@@ -481,31 +450,10 @@ test.describe('Node Interaction', () => {
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('groups/oversized_group')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const group = window.app!.graph.groups[0]
|
||||
return group ? [group.size[0], group.size[1]] : null
|
||||
})
|
||||
)
|
||||
.not.toBeNull()
|
||||
|
||||
const initialGroupSize = await comfyPage.page.evaluate(() => {
|
||||
const group = window.app!.graph.groups[0]
|
||||
return group ? [group.size[0], group.size[1]] : null
|
||||
})
|
||||
|
||||
await comfyPage.keyboard.selectAll()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.command.executeCommand('Comfy.Graph.FitGroupToContents')
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const group = window.app!.graph.groups[0]
|
||||
return group ? [group.size[0], group.size[1]] : null
|
||||
})
|
||||
)
|
||||
.not.toEqual(initialGroupSize)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'group-fit-to-contents.png'
|
||||
)
|
||||
@@ -514,19 +462,15 @@ test.describe('Node Interaction', () => {
|
||||
|
||||
test('Can pin/unpin nodes', { tag: '@screenshot' }, async ({ comfyPage }) => {
|
||||
await comfyPage.nodeOps.selectNodes(['CLIP Text Encode (Prompt)'])
|
||||
const nodeRef = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('CLIP Text Encode (Prompt)')
|
||||
)[0]
|
||||
|
||||
await comfyPage.command.executeCommand(
|
||||
'Comfy.Canvas.ToggleSelectedNodes.Pin'
|
||||
)
|
||||
await expect.poll(() => nodeRef.isPinned()).toBe(true)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('nodes-pinned.png')
|
||||
await comfyPage.command.executeCommand(
|
||||
'Comfy.Canvas.ToggleSelectedNodes.Pin'
|
||||
)
|
||||
await expect.poll(() => nodeRef.isPinned()).toBe(false)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('nodes-unpinned.png')
|
||||
})
|
||||
|
||||
@@ -535,15 +479,11 @@ test.describe('Node Interaction', () => {
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.nodeOps.selectNodes(['CLIP Text Encode (Prompt)'])
|
||||
const nodeRef = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('CLIP Text Encode (Prompt)')
|
||||
)[0]
|
||||
|
||||
await comfyPage.canvas.press('Control+b')
|
||||
await expect.poll(() => nodeRef.isBypassed()).toBe(true)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('nodes-bypassed.png')
|
||||
await comfyPage.canvas.press('Control+b')
|
||||
await expect.poll(() => nodeRef.isBypassed()).toBe(false)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('nodes-unbypassed.png')
|
||||
}
|
||||
)
|
||||
@@ -642,23 +582,23 @@ test.describe('Canvas Interaction', { tag: '@screenshot' }, () => {
|
||||
}
|
||||
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
await comfyPage.page.mouse.down()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
// Move mouse should not alter cursor style.
|
||||
await comfyPage.page.mouse.move(10, 20)
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
await comfyPage.page.mouse.up()
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
|
||||
await comfyPage.page.keyboard.down('Space')
|
||||
await expect.poll(() => getCursorStyle()).toBe('grab')
|
||||
expect(await getCursorStyle()).toBe('grab')
|
||||
await comfyPage.page.mouse.down()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
await comfyPage.page.mouse.up()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grab')
|
||||
expect(await getCursorStyle()).toBe('grab')
|
||||
await comfyPage.page.keyboard.up('Space')
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
})
|
||||
|
||||
// https://github.com/Comfy-Org/litegraph.js/pull/424
|
||||
@@ -675,27 +615,27 @@ test.describe('Canvas Interaction', { tag: '@screenshot' }, () => {
|
||||
|
||||
// Initial state check
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
|
||||
// Click and hold
|
||||
await comfyPage.page.mouse.down()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
|
||||
// Press space while holding click
|
||||
await comfyPage.page.keyboard.down('Space')
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
|
||||
// Release click while space is still down
|
||||
await comfyPage.page.mouse.up()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grab')
|
||||
expect(await getCursorStyle()).toBe('grab')
|
||||
|
||||
// Release space
|
||||
await comfyPage.page.keyboard.up('Space')
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
|
||||
// Move mouse - cursor should remain default
|
||||
await comfyPage.page.mouse.move(20, 20)
|
||||
await expect.poll(() => getCursorStyle()).toBe('default')
|
||||
expect(await getCursorStyle()).toBe('default')
|
||||
})
|
||||
|
||||
test('Can pan when dragging a link', async ({ comfyPage, comfyMouse }) => {
|
||||
@@ -786,14 +726,11 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
await comfyPage.settings.setSetting('Comfy.Workflow.Persist', false)
|
||||
await comfyPage.setup()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
return (window.app!.extensionManager as WorkspaceStore).workflow
|
||||
.openWorkflows.length
|
||||
})
|
||||
)
|
||||
.toBeGreaterThanOrEqual(1)
|
||||
const openCount = await comfyPage.page.evaluate(() => {
|
||||
return (window.app!.extensionManager as WorkspaceStore).workflow
|
||||
.openWorkflows.length
|
||||
})
|
||||
expect(openCount).toBeGreaterThanOrEqual(1)
|
||||
})
|
||||
|
||||
test('Restore workflow on reload (switch workflow)', async ({
|
||||
@@ -879,22 +816,14 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.topbar.getTabNames())
|
||||
.poll(() => comfyPage.menu.topbar.getTabNames(), { timeout: 5000 })
|
||||
.toEqual(expect.arrayContaining([workflowA, workflowB]))
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const tabs = await comfyPage.menu.topbar.getTabNames()
|
||||
return (
|
||||
tabs.indexOf(workflowA) < tabs.indexOf(workflowB) &&
|
||||
tabs.indexOf(workflowA) >= 0
|
||||
)
|
||||
})
|
||||
.toBe(true)
|
||||
const tabs = await comfyPage.menu.topbar.getTabNames()
|
||||
expect(tabs.indexOf(workflowA)).toBeLessThan(tabs.indexOf(workflowB))
|
||||
|
||||
await expect(comfyPage.menu.topbar.getActiveTab()).toContainText(
|
||||
workflowB
|
||||
)
|
||||
const activeWorkflowName = await comfyPage.menu.topbar.getActiveTabName()
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
|
||||
test('Restores sidebar workflows after reload', async ({ comfyPage }) => {
|
||||
@@ -903,18 +832,17 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
'Sidebar'
|
||||
)
|
||||
await comfyPage.menu.workflowsTab.open()
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.workflowsTab.getOpenedWorkflowNames())
|
||||
.toEqual(expect.arrayContaining([workflowA, workflowB]))
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const ws = await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
|
||||
return ws.indexOf(workflowA) < ws.indexOf(workflowB)
|
||||
})
|
||||
.toBe(true)
|
||||
await expect(comfyPage.menu.workflowsTab.activeWorkflowLabel).toHaveText(
|
||||
workflowB
|
||||
const openWorkflows =
|
||||
await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
|
||||
const activeWorkflowName =
|
||||
await comfyPage.menu.workflowsTab.getActiveWorkflowName()
|
||||
expect(openWorkflows).toEqual(
|
||||
expect.arrayContaining([workflowA, workflowB])
|
||||
)
|
||||
expect(openWorkflows.indexOf(workflowA)).toBeLessThan(
|
||||
openWorkflows.indexOf(workflowB)
|
||||
)
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -964,20 +892,12 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
})
|
||||
).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const tabs = await comfyPage.menu.topbar.getTabNames()
|
||||
return (
|
||||
tabs.includes(workflowA) &&
|
||||
tabs.includes(workflowB) &&
|
||||
tabs.indexOf(workflowA) < tabs.indexOf(workflowB)
|
||||
)
|
||||
})
|
||||
.toBe(true)
|
||||
const tabs = await comfyPage.menu.topbar.getTabNames()
|
||||
const activeWorkflowName = await comfyPage.menu.topbar.getActiveTabName()
|
||||
|
||||
await expect(comfyPage.menu.topbar.getActiveTab()).toContainText(
|
||||
workflowB
|
||||
)
|
||||
expect(tabs).toEqual(expect.arrayContaining([workflowA, workflowB]))
|
||||
expect(tabs.indexOf(workflowA)).toBeLessThan(tabs.indexOf(workflowB))
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
|
||||
test('Restores sidebar workflows after browser restart', async ({
|
||||
@@ -988,18 +908,17 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
'Sidebar'
|
||||
)
|
||||
await comfyPage.menu.workflowsTab.open()
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.workflowsTab.getOpenedWorkflowNames())
|
||||
.toEqual(expect.arrayContaining([workflowA, workflowB]))
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const ws = await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
|
||||
return ws.indexOf(workflowA) < ws.indexOf(workflowB)
|
||||
})
|
||||
.toBe(true)
|
||||
await expect(comfyPage.menu.workflowsTab.activeWorkflowLabel).toHaveText(
|
||||
workflowB
|
||||
const openWorkflows =
|
||||
await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
|
||||
const activeWorkflowName =
|
||||
await comfyPage.menu.workflowsTab.getActiveWorkflowName()
|
||||
expect(openWorkflows).toEqual(
|
||||
expect.arrayContaining([workflowA, workflowB])
|
||||
)
|
||||
expect(openWorkflows.indexOf(workflowA)).toBeLessThan(
|
||||
openWorkflows.indexOf(workflowB)
|
||||
)
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1025,7 +944,7 @@ test.describe('Load duplicate workflow', () => {
|
||||
await comfyPage.menu.workflowsTab.open()
|
||||
await comfyPage.command.executeCommand('Comfy.NewBlankWorkflow')
|
||||
await comfyPage.workflow.loadWorkflow('nodes/single_ksampler')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1156,9 +1075,8 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
position: DefaultGraphPositions.textEncodeNode1
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(1)
|
||||
const selectedCount = await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCount).toBe(1)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'legacy-click-node-select.png'
|
||||
)
|
||||
@@ -1193,9 +1111,8 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
}
|
||||
)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(clipNodes.length)
|
||||
const selectedCount = await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCount).toBe(clipNodes.length)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'standard-left-drag-select.png'
|
||||
)
|
||||
@@ -1238,9 +1155,8 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
position: DefaultGraphPositions.textEncodeNode1
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(1)
|
||||
const selectedCount = await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCount).toBe(1)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'standard-click-node-select.png'
|
||||
)
|
||||
@@ -1281,14 +1197,14 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
}
|
||||
)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBeGreaterThan(0)
|
||||
const selectedCountAfterDrag =
|
||||
await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCountAfterDrag).toBeGreaterThan(0)
|
||||
|
||||
await comfyPage.canvasOps.clickEmptySpace()
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(0)
|
||||
const selectedCountAfterClear =
|
||||
await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCountAfterClear).toBe(0)
|
||||
|
||||
await comfyPage.page.keyboard.down('Space')
|
||||
await comfyPage.canvasOps.dragAndDrop(
|
||||
@@ -1303,9 +1219,9 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
)
|
||||
await comfyPage.page.keyboard.up('Space')
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBe(0)
|
||||
const selectedCountAfterSpaceDrag =
|
||||
await comfyPage.nodeOps.getSelectedGraphNodesCount()
|
||||
expect(selectedCountAfterSpaceDrag).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1389,7 +1305,7 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
|
||||
)
|
||||
await comfyPage.page.mouse.move(50, 50)
|
||||
await comfyPage.page.mouse.down()
|
||||
await expect.poll(() => getCursorStyle()).toBe('grabbing')
|
||||
expect(await getCursorStyle()).toBe('grabbing')
|
||||
await comfyPage.page.mouse.up()
|
||||
})
|
||||
})
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 91 KiB After Width: | Height: | Size: 91 KiB |
@@ -83,10 +83,9 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
|
||||
)
|
||||
await action.click()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.settings.getSetting<boolean>('Comfy.Queue.ShowRunProgressBar')
|
||||
)
|
||||
.toBe(!settingBefore)
|
||||
const settingAfter = await comfyPage.settings.getSetting<boolean>(
|
||||
'Comfy.Queue.ShowRunProgressBar'
|
||||
)
|
||||
expect(settingAfter).toBe(!settingBefore)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -18,9 +18,9 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
await textBox.click()
|
||||
await textBox.fill('k')
|
||||
await expect(textBox).toHaveValue('k')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.TestCommand))
|
||||
.toBe(undefined)
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(
|
||||
undefined
|
||||
)
|
||||
})
|
||||
|
||||
test('Should not trigger modifier keybinding when typing in input fields', async ({
|
||||
@@ -35,9 +35,7 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
await textBox.fill('q')
|
||||
await textBox.press('Control+k')
|
||||
await expect(textBox).toHaveValue('q')
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.TestCommand))
|
||||
.toBe(true)
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(true)
|
||||
})
|
||||
|
||||
test('Should not trigger keybinding reserved by text input when typing in input fields', async ({
|
||||
@@ -51,8 +49,8 @@ test.describe('Keybindings', { tag: '@keyboard' }, () => {
|
||||
await textBox.click()
|
||||
await textBox.press('Control+v')
|
||||
await expect(textBox).toBeFocused()
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.TestCommand))
|
||||
.toBe(undefined)
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(
|
||||
undefined
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
import { expect } from '@playwright/test'
|
||||
import type { Locator } from '@playwright/test'
|
||||
|
||||
import { TestIds } from '@e2e/fixtures/selectors'
|
||||
|
||||
export class Load3DHelper {
|
||||
constructor(readonly node: Locator) {}
|
||||
|
||||
@@ -22,10 +19,6 @@ export class Load3DHelper {
|
||||
return this.node.locator('input[type="color"]')
|
||||
}
|
||||
|
||||
get openViewerButton(): Locator {
|
||||
return this.node.getByRole('button', { name: /open in 3d viewer/i })
|
||||
}
|
||||
|
||||
getUploadButton(label: string): Locator {
|
||||
return this.node.getByText(label)
|
||||
}
|
||||
@@ -44,10 +37,4 @@ export class Load3DHelper {
|
||||
el.dispatchEvent(new Event('input', { bubbles: true }))
|
||||
}, hex)
|
||||
}
|
||||
|
||||
async waitForModelLoaded(): Promise<void> {
|
||||
await expect(this.node.getByTestId(TestIds.loading.overlay)).toBeHidden({
|
||||
timeout: 30000
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
export class Load3DViewerHelper {
|
||||
readonly dialog: Locator
|
||||
|
||||
constructor(readonly page: Page) {
|
||||
this.dialog = page.locator('[aria-labelledby="global-load3d-viewer"]')
|
||||
}
|
||||
|
||||
get canvas(): Locator {
|
||||
return this.dialog.locator('canvas')
|
||||
}
|
||||
|
||||
get sidebar(): Locator {
|
||||
return this.dialog.getByTestId('load3d-viewer-sidebar')
|
||||
}
|
||||
|
||||
get cancelButton(): Locator {
|
||||
return this.dialog.getByRole('button', { name: /cancel/i })
|
||||
}
|
||||
|
||||
async waitForOpen(): Promise<void> {
|
||||
await expect(this.dialog).toBeVisible({ timeout: 10000 })
|
||||
}
|
||||
|
||||
async waitForClosed(): Promise<void> {
|
||||
await expect(this.dialog).toBeHidden({ timeout: 5000 })
|
||||
}
|
||||
}
|
||||
@@ -1,29 +1,31 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { assetPath } from '@e2e/fixtures/utils/paths'
|
||||
import { load3dTest as test } from '@e2e/fixtures/helpers/Load3DFixtures'
|
||||
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
|
||||
import { Load3DHelper } from './Load3DHelper'
|
||||
|
||||
test.describe('Load3D', () => {
|
||||
let load3d: Load3DHelper
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
await comfyPage.workflow.loadWorkflow('3d/load3d_node')
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeLocator('1')
|
||||
load3d = new Load3DHelper(node)
|
||||
})
|
||||
|
||||
test(
|
||||
'Renders canvas with upload buttons and controls menu',
|
||||
{ tag: ['@smoke', '@screenshot'] },
|
||||
async ({ load3d }) => {
|
||||
async () => {
|
||||
await expect(load3d.node).toBeVisible()
|
||||
|
||||
await expect(load3d.canvas).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const b = await load3d.canvas.boundingBox()
|
||||
return b?.width ?? 0
|
||||
})
|
||||
.toBeGreaterThan(0)
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const b = await load3d.canvas.boundingBox()
|
||||
return b?.height ?? 0
|
||||
})
|
||||
.toBeGreaterThan(0)
|
||||
const canvasBox = await load3d.canvas.boundingBox()
|
||||
expect(canvasBox!.width).toBeGreaterThan(0)
|
||||
expect(canvasBox!.height).toBeGreaterThan(0)
|
||||
|
||||
await expect(load3d.getUploadButton('upload 3d model')).toBeVisible()
|
||||
await expect(
|
||||
@@ -42,7 +44,7 @@ test.describe('Load3D', () => {
|
||||
test(
|
||||
'Controls menu opens and shows all categories',
|
||||
{ tag: ['@smoke', '@screenshot'] },
|
||||
async ({ load3d }) => {
|
||||
async () => {
|
||||
await load3d.openMenu()
|
||||
|
||||
await expect(load3d.getMenuCategory('Scene')).toBeVisible()
|
||||
@@ -61,7 +63,7 @@ test.describe('Load3D', () => {
|
||||
test(
|
||||
'Changing background color updates the scene',
|
||||
{ tag: ['@smoke', '@screenshot'] },
|
||||
async ({ comfyPage, load3d }) => {
|
||||
async ({ comfyPage }) => {
|
||||
await load3d.setBackgroundColor('#cc3333')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
@@ -88,72 +90,8 @@ test.describe('Load3D', () => {
|
||||
test(
|
||||
'Recording controls are visible for Load3D',
|
||||
{ tag: '@smoke' },
|
||||
async ({ load3d }) => {
|
||||
async () => {
|
||||
await expect(load3d.recordingButton).toBeVisible()
|
||||
}
|
||||
)
|
||||
|
||||
test(
|
||||
'Uploads a 3D model via button and renders it',
|
||||
{ tag: ['@screenshot'] },
|
||||
async ({ comfyPage, load3d }) => {
|
||||
const uploadResponsePromise = comfyPage.page.waitForResponse(
|
||||
(resp) => resp.url().includes('/upload/') && resp.status() === 200,
|
||||
{ timeout: 15000 }
|
||||
)
|
||||
|
||||
const fileChooserPromise = comfyPage.page.waitForEvent('filechooser')
|
||||
await load3d.getUploadButton('upload 3d model').click()
|
||||
const fileChooser = await fileChooserPromise
|
||||
await fileChooser.setFiles(assetPath('cube.obj'))
|
||||
|
||||
await uploadResponsePromise
|
||||
|
||||
const node = await comfyPage.nodeOps.getNodeRefById(1)
|
||||
const modelFileWidget = await node.getWidget(0)
|
||||
await expect
|
||||
.poll(() => modelFileWidget.getValue(), { timeout: 5000 })
|
||||
.toContain('cube.obj')
|
||||
|
||||
await load3d.waitForModelLoaded()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(load3d.node).toHaveScreenshot(
|
||||
'load3d-uploaded-cube-obj.png',
|
||||
{ maxDiffPixelRatio: 0.1 }
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
test(
|
||||
'Drag-and-drops a 3D model onto the canvas',
|
||||
{ tag: ['@screenshot'] },
|
||||
async ({ comfyPage, load3d }) => {
|
||||
const canvasBox = await load3d.canvas.boundingBox()
|
||||
expect(canvasBox, 'Canvas bounding box should exist').not.toBeNull()
|
||||
const dropPosition = {
|
||||
x: canvasBox!.x + canvasBox!.width / 2,
|
||||
y: canvasBox!.y + canvasBox!.height / 2
|
||||
}
|
||||
|
||||
await comfyPage.dragDrop.dragAndDropFile('cube.obj', {
|
||||
dropPosition,
|
||||
waitForUpload: true
|
||||
})
|
||||
|
||||
const node = await comfyPage.nodeOps.getNodeRefById(1)
|
||||
const modelFileWidget = await node.getWidget(0)
|
||||
await expect
|
||||
.poll(() => modelFileWidget.getValue(), { timeout: 5000 })
|
||||
.toContain('cube.obj')
|
||||
|
||||
await load3d.waitForModelLoaded()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(load3d.node).toHaveScreenshot(
|
||||
'load3d-dropped-cube-obj.png',
|
||||
{ maxDiffPixelRatio: 0.1 }
|
||||
)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 35 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 36 KiB |
@@ -1,56 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { assetPath } from '@e2e/fixtures/utils/paths'
|
||||
import { load3dViewerTest as test } from '@e2e/fixtures/helpers/Load3DFixtures'
|
||||
|
||||
test.describe('Load3D Viewer', () => {
|
||||
test.beforeEach(async ({ comfyPage, load3d }) => {
|
||||
// Upload cube.obj so the node has a model loaded
|
||||
const uploadResponsePromise = comfyPage.page.waitForResponse(
|
||||
(resp) => resp.url().includes('/upload/') && resp.status() === 200,
|
||||
{ timeout: 15000 }
|
||||
)
|
||||
const fileChooserPromise = comfyPage.page.waitForEvent('filechooser')
|
||||
await load3d.getUploadButton('upload 3d model').click()
|
||||
const fileChooser = await fileChooserPromise
|
||||
await fileChooser.setFiles(assetPath('cube.obj'))
|
||||
await uploadResponsePromise
|
||||
|
||||
const nodeRef = await comfyPage.nodeOps.getNodeRefById(1)
|
||||
const modelFileWidget = await nodeRef.getWidget(0)
|
||||
await expect
|
||||
.poll(() => modelFileWidget.getValue(), { timeout: 5000 })
|
||||
.toContain('cube.obj')
|
||||
|
||||
await load3d.waitForModelLoaded()
|
||||
})
|
||||
|
||||
test(
|
||||
'Opens viewer dialog with canvas and controls sidebar',
|
||||
{ tag: '@smoke' },
|
||||
async ({ load3d, viewer }) => {
|
||||
await load3d.openViewerButton.click()
|
||||
await viewer.waitForOpen()
|
||||
|
||||
await expect(viewer.canvas).toBeVisible()
|
||||
const canvasBox = await viewer.canvas.boundingBox()
|
||||
expect(canvasBox!.width).toBeGreaterThan(0)
|
||||
expect(canvasBox!.height).toBeGreaterThan(0)
|
||||
|
||||
await expect(viewer.sidebar).toBeVisible()
|
||||
await expect(viewer.cancelButton).toBeVisible()
|
||||
}
|
||||
)
|
||||
|
||||
test(
|
||||
'Cancel button closes the viewer dialog',
|
||||
{ tag: '@smoke' },
|
||||
async ({ load3d, viewer }) => {
|
||||
await load3d.openViewerButton.click()
|
||||
await viewer.waitForOpen()
|
||||
|
||||
await viewer.cancelButton.click()
|
||||
await viewer.waitForClosed()
|
||||
}
|
||||
)
|
||||
})
|
||||
@@ -60,17 +60,11 @@ test.describe(
|
||||
test(`Load workflow from URL ${url} (drop from different browser tabs)`, async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const initialNodeCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
|
||||
await comfyPage.dragDrop.dragAndDropURL(url)
|
||||
|
||||
// The drop triggers an async fetch → parse → loadGraphData chain.
|
||||
// Poll until the graph settles with the loaded workflow's nodes.
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 15000
|
||||
})
|
||||
.toBeGreaterThan(initialNodeCount)
|
||||
const readableName = url.split('/').pop()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
`dropped_workflow_url_${readableName}.png`
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -90,12 +84,12 @@ test.describe(
|
||||
const initialNodeCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
|
||||
const node = comfyPage.vueNodes.getNodeByTitle('KSampler')
|
||||
await expect.poll(() => node.boundingBox()).toBeTruthy()
|
||||
const box = (await node.boundingBox())!
|
||||
const box = await node.boundingBox()
|
||||
expect(box).not.toBeNull()
|
||||
|
||||
const dropPosition = {
|
||||
x: box.x + box.width / 2,
|
||||
y: box.y + box.height / 2
|
||||
x: box!.x + box!.width / 2,
|
||||
y: box!.y + box!.height / 2
|
||||
}
|
||||
|
||||
await comfyPage.dragDrop.dragAndDropURL(fakeUrl, {
|
||||
@@ -109,9 +103,8 @@ test.describe(
|
||||
{ timeout: 10000 }
|
||||
)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.not.toBe(initialNodeCount)
|
||||
const newNodeCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(newNodeCount).not.toBe(initialNodeCount)
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 135 KiB |
@@ -13,14 +13,6 @@ test.describe('LOD Threshold', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
// Load a workflow with some nodes to render
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
|
||||
// Should start at normal zoom (not low quality)
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.low_quality))
|
||||
.toBe(false)
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeCloseTo(1, 1)
|
||||
|
||||
// Get initial LOD state and settings
|
||||
const initialState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
@@ -31,55 +23,62 @@ test.describe('LOD Threshold', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Should start at normal zoom (not low quality)
|
||||
expect(initialState.lowQuality).toBe(false)
|
||||
expect(initialState.scale).toBeCloseTo(1, 1)
|
||||
|
||||
// Calculate expected threshold (8px / 14px ≈ 0.571)
|
||||
const expectedThreshold = initialState.minFontSize / 14
|
||||
// Can't access private _lowQualityZoomThreshold directly
|
||||
|
||||
// Zoom out just above threshold (should still be high quality)
|
||||
await comfyPage.canvasOps.zoom(120, 5) // Zoom out 5 steps
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return { lowQuality: canvas.low_quality, scale: canvas.ds.scale }
|
||||
})
|
||||
)
|
||||
.toMatchObject({ lowQuality: false })
|
||||
const aboveThresholdState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
|
||||
// If still above threshold, should be high quality
|
||||
if (aboveThresholdState.scale > expectedThreshold) {
|
||||
expect(aboveThresholdState.lowQuality).toBe(false)
|
||||
}
|
||||
|
||||
// Zoom out more to trigger LOD (below threshold)
|
||||
await comfyPage.canvasOps.zoom(120, 5) // Zoom out 5 more steps
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return { lowQuality: canvas.low_quality, scale: canvas.ds.scale }
|
||||
})
|
||||
)
|
||||
.toMatchObject({ lowQuality: true })
|
||||
// Check that LOD is now active
|
||||
const zoomedOutState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeLessThan(expectedThreshold)
|
||||
expect(zoomedOutState.scale).toBeLessThan(expectedThreshold)
|
||||
expect(zoomedOutState.lowQuality).toBe(true)
|
||||
|
||||
// Zoom back in to disable LOD (above threshold)
|
||||
await comfyPage.canvasOps.zoom(-120, 15) // Zoom in 15 steps
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return { lowQuality: canvas.low_quality, scale: canvas.ds.scale }
|
||||
})
|
||||
)
|
||||
.toMatchObject({ lowQuality: false })
|
||||
// Check that LOD is now inactive
|
||||
const zoomedInState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeGreaterThan(expectedThreshold)
|
||||
expect(zoomedInState.scale).toBeGreaterThan(expectedThreshold)
|
||||
expect(zoomedInState.lowQuality).toBe(false)
|
||||
})
|
||||
|
||||
test('Should update threshold when font size setting changes', async ({
|
||||
@@ -94,28 +93,36 @@ test.describe('LOD Threshold', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
)
|
||||
|
||||
// Check that font size updated
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.canvas.min_font_size_for_lod)
|
||||
)
|
||||
.toBe(14)
|
||||
const newState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
minFontSize: canvas.min_font_size_for_lod
|
||||
}
|
||||
})
|
||||
|
||||
expect(newState.minFontSize).toBe(14)
|
||||
// Expected threshold would be 14px / 14px = 1.0
|
||||
|
||||
// At default zoom, LOD should still be inactive (scale is exactly 1.0, not less than)
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.low_quality))
|
||||
.toBe(false)
|
||||
const lodState = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.canvas.low_quality
|
||||
})
|
||||
expect(lodState).toBe(false)
|
||||
|
||||
// Zoom out slightly to trigger LOD
|
||||
await comfyPage.canvasOps.zoom(120, 1) // Zoom out 1 step
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.low_quality))
|
||||
.toBe(true)
|
||||
const afterZoom = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeLessThan(1.0)
|
||||
expect(afterZoom.scale).toBeLessThan(1.0)
|
||||
expect(afterZoom.lowQuality).toBe(true)
|
||||
})
|
||||
|
||||
test('Should disable LOD when font size is set to 0', async ({
|
||||
@@ -131,19 +138,18 @@ test.describe('LOD Threshold', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// LOD should remain disabled even at very low zoom
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.canvas.min_font_size_for_lod)
|
||||
)
|
||||
.toBe(0)
|
||||
const state = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale,
|
||||
minFontSize: canvas.min_font_size_for_lod
|
||||
}
|
||||
})
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.low_quality))
|
||||
.toBe(false)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeLessThan(0.2) // Very zoomed out
|
||||
expect(state.minFontSize).toBe(0) // LOD disabled
|
||||
expect(state.lowQuality).toBe(false)
|
||||
expect(state.scale).toBeLessThan(0.2) // Very zoomed out
|
||||
})
|
||||
|
||||
test(
|
||||
@@ -163,39 +169,41 @@ test.describe('LOD Threshold', { tag: ['@screenshot', '@canvas'] }, () => {
|
||||
}, targetZoom)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Wait for LOD to activate before taking screenshot
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.canvas.low_quality)
|
||||
)
|
||||
.toBe(true)
|
||||
|
||||
// Take snapshot with LOD active (default 8px setting)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'lod-comparison-low-quality.png'
|
||||
)
|
||||
|
||||
const lowQualityState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
expect(lowQualityState.lowQuality).toBe(true)
|
||||
|
||||
// Disable LOD to see high quality at same zoom
|
||||
await comfyPage.settings.setSetting(
|
||||
'LiteGraph.Canvas.MinFontSizeForLOD',
|
||||
0
|
||||
)
|
||||
|
||||
// Wait for LOD to deactivate after setting change
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.canvas.low_quality)
|
||||
)
|
||||
.toBe(false)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Take snapshot with LOD disabled (full quality at same zoom)
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'lod-comparison-high-quality.png'
|
||||
)
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.page.evaluate(() => window.app!.canvas.ds.scale))
|
||||
.toBeCloseTo(targetZoom, 2)
|
||||
const highQualityState = await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
return {
|
||||
lowQuality: canvas.low_quality,
|
||||
scale: canvas.ds.scale
|
||||
}
|
||||
})
|
||||
expect(highQualityState.lowQuality).toBe(false)
|
||||
expect(highQualityState.scale).toBeCloseTo(targetZoom, 2)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
@@ -22,7 +22,10 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
}
|
||||
})
|
||||
})
|
||||
await expect(comfyPage.menu.buttons).toHaveCount(initialChildrenCount + 1)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const newChildrenCount = await comfyPage.menu.buttons.count()
|
||||
expect(newChildrenCount).toBe(initialChildrenCount + 1)
|
||||
})
|
||||
|
||||
test.describe('Workflows topbar tabs', () => {
|
||||
@@ -35,17 +38,15 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
test('Can show opened workflows', async ({ comfyPage }) => {
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.topbar.getTabNames())
|
||||
.toEqual(['Unsaved Workflow'])
|
||||
expect(await comfyPage.menu.topbar.getTabNames()).toEqual([
|
||||
'Unsaved Workflow'
|
||||
])
|
||||
})
|
||||
|
||||
test('Can close saved-workflow tabs', async ({ comfyPage }) => {
|
||||
const workflowName = `tempWorkflow-${test.info().title}`
|
||||
await comfyPage.menu.topbar.saveWorkflow(workflowName)
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.topbar.getTabNames())
|
||||
.toEqual([workflowName])
|
||||
expect(await comfyPage.menu.topbar.getTabNames()).toEqual([workflowName])
|
||||
await comfyPage.menu.topbar.closeWorkflowTab(workflowName)
|
||||
await expect
|
||||
.poll(() => comfyPage.menu.topbar.getTabNames())
|
||||
@@ -61,11 +62,10 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
const topLevelMenuItem = comfyPage.page
|
||||
.locator('a.p-menubar-item-link')
|
||||
.first()
|
||||
await expect
|
||||
.poll(() =>
|
||||
topLevelMenuItem.evaluate((el) => el.scrollWidth > el.clientWidth)
|
||||
)
|
||||
.toBe(false)
|
||||
const isTextCutoff = await topLevelMenuItem.evaluate((el) => {
|
||||
return el.scrollWidth > el.clientWidth
|
||||
})
|
||||
expect(isTextCutoff).toBe(false)
|
||||
})
|
||||
|
||||
test('Clicking on active state items does not close menu', async ({
|
||||
@@ -148,7 +148,7 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
const exportTag = comfyPage.page.locator('.keybinding-tag', {
|
||||
hasText: 'Ctrl + s'
|
||||
})
|
||||
await expect(exportTag).toHaveCount(1)
|
||||
expect(await exportTag.count()).toBe(1)
|
||||
})
|
||||
|
||||
test('Can catch error when executing command', async ({ comfyPage }) => {
|
||||
@@ -173,7 +173,7 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
})
|
||||
})
|
||||
await comfyPage.menu.topbar.triggerTopbarCommand(['ext', 'foo-command'])
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(1)
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(1)
|
||||
})
|
||||
|
||||
test('Can navigate Theme menu and switch between Dark and Light themes', async ({
|
||||
@@ -205,9 +205,7 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
await expect(async () => {
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(themeSubmenu).toBeVisible()
|
||||
await expect(lightThemeItem.locator('.pi-check')).not.toHaveClass(
|
||||
/invisible/
|
||||
)
|
||||
expect(await topbar.isMenuItemActive(lightThemeItem)).toBe(true)
|
||||
}).toPass({ timeout: 5000 })
|
||||
|
||||
// Screenshot with light theme active
|
||||
@@ -232,11 +230,9 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
await expect(async () => {
|
||||
await expect(menu).toBeVisible()
|
||||
await expect(themeItems2.submenu).toBeVisible()
|
||||
await expect(
|
||||
themeItems2.darkTheme.locator('.pi-check')
|
||||
).not.toHaveClass(/invisible/)
|
||||
await expect(themeItems2.lightTheme.locator('.pi-check')).toHaveClass(
|
||||
/invisible/
|
||||
expect(await topbar.isMenuItemActive(themeItems2.darkTheme)).toBe(true)
|
||||
expect(await topbar.isMenuItemActive(themeItems2.lightTheme)).toBe(
|
||||
false
|
||||
)
|
||||
}).toPass({ timeout: 5000 })
|
||||
|
||||
@@ -260,9 +256,9 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', position)
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.UseNewMenu'))
|
||||
.toBe('Top')
|
||||
expect(await comfyPage.settings.getSetting('Comfy.UseNewMenu')).toBe(
|
||||
'Top'
|
||||
)
|
||||
})
|
||||
|
||||
test(`Can migrate deprecated menu positions on initial load (${position})`, async ({
|
||||
@@ -270,9 +266,9 @@ test.describe('Menu', { tag: '@ui' }, () => {
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', position)
|
||||
await comfyPage.setup()
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.UseNewMenu'))
|
||||
.toBe('Top')
|
||||
expect(await comfyPage.settings.getSetting('Comfy.UseNewMenu')).toBe(
|
||||
'Top'
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -53,9 +53,13 @@ test.describe('Minimap', { tag: '@canvas' }, () => {
|
||||
await expect(minimapContainer).toBeVisible()
|
||||
|
||||
await toggleButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(minimapContainer).not.toBeVisible()
|
||||
|
||||
await toggleButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(minimapContainer).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -65,9 +69,13 @@ test.describe('Minimap', { tag: '@canvas' }, () => {
|
||||
await expect(minimapContainer).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(minimapContainer).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(minimapContainer).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -100,7 +108,6 @@ test.describe('Minimap', { tag: '@canvas' }, () => {
|
||||
canvas.ds.offset[1] = -600
|
||||
canvas.setDirty(true, true)
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect(minimap).toHaveScreenshot('minimap-after-pan.png')
|
||||
}
|
||||
)
|
||||
@@ -115,18 +122,20 @@ test.describe('Minimap', { tag: '@canvas' }, () => {
|
||||
const viewport = minimap.locator('.minimap-viewport')
|
||||
await expect(viewport).toBeVisible()
|
||||
|
||||
await expect(async () => {
|
||||
const vb = await viewport.boundingBox()
|
||||
const mb = await minimap.boundingBox()
|
||||
expect(vb).toBeTruthy()
|
||||
expect(mb).toBeTruthy()
|
||||
expect(vb!.width).toBeGreaterThan(0)
|
||||
expect(vb!.height).toBeGreaterThan(0)
|
||||
expect(vb!.x).toBeGreaterThanOrEqual(mb!.x)
|
||||
expect(vb!.y).toBeGreaterThanOrEqual(mb!.y)
|
||||
expect(vb!.x + vb!.width).toBeLessThanOrEqual(mb!.x + mb!.width)
|
||||
expect(vb!.y + vb!.height).toBeLessThanOrEqual(mb!.y + mb!.height)
|
||||
}).toPass({ timeout: 5000 })
|
||||
const minimapBox = await minimap.boundingBox()
|
||||
const viewportBox = await viewport.boundingBox()
|
||||
|
||||
expect(minimapBox).toBeTruthy()
|
||||
expect(viewportBox).toBeTruthy()
|
||||
expect(viewportBox!.width).toBeGreaterThan(0)
|
||||
expect(viewportBox!.height).toBeGreaterThan(0)
|
||||
|
||||
expect(viewportBox!.x + viewportBox!.width).toBeGreaterThan(minimapBox!.x)
|
||||
expect(viewportBox!.y + viewportBox!.height).toBeGreaterThan(
|
||||
minimapBox!.y
|
||||
)
|
||||
expect(viewportBox!.x).toBeLessThan(minimapBox!.x + minimapBox!.width)
|
||||
expect(viewportBox!.y).toBeLessThan(minimapBox!.y + minimapBox!.height)
|
||||
|
||||
await expect(minimap).toHaveScreenshot('minimap-with-viewport.png')
|
||||
}
|
||||
|
||||
@@ -10,7 +10,9 @@ test.describe(
|
||||
test('@mobile empty canvas', async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.ConfirmClear', false)
|
||||
await comfyPage.command.executeCommand('Comfy.ClearWorkflow')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
await expect(async () => {
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('mobile-empty-canvas.png')
|
||||
})
|
||||
|
||||
@@ -82,13 +82,7 @@ test.describe(
|
||||
'Comfy.NodeBadge.NodeIdBadgeMode',
|
||||
mode
|
||||
)
|
||||
await expect
|
||||
.poll(
|
||||
() =>
|
||||
comfyPage.settings.getSetting('Comfy.NodeBadge.NodeIdBadgeMode'),
|
||||
{ message: 'NodeIdBadgeMode setting should be applied' }
|
||||
)
|
||||
.toBe(mode)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.canvasOps.resetView()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
`node-badge-${mode}.png`
|
||||
@@ -110,11 +104,8 @@ test.describe(
|
||||
NodeBadgeMode.ShowAll
|
||||
)
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'unknown')
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.ColorPalette'), {
|
||||
message: 'ColorPalette setting should be applied'
|
||||
})
|
||||
.toBe('unknown')
|
||||
await comfyPage.nextFrame()
|
||||
// Click empty space to trigger canvas re-render.
|
||||
await comfyPage.canvasOps.clickEmptySpace()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'node-badge-unknown-color-palette.png'
|
||||
@@ -129,11 +120,8 @@ test.describe(
|
||||
NodeBadgeMode.ShowAll
|
||||
)
|
||||
await comfyPage.settings.setSetting('Comfy.ColorPalette', 'light')
|
||||
await expect
|
||||
.poll(() => comfyPage.settings.getSetting('Comfy.ColorPalette'), {
|
||||
message: 'ColorPalette setting should be applied'
|
||||
})
|
||||
.toBe('light')
|
||||
await comfyPage.nextFrame()
|
||||
// Click empty space to trigger canvas re-render.
|
||||
await comfyPage.canvasOps.clickEmptySpace()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'node-badge-light-color-palette.png'
|
||||
|
||||
@@ -1,105 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
|
||||
|
||||
test.describe(
|
||||
'Node context menu viewport overflow (#10824)',
|
||||
{ tag: '@ui' },
|
||||
() => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
// Keep the viewport well below the menu content height so overflow is guaranteed.
|
||||
await comfyPage.page.setViewportSize({ width: 1280, height: 520 })
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Disabled')
|
||||
await comfyPage.settings.setSetting('Comfy.Canvas.SelectionToolbox', true)
|
||||
await comfyPage.workflow.loadWorkflow('nodes/single_ksampler')
|
||||
await comfyPage.nextFrame()
|
||||
})
|
||||
|
||||
async function openMoreOptions(comfyPage: ComfyPage) {
|
||||
const ksamplerNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('KSampler')
|
||||
if (ksamplerNodes.length === 0) {
|
||||
throw new Error('No KSampler nodes found')
|
||||
}
|
||||
|
||||
// Drag the KSampler toward the lower-left so the menu has limited space below it.
|
||||
const nodePos = await ksamplerNodes[0].getPosition()
|
||||
const viewportSize = comfyPage.page.viewportSize()!
|
||||
const centerX = viewportSize.width / 3
|
||||
const centerY = viewportSize.height * 0.75
|
||||
await comfyPage.canvasOps.dragAndDrop(
|
||||
{ x: nodePos.x, y: nodePos.y },
|
||||
{ x: centerX, y: centerY }
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await ksamplerNodes[0].click('title')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.page.locator('.selection-toolbox')).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
const moreOptionsBtn = comfyPage.page.locator(
|
||||
'[data-testid="more-options-button"]'
|
||||
)
|
||||
await expect(moreOptionsBtn).toBeVisible({ timeout: 3000 })
|
||||
await moreOptionsBtn.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const menu = comfyPage.page.locator('.p-contextmenu')
|
||||
await expect(menu).toBeVisible({ timeout: 3000 })
|
||||
|
||||
// Wait for constrainMenuHeight (runs via requestAnimationFrame in onMenuShow)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
return menu
|
||||
}
|
||||
|
||||
test('last menu item "Remove" is reachable via scroll', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const menu = await openMoreOptions(comfyPage)
|
||||
const rootList = menu.locator(':scope > ul')
|
||||
|
||||
await expect
|
||||
.poll(
|
||||
() => rootList.evaluate((el) => el.scrollHeight > el.clientHeight),
|
||||
{
|
||||
message:
|
||||
'Menu should overflow vertically so this test exercises the viewport clamp',
|
||||
timeout: 3000
|
||||
}
|
||||
)
|
||||
.toBe(true)
|
||||
|
||||
// "Remove" is the last item in the More Options menu.
|
||||
// It must become reachable by scrolling the bounded menu list.
|
||||
const removeItem = menu.getByText('Remove', { exact: true })
|
||||
const didScroll = await rootList.evaluate((el) => {
|
||||
const previousScrollTop = el.scrollTop
|
||||
el.scrollTo({ top: el.scrollHeight })
|
||||
return el.scrollTop > previousScrollTop
|
||||
})
|
||||
expect(didScroll).toBe(true)
|
||||
await expect(removeItem).toBeVisible()
|
||||
})
|
||||
|
||||
test('last menu item "Remove" is clickable and removes the node', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const menu = await openMoreOptions(comfyPage)
|
||||
|
||||
const removeItem = menu.getByText('Remove', { exact: true })
|
||||
await removeItem.scrollIntoViewIfNeeded()
|
||||
await removeItem.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// The node should be removed from the graph
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), { timeout: 3000 })
|
||||
.toBe(0)
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -37,7 +37,7 @@ test.describe('Optional input', { tag: ['@screenshot', '@node'] }, () => {
|
||||
|
||||
test('Only optional inputs', async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('inputs/only_optional_inputs')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
|
||||
await expect(
|
||||
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
|
||||
).not.toBeVisible()
|
||||
|
||||
@@ -433,7 +433,8 @@ This is documentation for a custom node.
|
||||
const imageCount = await images.count()
|
||||
for (let i = 0; i < imageCount; i++) {
|
||||
const img = images.nth(i)
|
||||
await expect(img).not.toHaveAttribute('onerror')
|
||||
const onError = await img.getAttribute('onerror')
|
||||
expect(onError).toBeNull()
|
||||
}
|
||||
|
||||
// Check that javascript: links are sanitized
|
||||
@@ -441,7 +442,10 @@ This is documentation for a custom node.
|
||||
const linkCount = await links.count()
|
||||
for (let i = 0; i < linkCount; i++) {
|
||||
const link = links.nth(i)
|
||||
await expect(link).not.toHaveAttribute('href', /^javascript:/i)
|
||||
const href = await link.getAttribute('href')
|
||||
if (href !== null) {
|
||||
expect(href).not.toContain('javascript:')
|
||||
}
|
||||
}
|
||||
|
||||
// Safe content should remain
|
||||
@@ -508,7 +512,8 @@ This is English documentation.
|
||||
await expect(helpPage.locator('.p-progressspinner')).not.toBeVisible()
|
||||
|
||||
// Should show some content even on error
|
||||
await expect(helpPage).not.toHaveText('')
|
||||
const content = await helpPage.textContent()
|
||||
expect(content).toBeTruthy()
|
||||
})
|
||||
|
||||
test('Should update help content when switching between nodes', async ({
|
||||
|
||||
@@ -82,7 +82,9 @@ test.describe('Node Library Essentials Tab', { tag: '@ui' }, () => {
|
||||
const firstCard = comfyPage.page.locator('[data-node-name]').first()
|
||||
await expect(firstCard).toBeVisible()
|
||||
|
||||
await expect(firstCard).toHaveAttribute('data-node-name', /.+/)
|
||||
const nodeName = await firstCard.getAttribute('data-node-name')
|
||||
expect(nodeName).toBeTruthy()
|
||||
expect(nodeName!.length).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
test('Node library can switch between all and essentials tabs', async ({
|
||||
|
||||
@@ -31,9 +31,8 @@ test.describe('Node search box V2', { tag: '@node' }, () => {
|
||||
await comfyPage.page.keyboard.press('Enter')
|
||||
await expect(searchBoxV2.input).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialCount + 1)
|
||||
const newCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(newCount).toBe(initialCount + 1)
|
||||
})
|
||||
|
||||
test('Can add first default result with Enter', async ({ comfyPage }) => {
|
||||
@@ -50,9 +49,8 @@ test.describe('Node search box V2', { tag: '@node' }, () => {
|
||||
await comfyPage.page.keyboard.press('Enter')
|
||||
await expect(searchBoxV2.input).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialCount + 1)
|
||||
const newCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(newCount).toBe(initialCount + 1)
|
||||
})
|
||||
|
||||
test.describe('Category navigation', () => {
|
||||
@@ -83,7 +81,8 @@ test.describe('Node search box V2', { tag: '@node' }, () => {
|
||||
await searchBoxV2.categoryButton('sampling').click()
|
||||
|
||||
await expect(searchBoxV2.results.first()).toBeVisible()
|
||||
await expect.poll(() => searchBoxV2.results.count()).toBeGreaterThan(0)
|
||||
const count = await searchBoxV2.results.count()
|
||||
expect(count).toBeGreaterThan(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -143,9 +142,8 @@ test.describe('Node search box V2', { tag: '@node' }, () => {
|
||||
await comfyPage.page.keyboard.press('Enter')
|
||||
await expect(searchBoxV2.input).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialCount + 1)
|
||||
const newCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(newCount).toBe(initialCount + 1)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -41,9 +41,8 @@ test.describe('Node search box V2 extended', { tag: '@node' }, () => {
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await expect(searchBoxV2.input).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(initialCount)
|
||||
const newCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(newCount).toBe(initialCount)
|
||||
})
|
||||
|
||||
test('Search clears when reopening', async ({ comfyPage }) => {
|
||||
@@ -76,10 +75,9 @@ test.describe('Node search box V2 extended', { tag: '@node' }, () => {
|
||||
|
||||
await searchBoxV2.categoryButton('loaders').click()
|
||||
await expect(searchBoxV2.results.first()).toBeVisible()
|
||||
const loaderResults = await searchBoxV2.results.allTextContents()
|
||||
|
||||
await expect
|
||||
.poll(() => searchBoxV2.results.allTextContents())
|
||||
.not.toEqual(samplingResults)
|
||||
expect(samplingResults).not.toEqual(loaderResults)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -109,9 +107,8 @@ test.describe('Node search box V2 extended', { tag: '@node' }, () => {
|
||||
)
|
||||
await expect(filterChip).toBeVisible()
|
||||
await expect(searchBoxV2.results.first()).toBeVisible()
|
||||
await expect
|
||||
.poll(() => searchBoxV2.results.allTextContents())
|
||||
.not.toEqual(unfilteredResults)
|
||||
const filteredResults = await searchBoxV2.results.allTextContents()
|
||||
expect(filteredResults).not.toEqual(unfilteredResults)
|
||||
|
||||
// Remove filter by clicking the chip delete button
|
||||
await filterChip.getByTestId('chip-delete').click()
|
||||
|
||||
@@ -39,21 +39,18 @@ test.describe('Painter', () => {
|
||||
const canvas = node.locator('.widget-expands canvas')
|
||||
await expect(canvas).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(async () =>
|
||||
canvas.evaluate((el) => {
|
||||
const ctx = (el as HTMLCanvasElement).getContext('2d')
|
||||
if (!ctx) return true
|
||||
const data = ctx.getImageData(
|
||||
0,
|
||||
0,
|
||||
(el as HTMLCanvasElement).width,
|
||||
(el as HTMLCanvasElement).height
|
||||
)
|
||||
return data.data.every((v, i) => (i % 4 === 3 ? v === 0 : true))
|
||||
})
|
||||
const isEmptyBefore = await canvas.evaluate((el) => {
|
||||
const ctx = (el as HTMLCanvasElement).getContext('2d')
|
||||
if (!ctx) return true
|
||||
const data = ctx.getImageData(
|
||||
0,
|
||||
0,
|
||||
(el as HTMLCanvasElement).width,
|
||||
(el as HTMLCanvasElement).height
|
||||
)
|
||||
.toBe(true)
|
||||
return data.data.every((v, i) => (i % 4 === 3 ? v === 0 : true))
|
||||
})
|
||||
expect(isEmptyBefore).toBe(true)
|
||||
|
||||
const box = await canvas.boundingBox()
|
||||
if (!box) throw new Error('Canvas bounding box not found')
|
||||
@@ -71,24 +68,23 @@ test.describe('Painter', () => {
|
||||
await comfyPage.page.mouse.up()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(async () =>
|
||||
canvas.evaluate((el) => {
|
||||
const ctx = (el as HTMLCanvasElement).getContext('2d')
|
||||
if (!ctx) return false
|
||||
const data = ctx.getImageData(
|
||||
0,
|
||||
0,
|
||||
(el as HTMLCanvasElement).width,
|
||||
(el as HTMLCanvasElement).height
|
||||
)
|
||||
for (let i = 3; i < data.data.length; i += 4) {
|
||||
if (data.data[i] > 0) return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
)
|
||||
.toBe(true)
|
||||
await expect(async () => {
|
||||
const hasContent = await canvas.evaluate((el) => {
|
||||
const ctx = (el as HTMLCanvasElement).getContext('2d')
|
||||
if (!ctx) return false
|
||||
const data = ctx.getImageData(
|
||||
0,
|
||||
0,
|
||||
(el as HTMLCanvasElement).width,
|
||||
(el as HTMLCanvasElement).height
|
||||
)
|
||||
for (let i = 3; i < data.data.length; i += 4) {
|
||||
if (data.data[i] > 0) return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
expect(hasContent).toBe(true)
|
||||
}).toPass()
|
||||
|
||||
await expect(node).toHaveScreenshot('painter-after-stroke.png')
|
||||
}
|
||||
|
||||
@@ -327,7 +327,8 @@ test.describe('Performance', { tag: ['@perf'] }, () => {
|
||||
// Verify we actually entered the culling regime.
|
||||
// isNodeTooSmall triggers when max(width, height) * scale < 4px.
|
||||
// Typical nodes are ~200px wide, so scale must be < 0.02.
|
||||
await expect.poll(() => comfyPage.canvasOps.getScale()).toBeLessThan(0.02)
|
||||
const scale = await comfyPage.canvasOps.getScale()
|
||||
expect(scale).toBeLessThan(0.02)
|
||||
|
||||
// Idle at extreme zoom-out — most nodes should be culled
|
||||
for (let i = 0; i < 60; i++) {
|
||||
@@ -357,11 +358,9 @@ test.describe('Performance', { tag: ['@perf'] }, () => {
|
||||
|
||||
// Wait for the output widget to populate (execution_success)
|
||||
const outputNode = await comfyPage.nodeOps.getNodeRefById(1)
|
||||
await expect
|
||||
.poll(async () => (await outputNode.getWidget(0)).getValue(), {
|
||||
timeout: 10000
|
||||
})
|
||||
.toBe('foo')
|
||||
await expect(async () => {
|
||||
expect(await (await outputNode.getWidget(0)).getValue()).toBe('foo')
|
||||
}).toPass({ timeout: 10000 })
|
||||
|
||||
const m = await comfyPage.perf.stopMeasuring('workflow-execution')
|
||||
recordMeasurement(m)
|
||||
|
||||
@@ -17,6 +17,7 @@ test.describe('Errors tab - common', { tag: '@ui' }, () => {
|
||||
test('Should show Errors tab when errors exist', async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('missing/missing_nodes')
|
||||
await comfyPage.actionbar.propertiesButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const panel = new PropertiesPanelHelper(comfyPage.page)
|
||||
await expect(panel.errorsTabIcon).toBeVisible()
|
||||
@@ -31,6 +32,7 @@ test.describe('Errors tab - common', { tag: '@ui' }, () => {
|
||||
false
|
||||
)
|
||||
await comfyPage.actionbar.propertiesButton.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const panel = new PropertiesPanelHelper(comfyPage.page)
|
||||
await expect(panel.errorsTabIcon).not.toBeVisible()
|
||||
@@ -47,6 +49,7 @@ test.describe('Errors tab - common', { tag: '@ui' }, () => {
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('nodes/execution_error')
|
||||
await comfyPage.command.executeCommand('Comfy.QueuePrompt')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const errorOverlay = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.errorOverlay
|
||||
@@ -55,7 +58,6 @@ test.describe('Errors tab - common', { tag: '@ui' }, () => {
|
||||
await errorOverlay
|
||||
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
|
||||
.click()
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
|
||||
const runtimePanel = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.runtimeErrorPanel
|
||||
@@ -65,7 +67,7 @@ test.describe('Errors tab - common', { tag: '@ui' }, () => {
|
||||
const searchInput = comfyPage.page.getByPlaceholder(/^Search/)
|
||||
await searchInput.fill('nonexistent_query_xyz_12345')
|
||||
|
||||
await expect(runtimePanel).toHaveCount(0)
|
||||
await expect(runtimePanel).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -18,6 +18,7 @@ test.describe('Errors tab - Execution errors', { tag: '@ui' }, () => {
|
||||
async function openExecutionErrorTab(comfyPage: ComfyPage) {
|
||||
await comfyPage.workflow.loadWorkflow('nodes/execution_error')
|
||||
await comfyPage.command.executeCommand('Comfy.QueuePrompt')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const errorOverlay = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.errorOverlay
|
||||
|
||||
@@ -15,14 +15,11 @@ test.describe('Errors tab - Missing models', { tag: '@ui' }, () => {
|
||||
'Comfy.RightSidePanel.ShowErrorsTab',
|
||||
true
|
||||
)
|
||||
await expect
|
||||
.poll(async () => {
|
||||
return await comfyPage.page.evaluate(async (url: string) => {
|
||||
const response = await fetch(`${url}/api/devtools/cleanup_fake_model`)
|
||||
return response.ok
|
||||
}, comfyPage.url)
|
||||
})
|
||||
.toBeTruthy()
|
||||
const cleanupOk = await comfyPage.page.evaluate(async (url: string) => {
|
||||
const response = await fetch(`${url}/api/devtools/cleanup_fake_model`)
|
||||
return response.ok
|
||||
}, comfyPage.url)
|
||||
expect(cleanupOk).toBeTruthy()
|
||||
})
|
||||
|
||||
test('Should show missing models group in errors tab', async ({
|
||||
|
||||
@@ -15,6 +15,7 @@ test.describe('Properties panel position', () => {
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Sidebar.Location', 'left')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const propertiesPanel = comfyPage.page.getByTestId(
|
||||
TestIds.propertiesPanel.root
|
||||
@@ -24,24 +25,23 @@ test.describe('Properties panel position', () => {
|
||||
await expect(propertiesPanel).toBeVisible()
|
||||
await expect(sidebar).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const propsBoundingBox = await propertiesPanel.boundingBox()
|
||||
const sidebarBoundingBox = await sidebar.boundingBox()
|
||||
const propsBoundingBox = await propertiesPanel.boundingBox()
|
||||
const sidebarBoundingBox = await sidebar.boundingBox()
|
||||
|
||||
if (!propsBoundingBox || !sidebarBoundingBox) return false
|
||||
expect(propsBoundingBox).not.toBeNull()
|
||||
expect(sidebarBoundingBox).not.toBeNull()
|
||||
|
||||
return (
|
||||
propsBoundingBox.x > sidebarBoundingBox.x + sidebarBoundingBox.width
|
||||
)
|
||||
})
|
||||
.toBe(true)
|
||||
// Properties panel should be to the right of the sidebar
|
||||
expect(propsBoundingBox!.x).toBeGreaterThan(
|
||||
sidebarBoundingBox!.x + sidebarBoundingBox!.width
|
||||
)
|
||||
})
|
||||
|
||||
test('positions on the left when sidebar is on the right', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Sidebar.Location', 'right')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const propertiesPanel = comfyPage.page.getByTestId(
|
||||
TestIds.propertiesPanel.root
|
||||
@@ -51,18 +51,16 @@ test.describe('Properties panel position', () => {
|
||||
await expect(propertiesPanel).toBeVisible()
|
||||
await expect(sidebar).toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const propsBoundingBox = await propertiesPanel.boundingBox()
|
||||
const sidebarBoundingBox = await sidebar.boundingBox()
|
||||
const propsBoundingBox = await propertiesPanel.boundingBox()
|
||||
const sidebarBoundingBox = await sidebar.boundingBox()
|
||||
|
||||
if (!propsBoundingBox || !sidebarBoundingBox) return false
|
||||
expect(propsBoundingBox).not.toBeNull()
|
||||
expect(sidebarBoundingBox).not.toBeNull()
|
||||
|
||||
return (
|
||||
propsBoundingBox.x + propsBoundingBox.width < sidebarBoundingBox.x
|
||||
)
|
||||
})
|
||||
.toBe(true)
|
||||
// Properties panel should be to the left of the sidebar
|
||||
expect(propsBoundingBox!.x + propsBoundingBox!.width).toBeLessThan(
|
||||
sidebarBoundingBox!.x
|
||||
)
|
||||
})
|
||||
|
||||
test('close button icon updates based on sidebar location', async ({
|
||||
@@ -74,6 +72,7 @@ test.describe('Properties panel position', () => {
|
||||
|
||||
// When sidebar is on the left, panel is on the right
|
||||
await comfyPage.settings.setSetting('Comfy.Sidebar.Location', 'left')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(propertiesPanel).toBeVisible()
|
||||
const closeButtonLeft = propertiesPanel
|
||||
@@ -84,6 +83,7 @@ test.describe('Properties panel position', () => {
|
||||
|
||||
// When sidebar is on the right, panel is on the left
|
||||
await comfyPage.settings.setSetting('Comfy.Sidebar.Location', 'right')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const closeButtonRight = propertiesPanel
|
||||
.locator('button[aria-pressed]')
|
||||
|
||||
@@ -43,6 +43,7 @@ test.describe('Properties panel - Title editing', () => {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.canvas.deselectAll()
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect(panel.panelTitle).toContainText('Workflow Overview')
|
||||
await expect(panel.titleEditIcon).not.toBeVisible()
|
||||
})
|
||||
|
||||
@@ -30,7 +30,8 @@ test.describe('Properties panel - Workflow Overview', () => {
|
||||
comfyPage
|
||||
}) => {
|
||||
await panel.switchToTab('Nodes')
|
||||
await expect.poll(() => comfyPage.nodeOps.getNodeCount()).toBeGreaterThan(0)
|
||||
const nodeCount = await comfyPage.nodeOps.getNodeCount()
|
||||
expect(nodeCount).toBeGreaterThan(0)
|
||||
await expect(panel.contentArea.locator('text=KSampler')).toBeVisible()
|
||||
})
|
||||
|
||||
|
||||
@@ -46,6 +46,11 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
}, nodeName)
|
||||
}
|
||||
|
||||
const waitForWidgetUpdate = async (comfyPage: ComfyPage) => {
|
||||
// Force re-render to trigger first access of widget's options
|
||||
await comfyPage.page.mouse.click(400, 300)
|
||||
}
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.settings.setSetting('Comfy.NodeLibrary.NewDesign', false)
|
||||
@@ -80,9 +85,9 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
}) => {
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual(mockOptions)
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const widgetOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(widgetOptions).toEqual(mockOptions)
|
||||
})
|
||||
|
||||
test('lazy loads options when widget is added via workflow load', async ({
|
||||
@@ -91,28 +96,23 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await comfyPage.workflow.loadWorkflow('inputs/remote_widget')
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate((name) => {
|
||||
return (
|
||||
window.app!.graph!.nodes.find((node) => node.title === name) !=
|
||||
null
|
||||
)
|
||||
}, nodeName)
|
||||
)
|
||||
.toBe(true)
|
||||
const node = await comfyPage.page.evaluate((name) => {
|
||||
return window.app!.graph!.nodes.find((node) => node.title === name)
|
||||
}, nodeName)
|
||||
expect(node).toBeDefined()
|
||||
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual(mockOptions)
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const widgetOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(widgetOptions).toEqual(mockOptions)
|
||||
})
|
||||
|
||||
test('applies query parameters from input spec', async ({ comfyPage }) => {
|
||||
const nodeName = 'Remote Widget Node With Sort Query Param'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual([...mockOptions].sort())
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const widgetOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(widgetOptions).not.toEqual(mockOptions)
|
||||
expect(widgetOptions).toEqual([...mockOptions].sort())
|
||||
})
|
||||
|
||||
test('handles empty list of options', async ({ comfyPage }) => {
|
||||
@@ -125,7 +125,9 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await expect.poll(() => getWidgetOptions(comfyPage, nodeName)).toEqual([])
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const widgetOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(widgetOptions).toEqual([])
|
||||
})
|
||||
|
||||
test('falls back to default value when non-200 response', async ({
|
||||
@@ -140,9 +142,11 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual('Loading...')
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const widgetOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
|
||||
const defaultValue = 'Loading...'
|
||||
expect(widgetOptions).toEqual(defaultValue)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -181,6 +185,7 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
|
||||
// Select remote widget node
|
||||
await comfyPage.page.keyboard.press('Control+A')
|
||||
@@ -206,18 +211,18 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node With 300ms Refresh'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
// Wait for initial options to load before capturing baseline
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toBeTruthy()
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const initialOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
|
||||
// Click on the canvas to trigger widget refresh
|
||||
await comfyPage.page.mouse.click(400, 300)
|
||||
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.not.toEqual(initialOptions)
|
||||
await expect(async () => {
|
||||
const refreshedOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(refreshedOptions).not.toEqual(initialOptions)
|
||||
}).toPass({
|
||||
timeout: 2_000
|
||||
})
|
||||
})
|
||||
|
||||
test('does not refresh when TTL is not set', async ({ comfyPage }) => {
|
||||
@@ -232,10 +237,7 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
// Wait for initial fetch to complete
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual(['test'])
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
|
||||
// Force multiple re-renders
|
||||
for (let i = 0; i < 3; i++) {
|
||||
@@ -243,7 +245,7 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
await expect.poll(() => requestCount, { timeout: 1000 }).toBe(1) // Should only make initial request
|
||||
expect(requestCount).toBe(1) // Should only make initial request
|
||||
})
|
||||
|
||||
test('retries failed requests with backoff', async ({ comfyPage }) => {
|
||||
@@ -258,20 +260,17 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
|
||||
// Initial canvas click to trigger widget render
|
||||
await comfyPage.page.mouse.click(400, 300)
|
||||
|
||||
// Drive canvas redraws to let the retry scheduler fire
|
||||
// Wait for exponential backoff retries to accumulate timestamps
|
||||
await expect(async () => {
|
||||
await comfyPage.page.mouse.click(400, 300)
|
||||
await comfyPage.nextFrame()
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
expect(timestamps.length).toBeGreaterThanOrEqual(3)
|
||||
}).toPass({ timeout: 15000, intervals: [500, 1000, 1500] })
|
||||
}).toPass({ timeout: 10000, intervals: [500, 1000, 1500] })
|
||||
|
||||
// Verify backoff: last interval should exceed first
|
||||
// Verify exponential backoff between retries
|
||||
const intervals = timestamps.slice(1).map((t, i) => t - timestamps[i])
|
||||
expect(intervals[intervals.length - 1]).toBeGreaterThan(intervals[0])
|
||||
expect(intervals[1]).toBeGreaterThan(intervals[0])
|
||||
})
|
||||
|
||||
test('clicking refresh button forces a refresh', async ({ comfyPage }) => {
|
||||
@@ -289,19 +288,15 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
// Trigger initial fetch when adding node to the graph
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
// Wait for initial options to load before capturing baseline
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toBeTruthy()
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
const initialOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
|
||||
// Click refresh button
|
||||
await clickRefreshButton(comfyPage, nodeName)
|
||||
|
||||
// Verify refresh occurred
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.not.toEqual(initialOptions)
|
||||
const refreshedOptions = await getWidgetOptions(comfyPage, nodeName)
|
||||
expect(refreshedOptions).not.toEqual(initialOptions)
|
||||
})
|
||||
|
||||
test('control_after_refresh is applied after refresh', async ({
|
||||
@@ -327,18 +322,18 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
|
||||
// Trigger initial fetch when adding node to the graph
|
||||
await addRemoteWidgetNode(comfyPage, nodeName)
|
||||
// Wait for initial options to load
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toBeTruthy()
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
|
||||
// Click refresh button
|
||||
await clickRefreshButton(comfyPage, nodeName)
|
||||
|
||||
// Verify the selected value of the widget is the first option in the refreshed list
|
||||
await expect
|
||||
.poll(() => getWidgetValue(comfyPage, nodeName))
|
||||
.toEqual('new first option')
|
||||
await expect(async () => {
|
||||
const refreshedValue = await getWidgetValue(comfyPage, nodeName)
|
||||
expect(refreshedValue).toEqual('new first option')
|
||||
}).toPass({
|
||||
timeout: 2_000
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -361,12 +356,9 @@ test.describe('Remote COMBO Widget', { tag: '@widget' }, () => {
|
||||
// Add two widgets with same config
|
||||
const nodeName = 'Remote Widget Node'
|
||||
await addRemoteWidgetNode(comfyPage, nodeName, 2)
|
||||
// Wait for options to be populated before checking request count
|
||||
await expect
|
||||
.poll(() => getWidgetOptions(comfyPage, nodeName))
|
||||
.toEqual(mockOptions)
|
||||
await waitForWidgetUpdate(comfyPage)
|
||||
|
||||
await expect.poll(() => requestCount, { timeout: 1000 }).toBe(1) // Should reuse cached data
|
||||
expect(requestCount).toBe(1) // Should reuse cached data
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -16,9 +16,10 @@ test.describe(
|
||||
await comfyPage.canvasOps.rightClick()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-menu.png')
|
||||
await comfyPage.page.getByText('Add Node').click()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.getByText('loaders').click()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.getByText('Load VAE').click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('add-node-node-added.png')
|
||||
})
|
||||
@@ -27,7 +28,6 @@ test.describe(
|
||||
await comfyPage.canvasOps.rightClick()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-menu.png')
|
||||
await comfyPage.page.getByText('Add Group', { exact: true }).click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'add-group-group-added.png'
|
||||
@@ -41,7 +41,7 @@ test.describe(
|
||||
await comfyPage.contextMenu.clickMenuItem(
|
||||
'Convert to Group Node (Deprecated)'
|
||||
)
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.nodeOps.promptDialogInput.fill('GroupNode2CLIP')
|
||||
await comfyPage.page.keyboard.press('Enter')
|
||||
await comfyPage.nodeOps.promptDialogInput.waitFor({ state: 'hidden' })
|
||||
@@ -63,7 +63,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-node.png')
|
||||
await comfyPage.page.getByText('Properties Panel').click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'right-click-node-properties-panel.png'
|
||||
@@ -79,7 +78,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-node.png')
|
||||
await comfyPage.page.getByText('Collapse').click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'right-click-node-collapsed.png'
|
||||
@@ -103,7 +101,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.getByText('Collapse').click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'right-click-node-collapsed-badge.png'
|
||||
@@ -119,7 +116,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-node.png')
|
||||
await comfyPage.page.getByText('Bypass').click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'right-click-node-bypassed.png'
|
||||
@@ -136,7 +132,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('right-click-node.png')
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Pin")')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Get EmptyLatentImage node title position dynamically (for dragging)
|
||||
@@ -154,7 +149,7 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
'right-click-pinned-node.png'
|
||||
)
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Unpin")')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.canvas.click({
|
||||
position: DefaultGraphPositions.emptyLatentWidgetClick,
|
||||
button: 'right'
|
||||
@@ -174,7 +169,7 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Pin")')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.canvas.click({
|
||||
position: DefaultGraphPositions.emptyLatentWidgetClick,
|
||||
button: 'right'
|
||||
@@ -182,7 +177,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Unpin")')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Get EmptyLatentImage node title position dynamically (for dragging)
|
||||
@@ -205,7 +199,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Pin")')
|
||||
await comfyPage.page.keyboard.up('Control')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot('selected-nodes-pinned.png')
|
||||
await comfyPage.canvas.click({
|
||||
@@ -215,7 +208,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await comfyPage.page.mouse.move(10, 10)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.page.click('.litemenu-entry:has-text("Unpin")')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'selected-nodes-unpinned.png'
|
||||
@@ -226,7 +218,7 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
const nodeCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
const node = (await comfyPage.nodeOps.getFirstNodeRef())!
|
||||
await node.clickContextMenuOption('Pin')
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await comfyPage.nextFrame()
|
||||
await node.click('title', { button: 'right' })
|
||||
await expect(
|
||||
comfyPage.page.locator('.litemenu-entry:has-text("Unpin")')
|
||||
@@ -235,7 +227,8 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
'.litemenu-entry:has-text("Clone")'
|
||||
)
|
||||
await cloneItem.click()
|
||||
await comfyPage.contextMenu.waitForHidden()
|
||||
await expect(cloneItem).toHaveCount(0)
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
|
||||
.toBe(nodeCount + 1)
|
||||
|
||||
@@ -12,22 +12,24 @@ test.describe('@canvas Selection Rectangle', () => {
|
||||
})
|
||||
|
||||
test('Ctrl+A selects all nodes', async ({ comfyPage }) => {
|
||||
await expect
|
||||
.poll(() => comfyPage.vueNodes.getNodeCount())
|
||||
.toBeGreaterThan(0)
|
||||
const totalCount = await comfyPage.vueNodes.getNodeCount()
|
||||
expect(totalCount).toBeGreaterThan(0)
|
||||
|
||||
// Use canvas press for keyboard shortcuts (doesn't need click target)
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(totalCount)
|
||||
await expect
|
||||
.poll(() => comfyPage.vueNodes.getSelectedNodeCount())
|
||||
.toBe(totalCount)
|
||||
})
|
||||
|
||||
test('Click empty space deselects all', async ({ comfyPage }) => {
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.vueNodes.selectedNodes).not.toHaveCount(0)
|
||||
await expect
|
||||
.poll(() => comfyPage.vueNodes.getSelectedNodeCount())
|
||||
.toBeGreaterThan(0)
|
||||
|
||||
// Deselect by Ctrl+clicking the already-selected node (reliable cross-env)
|
||||
await comfyPage.page
|
||||
@@ -39,26 +41,26 @@ test.describe('@canvas Selection Rectangle', () => {
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(0)
|
||||
await expect.poll(() => comfyPage.vueNodes.getSelectedNodeCount()).toBe(0)
|
||||
})
|
||||
|
||||
test('Single click selects one node', async ({ comfyPage }) => {
|
||||
await comfyPage.page.getByText('Load Checkpoint').click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(1)
|
||||
await expect.poll(() => comfyPage.vueNodes.getSelectedNodeCount()).toBe(1)
|
||||
})
|
||||
|
||||
test('Ctrl+click adds to selection', async ({ comfyPage }) => {
|
||||
await comfyPage.page.getByText('Load Checkpoint').click()
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(1)
|
||||
await expect.poll(() => comfyPage.vueNodes.getSelectedNodeCount()).toBe(1)
|
||||
|
||||
await comfyPage.page.getByText('Empty Latent Image').click({
|
||||
modifiers: ['Control']
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(2)
|
||||
await expect.poll(() => comfyPage.vueNodes.getSelectedNodeCount()).toBe(2)
|
||||
})
|
||||
|
||||
test('Selected nodes have visual indicator', async ({ comfyPage }) => {
|
||||
@@ -73,17 +75,17 @@ test.describe('@canvas Selection Rectangle', () => {
|
||||
test('Drag-select rectangle selects multiple nodes', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(0)
|
||||
await expect.poll(() => comfyPage.vueNodes.getSelectedNodeCount()).toBe(0)
|
||||
|
||||
// Use Ctrl+A to select all, which is functionally equivalent to
|
||||
// drag-selecting the entire canvas and more reliable in CI
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.vueNodes.getNodeCount())
|
||||
.toBeGreaterThan(1)
|
||||
const totalCount = await comfyPage.vueNodes.getNodeCount()
|
||||
await expect(comfyPage.vueNodes.selectedNodes).toHaveCount(totalCount)
|
||||
await expect
|
||||
.poll(() => comfyPage.vueNodes.getSelectedNodeCount())
|
||||
.toBe(totalCount)
|
||||
expect(totalCount).toBeGreaterThan(1)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -59,19 +59,12 @@ test.describe('Selection Toolbox', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
await expect(toolboxContainer).toBeVisible()
|
||||
|
||||
// Verify toolbox is positioned (canvas-based positioning has different coordinates)
|
||||
await expect
|
||||
.poll(async () => await toolboxContainer.boundingBox())
|
||||
.not.toBeNull()
|
||||
const boundingBox = await toolboxContainer.boundingBox()
|
||||
expect(boundingBox).not.toBeNull()
|
||||
// Canvas-based positioning can vary, just verify toolbox appears in reasonable bounds
|
||||
await expect
|
||||
.poll(async () => (await toolboxContainer.boundingBox())?.x)
|
||||
.toBeGreaterThan(-200) // Not too far off-screen left
|
||||
await expect
|
||||
.poll(async () => (await toolboxContainer.boundingBox())?.x)
|
||||
.toBeLessThan(1000) // Not too far off-screen right
|
||||
await expect
|
||||
.poll(async () => (await toolboxContainer.boundingBox())?.y)
|
||||
.toBeGreaterThan(-100) // Not too far off-screen top
|
||||
expect(boundingBox!.x).toBeGreaterThan(-200) // Not too far off-screen left
|
||||
expect(boundingBox!.x).toBeLessThan(1000) // Not too far off-screen right
|
||||
expect(boundingBox!.y).toBeGreaterThan(-100) // Not too far off-screen top
|
||||
})
|
||||
|
||||
test('hide when select and drag happen at the same time', async ({
|
||||
@@ -176,7 +169,7 @@ test.describe('Selection Toolbox', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
const selectedNode = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('KSampler')
|
||||
)[0]
|
||||
await expect.poll(() => selectedNode.getProperty('color')).toBeDefined()
|
||||
expect(await selectedNode.getProperty('color')).not.toBeNull()
|
||||
})
|
||||
|
||||
test('color picker shows current color of selected nodes', async ({
|
||||
@@ -273,7 +266,7 @@ test.describe('Selection Toolbox', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
const selectedNode = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('KSampler')
|
||||
)[0]
|
||||
await expect.poll(() => selectedNode.getProperty('color')).toBeUndefined()
|
||||
expect(await selectedNode.getProperty('color')).toBeUndefined()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -52,11 +52,10 @@ test.describe('Selection Toolbox - Button Actions', { tag: '@ui' }, () => {
|
||||
await deleteButton.click({ force: true })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.graph!._nodes.length)
|
||||
)
|
||||
.toBe(initialCount - 1)
|
||||
const newCount = await comfyPage.page.evaluate(
|
||||
() => window.app!.graph!._nodes.length
|
||||
)
|
||||
expect(newCount).toBe(initialCount - 1)
|
||||
})
|
||||
|
||||
test('info button opens properties panel', async ({ comfyPage }) => {
|
||||
@@ -66,6 +65,8 @@ test.describe('Selection Toolbox - Button Actions', { tag: '@ui' }, () => {
|
||||
const infoButton = comfyPage.page.getByTestId('info-button')
|
||||
await expect(infoButton).toBeVisible()
|
||||
await infoButton.click({ force: true })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(comfyPage.page.getByTestId('properties-panel')).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -101,11 +102,10 @@ test.describe('Selection Toolbox - Button Actions', { tag: '@ui' }, () => {
|
||||
await deleteButton.click({ force: true })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.evaluate(() => window.app!.graph!._nodes.length)
|
||||
)
|
||||
.toBe(initialCount - 2)
|
||||
const newCount = await comfyPage.page.evaluate(
|
||||
() => window.app!.graph!._nodes.length
|
||||
)
|
||||
expect(newCount).toBe(initialCount - 2)
|
||||
})
|
||||
|
||||
test('bypass button toggles bypass on single node', async ({ comfyPage }) => {
|
||||
@@ -116,14 +116,14 @@ test.describe('Selection Toolbox - Button Actions', { tag: '@ui' }, () => {
|
||||
const nodeRef = (await comfyPage.nodeOps.getNodeRefsByTitle('KSampler'))[0]
|
||||
await selectNodeWithPan(comfyPage, nodeRef)
|
||||
|
||||
await expect.poll(() => nodeRef.isBypassed()).toBe(false)
|
||||
expect(await nodeRef.isBypassed()).toBe(false)
|
||||
|
||||
const bypassButton = comfyPage.page.getByTestId('bypass-button')
|
||||
await expect(bypassButton).toBeVisible()
|
||||
await bypassButton.click({ force: true })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => nodeRef.isBypassed()).toBe(true)
|
||||
expect(await nodeRef.isBypassed()).toBe(true)
|
||||
await expect(getNodeWrapper(comfyPage, 'KSampler')).toHaveClass(
|
||||
BYPASS_CLASS
|
||||
)
|
||||
@@ -131,7 +131,7 @@ test.describe('Selection Toolbox - Button Actions', { tag: '@ui' }, () => {
|
||||
await bypassButton.click({ force: true })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => nodeRef.isBypassed()).toBe(false)
|
||||
expect(await nodeRef.isBypassed()).toBe(false)
|
||||
await expect(getNodeWrapper(comfyPage, 'KSampler')).not.toHaveClass(
|
||||
BYPASS_CLASS
|
||||
)
|
||||
|
||||
@@ -94,6 +94,7 @@ test.describe(
|
||||
const nodeRef = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('KSampler')
|
||||
)[0]
|
||||
const initialShape = await nodeRef.getProperty<number>('shape')
|
||||
|
||||
await openMoreOptions(comfyPage)
|
||||
await comfyPage.page.getByText('Shape', { exact: true }).hover()
|
||||
@@ -105,7 +106,9 @@ test.describe(
|
||||
await comfyPage.page.getByText('Box', { exact: true }).click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect.poll(() => nodeRef.getProperty<number>('shape')).toBe(1)
|
||||
const newShape = await nodeRef.getProperty<number>('shape')
|
||||
expect(newShape).not.toBe(initialShape)
|
||||
expect(newShape).toBe(1)
|
||||
})
|
||||
|
||||
test('changes node color via Color submenu swatch', async ({
|
||||
@@ -114,6 +117,9 @@ test.describe(
|
||||
const nodeRef = (
|
||||
await comfyPage.nodeOps.getNodeRefsByTitle('KSampler')
|
||||
)[0]
|
||||
const initialColor = await nodeRef.getProperty<string | undefined>(
|
||||
'color'
|
||||
)
|
||||
|
||||
await openMoreOptions(comfyPage)
|
||||
await comfyPage.page.getByText('Color', { exact: true }).click()
|
||||
@@ -122,9 +128,11 @@ test.describe(
|
||||
await blueSwatch.first().click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect
|
||||
.poll(() => nodeRef.getProperty<string | undefined>('color'))
|
||||
.toBe('#223')
|
||||
const newColor = await nodeRef.getProperty<string | undefined>('color')
|
||||
expect(newColor).toBe('#223')
|
||||
if (initialColor) {
|
||||
expect(newColor).not.toBe(initialColor)
|
||||
}
|
||||
})
|
||||
|
||||
test('renames a node using Rename action', async ({ comfyPage }) => {
|
||||
@@ -142,9 +150,8 @@ test.describe(
|
||||
await input.fill('RenamedNode')
|
||||
await input.press('Enter')
|
||||
await comfyPage.nextFrame()
|
||||
await expect
|
||||
.poll(() => nodeRef.getProperty<string>('title'))
|
||||
.toBe('RenamedNode')
|
||||
const newTitle = await nodeRef.getProperty<string>('title')
|
||||
expect(newTitle).toBe('RenamedNode')
|
||||
})
|
||||
|
||||
test('closes More Options menu when clicking outside', async ({
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user