Compare commits

..

1 Commits

Author SHA1 Message Date
Kelly Yang
7b0d59552d test: add E2E tests for ImageCropV2 drag and free resize (PR 1/4)
- Update workflow asset to include LoadImage source node connected to
  ImageCropV2, enabling image injection in tests
- Add injectSourceImage helper that sets preview images via the Pinia
  nodeOutput store, triggering the composable's reactive image URL update
- Add setCropState / getCropState helpers for controlled test setup and
  assertion
- Add dragFrom / getResizeHandle helpers for pointer interaction
- Level 2 tests (drag): moves box, clamps right, clamps top-left,
  no-op when no image
- Level 3 tests (free resize): all 8 handle directions, MIN_CROP_SIZE
  enforcement, boundary clamping, handle count when ratio unlocked
2026-03-29 16:51:56 -07:00
166 changed files with 1506 additions and 14007 deletions

View File

@@ -1,278 +0,0 @@
---
name: reproduce-issue
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
---
# Issue Reproduction Skill
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
## Architecture
Two videos are produced:
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
```
Phase 1: Research → Read issue, understand prerequisites
Phase 2: Environment → Install custom nodes, load workflows, configure settings
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
Phase 5: Report → Generate a structured reproduction report
```
## Prerequisites
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
- `gh` CLI (authenticated, for reading issues)
- ComfyUI backend with Python environment (for installing custom nodes)
## Phase 1: Research the Issue
1. Fetch the issue details:
```bash
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
```
2. Extract from the issue body:
- **Reproduction steps** (the exact sequence)
- **Prerequisites**: specific workflows, custom nodes, settings, models
- **Environment**: OS, browser, ComfyUI version
- **Media**: screenshots or videos showing the bug
3. Search the codebase for related code:
- Find the feature/component mentioned in the issue
- Understand how it works currently
- Identify what state the UI needs to be in
## Phase 2: Environment Setup
Set up everything the issue requires BEFORE attempting reproduction.
### Custom Nodes
If the issue mentions custom nodes:
```bash
# Find the custom node repo
# Clone into ComfyUI's custom_nodes directory
cd <comfyui_path>/custom_nodes
git clone <custom_node_repo_url>
# Install dependencies if needed
cd <custom_node_name>
pip install -r requirements.txt 2>/dev/null || true
# Restart ComfyUI server to load the new nodes
```
### Workflows
If the issue references a specific workflow:
```bash
# Download workflow JSON if a URL is provided
curl -L "<workflow_url>" -o /tmp/test-workflow.json
# Load it via the API
curl -X POST http://127.0.0.1:8188/api/workflow \
-H "Content-Type: application/json" \
-d @/tmp/test-workflow.json
```
Or load via playwright-cli:
```bash
playwright-cli goto "http://127.0.0.1:8188"
# Drag-and-drop or use File > Open to load the workflow
```
### Settings
If the issue requires specific settings:
```bash
# Use playwright-cli to open settings and change them
playwright-cli press "Control+,"
playwright-cli snapshot
# Find and modify the relevant setting
```
## Phase 3: Interactive Exploration — Research Video
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
```bash
# Open browser and start video recording
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
# Take a snapshot to see current state
playwright-cli snapshot
# Interact based on what you see
playwright-cli click <ref>
playwright-cli fill <ref> "text"
playwright-cli press "Control+s"
# Check results
playwright-cli snapshot
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
```
### Key Principles
- **Observe before acting**: Always `snapshot` before interacting
- **Retry and adapt**: If a step fails, try a different approach
- **Document what works**: Keep notes on which steps trigger the bug
- **Don't give up**: Try multiple approaches if the first doesn't work
- **Establish prerequisites**: Many bugs require specific UI state:
- Save a workflow first (File > Save)
- Make changes to dirty the workflow
- Open multiple tabs
- Add specific node types
- Change settings
- Resize the window
### Common ComfyUI Interactions via playwright-cli
| Action | Command |
| ------------------- | -------------------------------------------------------------- |
| Open hamburger menu | `playwright-cli click` on the C logo button |
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
| Add node | Double-click canvas → type node name → select from results |
| Connect nodes | Drag from output slot to input slot |
| Save workflow | `playwright-cli press "Control+s"` |
| Save As | Menu > File > Save As |
| Select node | Click on the node |
| Delete node | Select → `playwright-cli press "Delete"` |
| Right-click menu | `playwright-cli click <ref> --button right` |
| Keyboard shortcut | `playwright-cli press "Control+z"` |
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
Once the bug is confirmed, **stop the research video** and **close the research browser**:
```bash
playwright-cli video-stop
playwright-cli close
```
Now start a **fresh browser session** for the clean reproduce video (Video 2).
**IMPORTANT constraints:**
- **Max 5 minutes** — the reproduce video must be short and focused
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
1. **Open browser and start recording**:
```bash
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
```
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
3. **Take key screenshots** at critical moments:
```bash
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
# ... trigger the bug ...
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
```
4. **Stop recording and close** immediately after the bug is demonstrated:
```bash
playwright-cli video-stop
playwright-cli close
```
## Phase 5: Generate Report
Create a reproduction report at `tmp/qa/reproduce-report.md`:
```markdown
# Issue Reproduction Report
- **Issue**: <issue_url>
- **Title**: <issue_title>
- **Date**: <today>
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
## Environment
- ComfyUI Server: <url>
- OS: <os>
- Custom Nodes Installed: <list or "none">
- Settings Changed: <list or "none">
## Prerequisites
List everything that had to be set up before the bug could be triggered:
1. ...
2. ...
## Reproduction Steps
Minimal steps to reproduce (the clean sequence):
1. ...
2. ...
3. ...
## Expected Behavior
<from the issue>
## Actual Behavior
<what actually happened>
## Evidence
- Research video: `research-video/video.webm` (full exploration session)
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
- Screenshots: `before-bug.png`, `bug-visible.png`
## Root Cause Analysis (if identified)
<code pointers, hypothesis about what's going wrong>
## Notes
<any additional observations, workarounds discovered, related issues>
```
## Handling Failures
If the bug **cannot be reproduced**:
1. Document what you tried and why it didn't work
2. Check if the issue was already fixed (search git log for related commits)
3. Check if it's environment-specific (OS, browser, specific version)
4. Set report status to "Not Reproduced" with detailed notes
5. The report is still valuable — it saves others from repeating the same investigation
## CI Integration
In CI, this skill runs as a Claude Code agent with:
- `ANTHROPIC_API_KEY` for Claude
- `GEMINI_API_KEY` for initial issue analysis (optional)
- ComfyUI server pre-started in the container
- `playwright-cli` pre-installed
The CI workflow:
1. Gemini generates a reproduce guide (markdown) from the issue
2. Claude agent receives the guide and runs this skill
3. Claude explores interactively, installs dependencies, retries
4. Claude records a clean demo once reproduced
5. Video and report are uploaded as artifacts

View File

@@ -1,277 +0,0 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Automated quality assurance for the ComfyUI frontend. The pipeline reproduces reported bugs using Playwright E2E tests, records video evidence, and deploys reports to Cloudflare Pages.
## Architecture Overview
The QA pipeline uses a **three-phase approach**:
1. **RESEARCH** — Claude writes Playwright E2E tests to reproduce bugs (assertion-backed, no hallucination)
2. **REPRODUCE** — Deterministic replay of the research test with video recording
3. **REPORT** — Deploy results to Cloudflare Pages with badge, video, and verdict
### Key Design Decision
Earlier iterations used AI vision (Gemini) to drive a browser and judge results from video. This was abandoned after discovering **AI reviewers hallucinate** — Gemini reported "REPRODUCED" when videos showed idle screens. The current approach uses **Playwright assertions** as the source of truth: if the test passes, the bug is proven.
## Prerequisites
- Node.js 22+
- `pnpm` package manager
- `gh` CLI (authenticated)
- Playwright browsers: `npx playwright install chromium`
- Environment variables:
- `GEMINI_API_KEY` — for PR analysis and video review
- `ANTHROPIC_API_KEY` — for Claude Agent SDK (research phase)
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — for report deployment
## Pipeline Scripts
| Script | Role | Model |
|---|---|---|
| `scripts/qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide | gemini-3.1-pro-preview |
| `scripts/qa-agent.ts` | Research phase: Claude writes E2E tests | claude-sonnet-4-6 (Agent SDK) |
| `scripts/qa-record.ts` | Before/after video recording with Gemini-driven actions | gemini-3.1-pro-preview |
| `scripts/qa-reproduce.ts` | Deterministic replay with narration | gemini-3-flash-preview |
| `scripts/qa-video-review.ts` | Video comparison review | gemini-3-flash-preview |
| `scripts/qa-generate-test.ts` | Regression test generation from QA report | gemini-3-flash-preview |
| `scripts/qa-deploy-pages.sh` | Deploy to Cloudflare Pages + badge | — |
| `scripts/qa-batch.sh` | Batch-trigger QA for multiple issues | — |
| `scripts/qa-report-template.html` | Report site (light/dark, seekbar, copy badge) | — |
## Triggering QA
### Via GitHub Labels
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after comparison)
- **`qa-full`** — Full QA (3-OS matrix, after-only)
- **`qa-issue`** — Reproduce a bug from an issue
### Via Batch Script
```bash
# Trigger QA for specific issue numbers
./scripts/qa-batch.sh 10394 10238 9996
# From a triage file (top 5 Tier 1 issues)
./scripts/qa-batch.sh --from tmp/issues.md --top 5
# Preview without pushing
./scripts/qa-batch.sh --dry-run 10394
# Clean up old trigger branches
./scripts/qa-batch.sh --cleanup
```
### Via Workflow Dispatch
Go to Actions → "PR: QA" → Run workflow → choose mode (focused/full).
## CI Workflow (`.github/workflows/pr-qa.yaml`)
```
resolve-matrix → analyze-pr ──┐
├→ qa-before (main branch, worktree build)
├→ qa-after (PR branch)
└→ report (video review, deploy, comment)
```
Before/after jobs run **in parallel** on separate runners for clean isolation.
### Issue Reproduce Mode
For issues (not PRs), the pipeline:
1. Fetches the issue body and comments
2. Runs `qa-analyze-pr.ts --type issue` to generate a QA guide
3. Runs the research phase (Claude writes E2E test to reproduce)
4. Records video of the test execution
5. Posts results as a comment on the issue
## Running Locally
### Step 1: Environment Setup
```bash
# Ensure ComfyUI server is running
# Default: http://127.0.0.1:8188
# Install Playwright browsers
npx playwright install chromium
```
### Step 2: Analyze the Issue/PR
```bash
# For a PR
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides
# For an issue
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides \
--type issue
```
### Step 3: Record Before/After
```bash
# Before (main branch)
pnpm exec tsx scripts/qa-record.ts \
--mode before \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-before \
--qa-guide qa-guides/qa-guide-1.json
# After (PR branch)
pnpm exec tsx scripts/qa-record.ts \
--mode after \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-after \
--qa-guide qa-guides/qa-guide-1.json
```
### Step 4: Review Videos
```bash
pnpm exec tsx scripts/qa-video-review.ts \
--artifacts-dir /tmp/qa-artifacts \
--video-file qa-session.mp4 \
--before-video qa-before-session.mp4 \
--output-dir /tmp/video-reviews \
--pr-context /tmp/pr-context.txt
```
## Research Phase Details (`qa-agent.ts`)
Claude receives:
- The issue description and comments
- A QA guide from `qa-analyze-pr.ts`
- An accessibility tree snapshot of the current UI
Claude's tools:
- **`inspect(selector?)`** — Read a11y tree to discover element selectors
- **`writeTest(code)`** — Write a Playwright `.spec.ts` file
- **`runTest()`** — Execute the test and get pass/fail + errors
- **`done(verdict, summary, evidence, testCode)`** — Finish with verdict
The test uses the project's Playwright fixtures (`comfyPageFixture`), giving access to `comfyPage.page`, `comfyPage.menu`, `comfyPage.settings`, etc.
### Verdict Logic
- **REPRODUCED** — Test passes (asserting the bug exists) → bug is proven
- **NOT_REPRODUCIBLE** — Claude exhausted attempts, test cannot pass
- **INCONCLUSIVE** — Agent timed out or encountered infrastructure issues
Auto-completion: if a test passed but `done()` was never called, the pipeline auto-completes with REPRODUCED.
## Manual QA (Fallback)
When the automated pipeline isn't suitable (e.g., visual-only bugs, complex multi-step interactions), use **playwright-cli** for manual browser interaction:
```bash
# Install
npm install -g @playwright/cli@latest
# Open browser and navigate
playwright-cli open http://127.0.0.1:8188
# Get element references
playwright-cli snapshot
# Interact
playwright-cli click e1
playwright-cli fill e2 "test text"
playwright-cli press Escape
playwright-cli screenshot --filename=f.png
```
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation to refresh refs.
## Manual QA Test Plan
When performing manual QA (either via playwright-cli or the automated pipeline), systematically test each area below.
### Application Load & Routes
| Test | Steps |
|---|---|
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### Canvas & Graph View
| Test | Steps |
|---|---|
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Context menus | Right-click node vs empty canvas — different menus |
### Sidebar Tabs
| Test | Steps |
|---|---|
| Workflows tab | Press W — workflows sidebar opens |
| Node Library tab | Press N — node library opens |
| Model Library tab | Press M — model library opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
### Settings Dialog
| Test | Steps |
|---|---|
| Open settings | Press Ctrl+, or click settings button |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Close settings | Press Escape or click close button |
### Execution & Queue
| Test | Steps |
|---|---|
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
## Report Site
Deployed to Cloudflare Pages at `https://comfy-qa.pages.dev/<branch>/`.
Features:
- Light/dark theme
- Seekable video player with preload
- Copy badge button (markdown)
- Date-stamped badges (e.g., `QA0327`)
- Vertical box badge for issues and PRs
## Known Issues & Troubleshooting
See `docs/qa/TROUBLESHOOTING.md` for common failures:
- `set -euo pipefail` + grep with no match → append `|| true`
- `__name is not defined` in `page.evaluate` → use `addScriptTag`
- Cursor not visible in videos → monkey-patch `page.mouse` methods
- Agent not calling `done()` → auto-complete from passing test
## Backlog
See `docs/qa/backlog.md` for planned improvements:
- **Type B comparison**: Different commits for regression detection
- **Type C comparison**: Cross-browser testing
- **Pre-seed assets**: Upload test images before recording
- **Lazy a11y tree**: Reduce token usage with `inspect(selector)` vs full dump

View File

@@ -44,17 +44,12 @@ runs:
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Start ComfyUI server
if: ${{ inputs.launch_server == 'true' }}
shell: bash
working-directory: ComfyUI
env:
EXTRA_SERVER_PARAMS: ${{ inputs.extra_server_params }}
run: |
python main.py --cpu --multi-user --front-end-root ../dist $EXTRA_SERVER_PARAMS &
for i in $(seq 1 300); do
curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1 && echo "Server ready" && exit 0
sleep 2
done
echo "::error::ComfyUI server did not start within 600s" && exit 1
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
wait-for-it --service 127.0.0.1:8188 -t 600

View File

@@ -95,7 +95,7 @@ jobs:
if npx license-checker-rseidelsohn@4 \
--production \
--summary \
--excludePackages '@comfyorg/comfyui-frontend;@comfyorg/design-system;@comfyorg/ingest-types;@comfyorg/registry-types;@comfyorg/shared-frontend-utils;@comfyorg/tailwind-utils;@comfyorg/comfyui-electron-types' \
--excludePackages '@comfyorg/comfyui-frontend;@comfyorg/design-system;@comfyorg/registry-types;@comfyorg/shared-frontend-utils;@comfyorg/tailwind-utils;@comfyorg/comfyui-electron-types' \
--clarificationsFile .github/license-clarifications.json \
--onlyAllow 'MIT;MIT*;Apache-2.0;BSD-2-Clause;BSD-3-Clause;ISC;0BSD;BlueOak-1.0.0;Python-2.0;CC0-1.0;Unlicense;(MIT OR Apache-2.0);(MIT OR GPL-3.0);(Apache-2.0 OR MIT);(MPL-2.0 OR Apache-2.0);CC-BY-4.0;CC-BY-3.0;GPL-3.0-only'; then
echo ''

View File

@@ -1,4 +1,4 @@
# Description: Unit and component testing with Vitest + coverage reporting
# Description: Unit and component testing with Vitest
name: 'CI: Tests Unit'
on:
@@ -23,12 +23,5 @@ jobs:
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Run Vitest tests with coverage
run: pnpm test:coverage
- name: Upload coverage to Codecov
if: always()
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
with:
files: coverage/lcov.info
fail_ci_if_error: false
- name: Run Vitest tests
run: pnpm test:unit

File diff suppressed because it is too large Load Diff

5
.gitignore vendored
View File

@@ -99,7 +99,4 @@ vitest.config.*.timestamp*
# Weekly docs check output
/output.txt
.amp
.playwright-cli/
.playwright/
.claude/scheduled_tasks.lock
.amp

View File

@@ -9,7 +9,6 @@
"packages/registry-types/src/comfyRegistryTypes.ts",
"public/materialdesignicons.min.css",
"src/types/generatedManagerTypes.ts",
"**/__fixtures__/**/*.json",
"scripts/qa-report-template.html"
"**/__fixtures__/**/*.json"
]
}

View File

@@ -1,47 +0,0 @@
<script setup lang="ts">
const features = [
{ icon: '📚', label: 'Guided Tutorials' },
{ icon: '🎥', label: 'Video Courses' },
{ icon: '🛠️', label: 'Hands-on Projects' }
]
</script>
<template>
<section class="bg-charcoal-800 py-24">
<div class="mx-auto max-w-3xl px-6 text-center">
<!-- Badge -->
<span
class="inline-block rounded-full bg-brand-yellow/10 px-4 py-1.5 text-xs uppercase tracking-widest text-brand-yellow"
>
COMFY ACADEMY
</span>
<h2 class="mt-6 text-3xl font-bold text-white">Master AI Workflows</h2>
<p class="mt-4 text-smoke-700">
Learn to build professional AI workflows with guided tutorials, video
courses, and hands-on projects.
</p>
<!-- Feature bullets -->
<div class="mt-8 flex flex-wrap items-center justify-center gap-8">
<div
v-for="feature in features"
:key="feature.label"
class="flex items-center gap-2 text-sm text-white"
>
<span aria-hidden="true">{{ feature.icon }}</span>
<span>{{ feature.label }}</span>
</div>
</div>
<!-- CTA -->
<a
href="/academy"
class="mt-8 inline-block rounded-full bg-brand-yellow px-8 py-3 text-sm font-semibold text-black transition-opacity hover:opacity-90"
>
EXPLORE ACADEMY
</a>
</div>
</section>
</template>

View File

@@ -1,66 +0,0 @@
<script setup lang="ts">
const cards = [
{
icon: '🖥️',
title: 'Comfy Desktop',
description: 'Full power on your local machine. Free and open source.',
cta: 'DOWNLOAD',
href: '/download',
outlined: false
},
{
icon: '☁️',
title: 'Comfy Cloud',
description: 'Run workflows in the cloud. No GPU required.',
cta: 'TRY CLOUD',
href: 'https://app.comfy.org',
outlined: false
},
{
icon: '⚡',
title: 'Comfy API',
description: 'Integrate AI generation into your applications.',
cta: 'VIEW DOCS',
href: 'https://docs.comfy.org',
outlined: true
}
]
</script>
<template>
<section class="bg-charcoal-800 py-24">
<div class="mx-auto max-w-5xl px-6">
<h2 class="text-center text-3xl font-bold text-white">
Choose Your Way to Comfy
</h2>
<!-- CTA cards -->
<div class="mt-12 grid grid-cols-1 gap-6 md:grid-cols-3">
<a
v-for="card in cards"
:key="card.title"
:href="card.href"
class="flex flex-1 flex-col items-center rounded-xl border border-white/10 bg-charcoal-600 p-8 text-center transition-colors hover:border-brand-yellow"
>
<span class="text-4xl" aria-hidden="true">{{ card.icon }}</span>
<h3 class="mt-4 text-xl font-semibold text-white">
{{ card.title }}
</h3>
<p class="mt-2 text-sm text-smoke-700">
{{ card.description }}
</p>
<span
class="mt-6 inline-block rounded-full px-6 py-2 text-sm font-semibold transition-opacity hover:opacity-90"
:class="
card.outlined
? 'border border-brand-yellow text-brand-yellow'
: 'bg-brand-yellow text-black'
"
>
{{ card.cta }}
</span>
</a>
</div>
</div>
</section>
</template>

View File

@@ -1,77 +0,0 @@
<!-- TODO: Replace placeholder content with real quotes and case studies -->
<script setup lang="ts">
const studies = [
{
title: 'New Pipelines with Chord Mode',
body: 'For AI-assisted texture and environment generation across studio pipelines.',
highlight: false,
gridClass: 'md:row-span-2'
},
{
title: 'AI-Assisted Texture and Environment',
body: 'For AI-assisted texture and environment generation across studio pipelines.',
highlight: false,
gridClass: 'min-h-[300px] lg:col-span-2'
},
{
title: 'Open-sourced the Chord Mode',
body: 'For AI-assisted texture and environment generation across studio pipelines.',
highlight: false,
gridClass: 'min-h-[200px]'
},
{
title: 'Environment Generation',
body: 'For AI-assisted texture and environment generation across studio pipelines.',
highlight: true,
gridClass: 'min-h-[200px]'
}
]
</script>
<template>
<section class="bg-black px-6 py-24">
<div class="mx-auto max-w-7xl">
<header class="mb-12">
<h2 class="text-3xl font-bold text-white">Customer Stories</h2>
<p class="mt-2 text-smoke-700">
See how leading studios use Comfy in production
</p>
</header>
<!-- Bento grid -->
<div
class="relative grid grid-cols-1 gap-4 md:grid-cols-2 lg:grid-cols-3"
>
<article
v-for="study in studies"
:key="study.title"
class="flex flex-col justify-end rounded-2xl border border-brand-yellow/30 p-6"
:class="[
study.gridClass,
study.highlight ? 'bg-brand-yellow' : 'bg-charcoal-600'
]"
>
<h3
class="font-semibold"
:class="study.highlight ? 'text-black' : 'text-white'"
>
{{ study.title }}
</h3>
<p
class="mt-2 text-sm"
:class="study.highlight ? 'text-black/70' : 'text-smoke-700'"
>
{{ study.body }}
</p>
<a
href="/case-studies"
class="mt-4 text-sm underline"
:class="study.highlight ? 'text-black' : 'text-brand-yellow'"
>
READ CASE STUDY
</a>
</article>
</div>
</div>
</section>
</template>

View File

@@ -1,62 +0,0 @@
<script setup lang="ts">
const steps = [
{
number: '1',
title: 'Download & Sign Up',
description: 'Get Comfy Desktop for free or create a Cloud account'
},
{
number: '2',
title: 'Load a Workflow',
description:
'Choose from thousands of community workflows or build your own'
},
{
number: '3',
title: 'Generate',
description: 'Hit run and watch your AI workflow come to life'
}
]
</script>
<template>
<section class="border-t border-white/10 bg-black py-24">
<div class="mx-auto max-w-7xl px-6 text-center">
<h2 class="text-3xl font-bold text-white">Get Started in Minutes</h2>
<p class="mt-4 text-smoke-700">
From download to your first AI-generated output in three simple steps
</p>
<!-- Steps -->
<div class="mt-16 grid grid-cols-1 gap-8 md:grid-cols-3">
<div v-for="(step, index) in steps" :key="step.number" class="relative">
<!-- Connecting line between steps (desktop only) -->
<div
v-if="index < steps.length - 1"
class="absolute right-0 top-8 hidden w-full translate-x-1/2 border-t border-brand-yellow/20 md:block"
/>
<div class="relative">
<span class="text-6xl font-bold text-brand-yellow/20">
{{ step.number }}
</span>
<h3 class="mt-2 text-xl font-semibold text-white">
{{ step.title }}
</h3>
<p class="mt-2 text-sm text-smoke-700">
{{ step.description }}
</p>
</div>
</div>
</div>
<!-- CTA -->
<a
href="/download"
class="mt-12 inline-block rounded-full bg-brand-yellow px-8 py-3 text-sm font-semibold text-black transition-opacity hover:opacity-90"
>
DOWNLOAD COMFY
</a>
</div>
</section>
</template>

View File

@@ -1,68 +0,0 @@
<script setup lang="ts">
const ctaButtons = [
{
label: 'GET STARTED',
href: 'https://app.comfy.org',
variant: 'solid' as const
},
{
label: 'LEARN MORE',
href: '/about',
variant: 'outline' as const
}
]
</script>
<template>
<section
class="relative flex min-h-screen items-center overflow-hidden bg-black pt-16"
>
<div
class="mx-auto flex w-full max-w-7xl flex-col items-center gap-12 px-6 md:flex-row md:gap-0"
>
<!-- Left: C Monogram -->
<div class="flex w-full items-center justify-center md:w-[55%]">
<div class="relative -ml-12 -rotate-15 md:-ml-24" aria-hidden="true">
<div
class="h-64 w-64 rounded-full border-[40px] border-brand-yellow md:h-[28rem] md:w-[28rem] md:border-[64px] lg:h-[36rem] lg:w-[36rem] lg:border-[80px]"
>
<!-- Gap on the right side to form "C" shape -->
<div
class="absolute right-0 top-1/2 h-32 w-24 -translate-y-1/2 translate-x-1/2 bg-black md:h-48 md:w-36 lg:h-64 lg:w-48"
/>
</div>
</div>
</div>
<!-- Right: Text content -->
<div class="flex w-full flex-col items-start md:w-[45%]">
<h1
class="text-5xl font-bold leading-tight tracking-tight text-white md:text-6xl lg:text-7xl"
>
Professional Control of Visual AI
</h1>
<p class="mt-6 max-w-lg text-lg text-smoke-700">
Comfy is the AI creation engine for visual professionals who demand
control over every model, every parameter, and every output.
</p>
<div class="mt-8 flex flex-wrap gap-4">
<a
v-for="btn in ctaButtons"
:key="btn.label"
:href="btn.href"
class="rounded-full px-8 py-3 text-sm font-semibold transition-opacity hover:opacity-90"
:class="
btn.variant === 'solid'
? 'bg-brand-yellow text-black'
: 'border border-brand-yellow text-brand-yellow'
"
>
{{ btn.label }}
</a>
</div>
</div>
</div>
</section>
</template>

View File

@@ -1,26 +0,0 @@
<template>
<section class="bg-black py-24">
<div class="mx-auto max-w-4xl px-6 text-center">
<!-- Decorative quote mark -->
<span class="text-6xl text-brand-yellow opacity-30" aria-hidden="true">
&laquo;
</span>
<h2 class="text-4xl font-bold text-white md:text-5xl">
Method, Not Magic
</h2>
<p class="mx-auto mt-6 max-w-2xl text-lg leading-relaxed text-smoke-700">
We believe in giving creators real control over AI. Not black boxes. Not
magic buttons. But transparent, reproducible, node-by-node control over
every step of the creative process.
</p>
<!-- Separator line -->
<div
class="mx-auto mt-8 h-0.5 w-24 bg-brand-yellow opacity-30"
aria-hidden="true"
/>
</div>
</section>
</template>

View File

@@ -1,51 +0,0 @@
<!-- TODO: Replace with actual workflow demo content -->
<script setup lang="ts">
const features = ['Node-Based Editor', 'Real-Time Preview', 'Version Control']
</script>
<template>
<section class="bg-charcoal-800 py-24">
<div class="mx-auto max-w-7xl px-6">
<!-- Section header -->
<div class="text-center">
<h2 class="text-3xl font-bold text-white">See Comfy in Action</h2>
<p class="mx-auto mt-4 max-w-2xl text-smoke-700">
Watch how professionals build AI workflows with unprecedented control
</p>
</div>
<!-- Placeholder video area -->
<div
class="mt-12 flex aspect-video items-center justify-center rounded-2xl border border-white/10 bg-charcoal-600"
>
<div class="flex flex-col items-center gap-4">
<!-- Play button triangle -->
<div
class="flex h-16 w-16 items-center justify-center rounded-full border-2 border-white/20"
aria-hidden="true"
>
<div
class="ml-1 h-0 w-0 border-y-8 border-l-[14px] border-y-transparent border-l-white"
/>
</div>
<p class="text-sm text-smoke-700">Workflow Demo Coming Soon</p>
</div>
</div>
<!-- Feature labels -->
<div class="mt-8 flex flex-wrap items-center justify-center gap-6">
<div
v-for="feature in features"
:key="feature"
class="flex items-center gap-2"
>
<span
class="h-2 w-2 rounded-full bg-brand-yellow"
aria-hidden="true"
/>
<span class="text-sm text-smoke-700">{{ feature }}</span>
</div>
</div>
</div>
</section>
</template>

View File

@@ -1,58 +0,0 @@
<script setup lang="ts">
const logos = [
'Harman',
'Tencent',
'Nike',
'HP',
'Autodesk',
'Apple',
'Ubisoft',
'Lucid',
'Amazon',
'Netflix',
'Pixomondo',
'EA'
]
const metrics = [
{ value: '60K+', label: 'Custom Nodes' },
{ value: '106K+', label: 'GitHub Stars' },
{ value: '500K+', label: 'Community Members' }
]
</script>
<template>
<section class="border-y border-white/10 bg-black py-16">
<div class="mx-auto max-w-7xl px-6">
<!-- Heading -->
<p
class="text-center text-xs font-medium uppercase tracking-widest text-smoke-700"
>
Trusted by Industry Leaders
</p>
<!-- Logo row -->
<div
class="mt-10 flex flex-wrap items-center justify-center gap-4 md:gap-6"
>
<span
v-for="company in logos"
:key="company"
class="rounded-full border border-white/10 px-6 py-2 text-sm text-smoke-700"
>
{{ company }}
</span>
</div>
<!-- Metrics row -->
<div
class="mt-14 flex flex-col items-center justify-center gap-10 sm:flex-row sm:gap-12"
>
<div v-for="metric in metrics" :key="metric.label" class="text-center">
<p class="text-3xl font-bold text-white">{{ metric.value }}</p>
<p class="mt-1 text-sm text-smoke-700">{{ metric.label }}</p>
</div>
</div>
</div>
</section>
</template>

View File

@@ -1,92 +0,0 @@
<script setup lang="ts">
import { computed, ref } from 'vue'
const activeFilter = ref('All')
const industries = ['All', 'VFX', 'Gaming', 'Advertising', 'Photography']
const testimonials = [
{
quote:
'Comfy has transformed our VFX pipeline. The node-based approach gives us unprecedented control over every step of the generation process.',
name: 'Sarah Chen',
title: 'Lead Technical Artist',
company: 'Studio Alpha',
industry: 'VFX'
},
{
quote:
'The level of control over AI generation is unmatched. We can iterate on game assets faster than ever before.',
name: 'Marcus Rivera',
title: 'Creative Director',
company: 'PixelForge',
industry: 'Gaming'
},
{
quote:
'We\u2019ve cut our iteration time by 70%. Comfy workflows let our team produce high-quality creative assets at scale.',
name: 'Yuki Tanaka',
title: 'Head of AI',
company: 'CreativeX',
industry: 'Advertising'
}
]
const filteredTestimonials = computed(() => {
if (activeFilter.value === 'All') return testimonials
return testimonials.filter((t) => t.industry === activeFilter.value)
})
</script>
<template>
<section class="bg-black py-24">
<div class="mx-auto max-w-7xl px-6">
<h2 class="text-center text-3xl font-bold text-white">
What Professionals Say
</h2>
<!-- Industry filter pills -->
<div class="mt-8 flex flex-wrap items-center justify-center gap-3">
<button
v-for="industry in industries"
:key="industry"
class="cursor-pointer rounded-full px-4 py-1.5 text-sm transition-colors"
:class="
activeFilter === industry
? 'bg-brand-yellow text-black'
: 'border border-white/10 text-smoke-700 hover:border-brand-yellow'
"
@click="activeFilter = industry"
>
{{ industry }}
</button>
</div>
<!-- Testimonial cards -->
<div class="mt-12 grid grid-cols-1 gap-6 md:grid-cols-3">
<article
v-for="testimonial in filteredTestimonials"
:key="testimonial.name"
class="rounded-xl border border-white/10 bg-charcoal-600 p-6"
>
<blockquote class="text-base italic text-white">
&ldquo;{{ testimonial.quote }}&rdquo;
</blockquote>
<p class="mt-4 text-sm font-semibold text-white">
{{ testimonial.name }}
</p>
<p class="text-sm text-smoke-700">
{{ testimonial.title }}, {{ testimonial.company }}
</p>
<span
class="mt-3 inline-block rounded-full bg-white/5 px-2 py-0.5 text-xs text-smoke-700"
>
{{ testimonial.industry }}
</span>
</article>
</div>
</div>
</section>
</template>

View File

@@ -1,74 +0,0 @@
<!-- TODO: Wire category content swap when final assets arrive -->
<script setup lang="ts">
import { ref } from 'vue'
const categories = [
'VFX & Animation',
'Creative Agencies',
'Gaming',
'eCommerce & Fashion',
'Community & Hobbyists'
]
const activeCategory = ref(0)
</script>
<template>
<section class="bg-charcoal-800 px-6 py-24">
<div class="mx-auto max-w-7xl">
<div class="flex flex-col items-center gap-12 lg:flex-row lg:gap-8">
<!-- Left placeholder image (desktop only) -->
<div class="hidden flex-1 lg:block">
<div
class="aspect-[2/3] rounded-full border border-white/10 bg-charcoal-600"
/>
</div>
<!-- Center content -->
<div class="flex flex-col items-center text-center lg:flex-[2]">
<h2 class="text-3xl font-bold text-white">
Built for Every Creative Industry
</h2>
<nav
class="mt-10 flex flex-col items-center gap-4"
aria-label="Industry categories"
>
<button
v-for="(category, index) in categories"
:key="category"
class="transition-colors"
:class="
index === activeCategory
? 'text-2xl text-white'
: 'text-xl text-ash-500 hover:text-white/70'
"
@click="activeCategory = index"
>
{{ category }}
</button>
</nav>
<p class="mt-10 max-w-lg text-smoke-700">
Powered by 60,000+ nodes, thousands of workflows, and a community
that builds faster than any one company could.
</p>
<a
href="/workflows"
class="mt-8 rounded-full border border-brand-yellow px-8 py-3 text-sm font-semibold text-brand-yellow transition-colors hover:bg-brand-yellow hover:text-black"
>
EXPLORE WORKFLOWS
</a>
</div>
<!-- Right placeholder image (desktop only) -->
<div class="hidden flex-1 lg:block">
<div
class="aspect-[2/3] rounded-3xl border border-white/10 bg-charcoal-600"
/>
</div>
</div>
</div>
</section>
</template>

View File

@@ -1,67 +0,0 @@
<script setup lang="ts">
const pillars = [
{
icon: '⚡',
title: 'Build',
description:
'Design complex AI workflows visually with our node-based editor'
},
{
icon: '🎨',
title: 'Customize',
description: 'Fine-tune every parameter across any model architecture'
},
{
icon: '🔧',
title: 'Refine',
description:
'Iterate on outputs with precision controls and real-time preview'
},
{
icon: '⚙️',
title: 'Automate',
description:
'Scale your workflows with batch processing and API integration'
},
{
icon: '🚀',
title: 'Run',
description: 'Deploy locally or in the cloud with identical results'
}
]
</script>
<template>
<section class="bg-black px-6 py-24">
<div class="mx-auto max-w-7xl">
<header class="mb-16 text-center">
<h2 class="text-3xl font-bold text-white md:text-4xl">
The Building Blocks of AI Production
</h2>
<p class="mt-4 text-smoke-700">
Five powerful capabilities that give you complete control
</p>
</header>
<div class="grid grid-cols-1 gap-6 sm:grid-cols-2 lg:grid-cols-5">
<article
v-for="pillar in pillars"
:key="pillar.title"
class="rounded-xl border border-white/10 bg-charcoal-600 p-6 transition-colors hover:border-brand-yellow"
>
<div
class="flex h-12 w-12 items-center justify-center rounded-full bg-brand-yellow text-xl"
>
{{ pillar.icon }}
</div>
<h3 class="mt-4 text-lg font-semibold text-white">
{{ pillar.title }}
</h3>
<p class="mt-2 text-sm text-smoke-700">
{{ pillar.description }}
</p>
</article>
</div>
</div>
</section>
</template>

View File

@@ -1,34 +0,0 @@
---
import BaseLayout from '../layouts/BaseLayout.astro'
import SiteNav from '../components/SiteNav.vue'
import HeroSection from '../components/HeroSection.vue'
import SocialProofBar from '../components/SocialProofBar.vue'
import ProductShowcase from '../components/ProductShowcase.vue'
import ValuePillars from '../components/ValuePillars.vue'
import UseCaseSection from '../components/UseCaseSection.vue'
import CaseStudySpotlight from '../components/CaseStudySpotlight.vue'
import TestimonialsSection from '../components/TestimonialsSection.vue'
import GetStartedSection from '../components/GetStartedSection.vue'
import CTASection from '../components/CTASection.vue'
import ManifestoSection from '../components/ManifestoSection.vue'
import AcademySection from '../components/AcademySection.vue'
import SiteFooter from '../components/SiteFooter.vue'
---
<BaseLayout title="Comfy — Professional Control of Visual AI">
<SiteNav client:load />
<main>
<HeroSection />
<SocialProofBar />
<ProductShowcase />
<ValuePillars />
<UseCaseSection client:visible />
<CaseStudySpotlight />
<TestimonialsSection client:visible />
<GetStartedSection />
<CTASection />
<ManifestoSection />
<AcademySection />
</main>
<SiteFooter />
</BaseLayout>

View File

@@ -1,34 +0,0 @@
---
import BaseLayout from '../../layouts/BaseLayout.astro'
import SiteNav from '../../components/SiteNav.vue'
import HeroSection from '../../components/HeroSection.vue'
import SocialProofBar from '../../components/SocialProofBar.vue'
import ProductShowcase from '../../components/ProductShowcase.vue'
import ValuePillars from '../../components/ValuePillars.vue'
import UseCaseSection from '../../components/UseCaseSection.vue'
import CaseStudySpotlight from '../../components/CaseStudySpotlight.vue'
import TestimonialsSection from '../../components/TestimonialsSection.vue'
import GetStartedSection from '../../components/GetStartedSection.vue'
import CTASection from '../../components/CTASection.vue'
import ManifestoSection from '../../components/ManifestoSection.vue'
import AcademySection from '../../components/AcademySection.vue'
import SiteFooter from '../../components/SiteFooter.vue'
---
<BaseLayout title="Comfy — 视觉 AI 的专业控制">
<SiteNav client:load />
<main>
<HeroSection />
<SocialProofBar />
<ProductShowcase />
<ValuePillars />
<UseCaseSection client:visible />
<CaseStudySpotlight />
<TestimonialsSection client:visible />
<GetStartedSection />
<CTASection />
<ManifestoSection />
<AcademySection />
</main>
<SiteFooter />
</BaseLayout>

View File

@@ -119,7 +119,7 @@ When writing new tests, follow these patterns:
```typescript
// Import the test fixture
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
test.describe('Feature Name', () => {
// Set up test environment if needed
@@ -148,12 +148,6 @@ Always check for existing helpers and fixtures before implementing new ones:
Most common testing needs are already addressed by these helpers, which will make your tests more consistent and reliable.
### Import Conventions
- Prefer `@e2e/*` for imports within `browser_tests/`
- Continue using `@/*` for imports from `src/`
- Avoid introducing new deep relative imports within `browser_tests/` when the alias is available
### Key Testing Patterns
1. **Focus elements explicitly**:

View File

@@ -1,30 +0,0 @@
{
"1": {
"class_type": "KSampler",
"inputs": {
"seed": 42,
"steps": 20,
"cfg": 8.0,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 1.0
},
"_meta": { "title": "KSampler" }
},
"2": {
"class_type": "NonExistentCustomNode_XYZ_12345",
"inputs": {
"input1": "test"
},
"_meta": { "title": "Missing Node" }
},
"3": {
"class_type": "EmptyLatentImage",
"inputs": {
"width": 512,
"height": 512,
"batch_size": 1
},
"_meta": { "title": "Empty Latent Image" }
}
}

View File

@@ -0,0 +1,69 @@
{
"last_node_id": 2,
"last_link_id": 1,
"nodes": [
{
"id": 1,
"type": "ImageCropV2",
"pos": [400, 50],
"size": [200, 300],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 1
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": null
}
],
"properties": {
"Node name for S&R": "ImageCropV2"
},
"widgets_values": [{ "x": 0, "y": 0, "width": 512, "height": 512 }]
},
{
"id": 2,
"type": "LoadImage",
"pos": [50, 50],
"size": [315, 314],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [1]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": ["example.png", "image"]
}
],
"links": [[1, 2, 0, 1, 0, "IMAGE"]],
"groups": [],
"config": {},
"extra": {
"ds": {
"offset": [0, 0],
"scale": 1
}
},
"version": 0.4
}

View File

@@ -1,43 +1,46 @@
import type { APIRequestContext, Locator, Page } from '@playwright/test'
import { test as base } from '@playwright/test'
import type {
APIRequestContext,
ExpectMatcherState,
Locator,
Page
} from '@playwright/test'
import { test as base, expect } from '@playwright/test'
import { config as dotenvConfig } from 'dotenv'
import { TestIds } from './selectors'
import { NodeBadgeMode } from '../../src/types/nodeSource'
import { ComfyActionbar } from '@e2e/helpers/actionbar'
import { ComfyTemplates } from '@e2e/helpers/templates'
import { ComfyMouse } from '@e2e/fixtures/ComfyMouse'
import { TestIds } from '@e2e/fixtures/selectors'
import { comfyExpect } from '@e2e/fixtures/utils/customMatchers'
import { assetPath } from '@e2e/fixtures/utils/paths'
import { sleep } from '@e2e/fixtures/utils/timing'
import { VueNodeHelpers } from '@e2e/fixtures/VueNodeHelpers'
import { BottomPanel } from '@e2e/fixtures/components/BottomPanel'
import { ComfyNodeSearchBox } from '@e2e/fixtures/components/ComfyNodeSearchBox'
import { ComfyNodeSearchBoxV2 } from '@e2e/fixtures/components/ComfyNodeSearchBoxV2'
import { ContextMenu } from '@e2e/fixtures/components/ContextMenu'
import { QueuePanel } from '@e2e/fixtures/components/QueuePanel'
import { SettingDialog } from '@e2e/fixtures/components/SettingDialog'
import { ComfyActionbar } from '../helpers/actionbar'
import { ComfyTemplates } from '../helpers/templates'
import { ComfyMouse } from './ComfyMouse'
import { VueNodeHelpers } from './VueNodeHelpers'
import { ComfyNodeSearchBox } from './components/ComfyNodeSearchBox'
import { ComfyNodeSearchBoxV2 } from './components/ComfyNodeSearchBoxV2'
import { ContextMenu } from './components/ContextMenu'
import { SettingDialog } from './components/SettingDialog'
import { BottomPanel } from './components/BottomPanel'
import {
AssetsSidebarTab,
NodeLibrarySidebarTab,
WorkflowsSidebarTab
} from '@e2e/fixtures/components/SidebarTab'
import { Topbar } from '@e2e/fixtures/components/Topbar'
import { AppModeHelper } from '@e2e/fixtures/helpers/AppModeHelper'
import { AssetsHelper } from '@e2e/fixtures/helpers/AssetsHelper'
import { CanvasHelper } from '@e2e/fixtures/helpers/CanvasHelper'
import { ClipboardHelper } from '@e2e/fixtures/helpers/ClipboardHelper'
import { CommandHelper } from '@e2e/fixtures/helpers/CommandHelper'
import { DragDropHelper } from '@e2e/fixtures/helpers/DragDropHelper'
import { FeatureFlagHelper } from '@e2e/fixtures/helpers/FeatureFlagHelper'
import { KeyboardHelper } from '@e2e/fixtures/helpers/KeyboardHelper'
import { NodeOperationsHelper } from '@e2e/fixtures/helpers/NodeOperationsHelper'
import { PerformanceHelper } from '@e2e/fixtures/helpers/PerformanceHelper'
import { QueueHelper } from '@e2e/fixtures/helpers/QueueHelper'
import { SettingsHelper } from '@e2e/fixtures/helpers/SettingsHelper'
import { SubgraphHelper } from '@e2e/fixtures/helpers/SubgraphHelper'
import { ToastHelper } from '@e2e/fixtures/helpers/ToastHelper'
import { WorkflowHelper } from '@e2e/fixtures/helpers/WorkflowHelper'
} from './components/SidebarTab'
import { Topbar } from './components/Topbar'
import { AssetsHelper } from './helpers/AssetsHelper'
import { CanvasHelper } from './helpers/CanvasHelper'
import { PerformanceHelper } from './helpers/PerformanceHelper'
import { QueueHelper } from './helpers/QueueHelper'
import { ClipboardHelper } from './helpers/ClipboardHelper'
import { CommandHelper } from './helpers/CommandHelper'
import { DragDropHelper } from './helpers/DragDropHelper'
import { FeatureFlagHelper } from './helpers/FeatureFlagHelper'
import { KeyboardHelper } from './helpers/KeyboardHelper'
import { NodeOperationsHelper } from './helpers/NodeOperationsHelper'
import { SettingsHelper } from './helpers/SettingsHelper'
import { AppModeHelper } from './helpers/AppModeHelper'
import { SubgraphHelper } from './helpers/SubgraphHelper'
import { ToastHelper } from './helpers/ToastHelper'
import { WorkflowHelper } from './helpers/WorkflowHelper'
import type { NodeReference } from './utils/litegraphUtils'
import { assetPath } from './utils/paths'
import type { WorkspaceStore } from '../types/globals'
dotenvConfig()
@@ -122,7 +125,7 @@ type KeysOfType<T, Match> = {
}[keyof T]
class ConfirmDialog {
public readonly root: Locator
private readonly root: Locator
public readonly delete: Locator
public readonly overwrite: Locator
public readonly reject: Locator
@@ -197,7 +200,6 @@ export class ComfyPage {
public readonly featureFlags: FeatureFlagHelper
public readonly command: CommandHelper
public readonly bottomPanel: BottomPanel
public readonly queuePanel: QueuePanel
public readonly perf: PerformanceHelper
public readonly assets: AssetsHelper
public readonly queue: QueueHelper
@@ -245,7 +247,6 @@ export class ComfyPage {
this.featureFlags = new FeatureFlagHelper(page)
this.command = new CommandHelper(page)
this.bottomPanel = new BottomPanel(page)
this.queuePanel = new QueuePanel(page)
this.perf = new PerformanceHelper(page)
this.assets = new AssetsHelper(page)
this.queue = new QueueHelper(page)
@@ -359,7 +360,7 @@ export class ComfyPage {
}
async delay(ms: number) {
return sleep(ms)
return new Promise((resolve) => setTimeout(resolve, ms))
}
/**
@@ -477,4 +478,49 @@ export const comfyPageFixture = base.extend<{
}
})
export { comfyExpect }
const makeMatcher = function <T>(
getValue: (node: NodeReference) => Promise<T> | T,
type: string
) {
return async function (
this: ExpectMatcherState,
node: NodeReference,
options?: { timeout?: number; intervals?: number[] }
) {
const value = await getValue(node)
let assertion = expect(
value,
'Node is ' + (this.isNot ? '' : 'not ') + type
)
if (this.isNot) {
assertion = assertion.not
}
await expect(async () => {
assertion.toBeTruthy()
}).toPass({ timeout: 250, ...options })
return {
pass: !this.isNot,
message: () => 'Node is ' + (this.isNot ? 'not ' : '') + type
}
}
}
export const comfyExpect = expect.extend({
toBePinned: makeMatcher((n) => n.isPinned(), 'pinned'),
toBeBypassed: makeMatcher((n) => n.isBypassed(), 'bypassed'),
toBeCollapsed: makeMatcher((n) => n.isCollapsed(), 'collapsed'),
async toHaveFocus(locator: Locator, options = { timeout: 256 }) {
const isFocused = await locator.evaluate(
(el) => el === document.activeElement
)
await expect(async () => {
expect(isFocused).toBe(!this.isNot)
}).toPass(options)
return {
pass: isFocused,
message: () => `Expected element to ${isFocused ? 'not ' : ''}be focused.`
}
}
})

View File

@@ -1,24 +0,0 @@
import type { Locator, Page } from '@playwright/test'
import { comfyExpect as expect } from '../ComfyPage'
import { TestIds } from '../selectors'
export class QueuePanel {
readonly overlayToggle: Locator
readonly moreOptionsButton: Locator
constructor(readonly page: Page) {
this.overlayToggle = page.getByTestId(TestIds.queue.overlayToggle)
this.moreOptionsButton = page.getByLabel(/More options/i).first()
}
async openClearHistoryDialog() {
await this.moreOptionsButton.click()
const clearHistoryAction = this.page.getByTestId(
TestIds.queue.clearHistoryAction
)
await expect(clearHistoryAction).toBeVisible()
await clearHistoryAction.click()
}
}

View File

@@ -1,5 +1,4 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import type { WorkspaceStore } from '../../types/globals'
import { TestIds } from '../selectors'
@@ -175,8 +174,6 @@ export class AssetsSidebarTab extends SidebarTab {
super(page, 'assets')
}
// --- Tab navigation ---
get generatedTab() {
return this.page.getByRole('tab', { name: 'Generated' })
}
@@ -185,8 +182,6 @@ export class AssetsSidebarTab extends SidebarTab {
return this.page.getByRole('tab', { name: 'Imported' })
}
// --- Empty state ---
get emptyStateMessage() {
return this.page.getByText(
'Upload files or generate content to see them here'
@@ -197,169 +192,8 @@ export class AssetsSidebarTab extends SidebarTab {
return this.page.getByText(title)
}
// --- Search & filter ---
get searchInput() {
return this.page.getByPlaceholder('Search Assets...')
}
get settingsButton() {
return this.page.getByRole('button', { name: 'View settings' })
}
// --- View mode ---
get listViewOption() {
return this.page.getByText('List view')
}
get gridViewOption() {
return this.page.getByText('Grid view')
}
// --- Sort options (cloud-only, shown inside settings popover) ---
get sortNewestFirst() {
return this.page.getByText('Newest first')
}
get sortOldestFirst() {
return this.page.getByText('Oldest first')
}
// --- Asset cards ---
get assetCards() {
return this.page.locator('[role="button"][data-selected]')
}
getAssetCardByName(name: string) {
return this.page.locator('[role="button"][data-selected]', {
hasText: name
})
}
get selectedCards() {
return this.page.locator('[data-selected="true"]')
}
// --- List view items ---
get listViewItems() {
return this.page.locator(
'.sidebar-content-container [role="button"][tabindex="0"]'
)
}
// --- Selection footer ---
get selectionFooter() {
return this.page
.locator('.sidebar-content-container')
.locator('..')
.locator('[class*="h-18"]')
}
get selectionCountButton() {
return this.page.getByText(/Assets Selected: \d+/)
}
get deselectAllButton() {
return this.page.getByText('Deselect all')
}
get deleteSelectedButton() {
return this.page
.getByTestId('assets-delete-selected')
.or(this.page.locator('button:has(.icon-\\[lucide--trash-2\\])').last())
.first()
}
get downloadSelectedButton() {
return this.page
.getByTestId('assets-download-selected')
.or(this.page.locator('button:has(.icon-\\[lucide--download\\])').last())
.first()
}
// --- Context menu ---
contextMenuItem(label: string) {
return this.page.locator('.p-contextmenu').getByText(label)
}
// --- Folder view ---
get backToAssetsButton() {
return this.page.getByText('Back to all assets')
}
// --- Loading ---
get skeletonLoaders() {
return this.page.locator('.sidebar-content-container .animate-pulse')
}
// --- Helpers ---
override async open() {
// Remove any toast notifications that may overlay the sidebar button
await this.dismissToasts()
await super.open()
await this.generatedTab.waitFor({ state: 'visible' })
}
/** Dismiss all visible toast notifications by clicking their close buttons. */
async dismissToasts() {
const closeButtons = this.page.locator('.p-toast-close-button')
for (const btn of await closeButtons.all()) {
await btn.click({ force: true }).catch(() => {})
}
// Wait for all toast elements to fully animate out and detach from DOM
await expect(this.page.locator('.p-toast-message'))
.toHaveCount(0, { timeout: 5000 })
.catch(() => {})
}
async switchToImported() {
await this.dismissToasts()
await this.importedTab.click()
await expect(this.importedTab).toHaveAttribute('aria-selected', 'true', {
timeout: 3000
})
}
async switchToGenerated() {
await this.dismissToasts()
await this.generatedTab.click()
await expect(this.generatedTab).toHaveAttribute('aria-selected', 'true', {
timeout: 3000
})
}
async openSettingsMenu() {
await this.dismissToasts()
await this.settingsButton.click()
// Wait for popover content to render
await this.listViewOption
.or(this.gridViewOption)
.first()
.waitFor({ state: 'visible', timeout: 3000 })
}
async rightClickAsset(name: string) {
const card = this.getAssetCardByName(name)
await card.click({ button: 'right' })
await this.page
.locator('.p-contextmenu')
.waitFor({ state: 'visible', timeout: 3000 })
}
async waitForAssets(count?: number) {
if (count !== undefined) {
await expect(this.assetCards).toHaveCount(count, { timeout: 5000 })
} else {
await this.assetCards.first().waitFor({ state: 'visible', timeout: 5000 })
}
}
}

View File

@@ -1,77 +0,0 @@
import type { Locator, Page } from '@playwright/test'
import { BaseDialog } from './BaseDialog'
export class SignInDialog extends BaseDialog {
readonly emailInput: Locator
readonly passwordInput: Locator
readonly signInButton: Locator
readonly forgotPasswordLink: Locator
readonly apiKeyButton: Locator
readonly termsLink: Locator
readonly privacyLink: Locator
constructor(page: Page) {
super(page)
this.emailInput = this.root.locator('#comfy-org-sign-in-email')
this.passwordInput = this.root.locator('#comfy-org-sign-in-password')
this.signInButton = this.root.getByRole('button', { name: 'Sign in' })
this.forgotPasswordLink = this.root.getByText('Forgot password?')
this.apiKeyButton = this.root.getByRole('button', {
name: 'Comfy API Key'
})
this.termsLink = this.root.getByRole('link', { name: 'Terms of Use' })
this.privacyLink = this.root.getByRole('link', { name: 'Privacy Policy' })
}
async open() {
await this.page.evaluate(() => {
void window.app!.extensionManager.dialog.showSignInDialog()
})
await this.waitForVisible()
}
get heading() {
return this.root.getByRole('heading').first()
}
get signUpLink() {
return this.root.getByText('Sign up', { exact: true })
}
get signInLink() {
return this.root.getByText('Sign in', { exact: true })
}
get signUpEmailInput() {
return this.root.locator('#comfy-org-sign-up-email')
}
get signUpPasswordInput() {
return this.root.locator('#comfy-org-sign-up-password')
}
get signUpConfirmPasswordInput() {
return this.root.locator('#comfy-org-sign-up-confirm-password')
}
get signUpButton() {
return this.root.getByRole('button', { name: 'Sign up', exact: true })
}
get apiKeyHeading() {
return this.root.getByRole('heading', { name: 'API Key' })
}
get apiKeyInput() {
return this.root.locator('#comfy-org-api-key')
}
get backButton() {
return this.root.getByRole('button', { name: 'Back' })
}
get dividerText() {
return this.root.getByText('Or continue with')
}
}

View File

@@ -1,41 +0,0 @@
# Mock Data Fixtures
Deterministic mock data for browser (Playwright) tests. Each fixture
exports typed objects that conform to generated types from
`packages/ingest-types` or Zod schemas in `src/schemas/`.
## Usage with `page.route()`
> **Note:** `comfyPageFixture` navigates to the app during `setup()`,
> before the test body runs. Routes must be registered before navigation
> to intercept initial page-load requests. Set up routes in a custom
> fixture or `test.beforeEach` that runs before `comfyPage.setup()`.
```ts
import { createMockNodeDefinitions } from '../fixtures/data/nodeDefinitions'
import { mockSystemStats } from '../fixtures/data/systemStats'
// Extend the base set with test-specific nodes
const nodeDefs = createMockNodeDefinitions({
MyCustomNode: {
/* ... */
}
})
await page.route('**/api/object_info', (route) =>
route.fulfill({ json: nodeDefs })
)
await page.route('**/api/system_stats', (route) =>
route.fulfill({ json: mockSystemStats })
)
```
## Adding new fixtures
1. Locate the generated type in `packages/ingest-types` or Zod schema
in `src/schemas/` for the endpoint you need.
2. Create a new `.ts` file here that imports and satisfies the
corresponding TypeScript type.
3. Keep values realistic but stable — avoid dates, random IDs, or
values that would cause test flakiness.

View File

@@ -1,155 +0,0 @@
import type { ComfyNodeDef } from '@/schemas/nodeDefSchema'
/**
* Base node definitions covering the default workflow.
* Use {@link createMockNodeDefinitions} to extend with per-test overrides.
*/
const baseNodeDefinitions: Record<string, ComfyNodeDef> = {
KSampler: {
input: {
required: {
model: ['MODEL', {}],
seed: [
'INT',
{
default: 0,
min: 0,
max: 0xfffffffffffff,
control_after_generate: true
}
],
steps: ['INT', { default: 20, min: 1, max: 10000 }],
cfg: ['FLOAT', { default: 8.0, min: 0.0, max: 100.0, step: 0.1 }],
sampler_name: [['euler', 'euler_ancestral', 'heun', 'dpm_2'], {}],
scheduler: [['normal', 'karras', 'exponential', 'simple'], {}],
positive: ['CONDITIONING', {}],
negative: ['CONDITIONING', {}],
latent_image: ['LATENT', {}]
},
optional: {
denoise: ['FLOAT', { default: 1.0, min: 0.0, max: 1.0, step: 0.01 }]
}
},
output: ['LATENT'],
output_is_list: [false],
output_name: ['LATENT'],
name: 'KSampler',
display_name: 'KSampler',
description: 'Samples latents using the provided model and conditioning.',
category: 'sampling',
output_node: false,
python_module: 'nodes',
deprecated: false,
experimental: false
},
CheckpointLoaderSimple: {
input: {
required: {
ckpt_name: [
['v1-5-pruned.safetensors', 'sd_xl_base_1.0.safetensors'],
{}
]
}
},
output: ['MODEL', 'CLIP', 'VAE'],
output_is_list: [false, false, false],
output_name: ['MODEL', 'CLIP', 'VAE'],
name: 'CheckpointLoaderSimple',
display_name: 'Load Checkpoint',
description: 'Loads a diffusion model checkpoint.',
category: 'loaders',
output_node: false,
python_module: 'nodes',
deprecated: false,
experimental: false
},
CLIPTextEncode: {
input: {
required: {
text: ['STRING', { multiline: true, dynamicPrompts: true }],
clip: ['CLIP', {}]
}
},
output: ['CONDITIONING'],
output_is_list: [false],
output_name: ['CONDITIONING'],
name: 'CLIPTextEncode',
display_name: 'CLIP Text Encode (Prompt)',
description: 'Encodes a text prompt using a CLIP model.',
category: 'conditioning',
output_node: false,
python_module: 'nodes',
deprecated: false,
experimental: false
},
EmptyLatentImage: {
input: {
required: {
width: ['INT', { default: 512, min: 16, max: 16384, step: 8 }],
height: ['INT', { default: 512, min: 16, max: 16384, step: 8 }],
batch_size: ['INT', { default: 1, min: 1, max: 4096 }]
}
},
output: ['LATENT'],
output_is_list: [false],
output_name: ['LATENT'],
name: 'EmptyLatentImage',
display_name: 'Empty Latent Image',
description: 'Creates an empty latent image of the specified dimensions.',
category: 'latent',
output_node: false,
python_module: 'nodes',
deprecated: false,
experimental: false
},
VAEDecode: {
input: {
required: {
samples: ['LATENT', {}],
vae: ['VAE', {}]
}
},
output: ['IMAGE'],
output_is_list: [false],
output_name: ['IMAGE'],
name: 'VAEDecode',
display_name: 'VAE Decode',
description: 'Decodes latent images back into pixel space.',
category: 'latent',
output_node: false,
python_module: 'nodes',
deprecated: false,
experimental: false
},
SaveImage: {
input: {
required: {
images: ['IMAGE', {}],
filename_prefix: ['STRING', { default: 'ComfyUI' }]
}
},
output: [],
output_is_list: [],
output_name: [],
name: 'SaveImage',
display_name: 'Save Image',
description: 'Saves images to the output directory.',
category: 'image',
output_node: true,
python_module: 'nodes',
deprecated: false,
experimental: false
}
}
export function createMockNodeDefinitions(
overrides?: Record<string, ComfyNodeDef>
): Record<string, ComfyNodeDef> {
const base = structuredClone(baseNodeDefinitions)
return overrides ? { ...base, ...overrides } : base
}

View File

@@ -1,22 +0,0 @@
import type { SystemStatsResponse } from '@comfyorg/ingest-types'
export const mockSystemStats: SystemStatsResponse = {
system: {
os: 'posix',
python_version: '3.11.9 (main, Apr 2 2024, 08:25:04) [GCC 13.2.0]',
embedded_python: false,
comfyui_version: '0.3.10',
pytorch_version: '2.4.0+cu124',
argv: ['main.py', '--listen', '0.0.0.0'],
ram_total: 67108864000,
ram_free: 52428800000
},
devices: [
{
name: 'NVIDIA GeForce RTX 4090',
type: 'cuda',
vram_total: 25769803776,
vram_free: 23622320128
}
]
}

View File

@@ -5,63 +5,6 @@ import type { RawJobListItem } from '../../../src/platform/remote/comfyui/jobs/j
const jobsListRoutePattern = /\/api\/jobs(?:\?.*)?$/
const inputFilesRoutePattern = /\/internal\/files\/input(?:\?.*)?$/
/** Factory to create a mock completed job with preview output. */
export function createMockJob(
overrides: Partial<RawJobListItem> & { id: string }
): RawJobListItem {
const now = Date.now() / 1000
return {
status: 'completed',
create_time: now,
execution_start_time: now,
execution_end_time: now + 5,
preview_output: {
filename: `output_${overrides.id}.png`,
subfolder: '',
type: 'output',
nodeId: '1',
mediaType: 'images'
},
outputs_count: 1,
priority: 0,
...overrides
}
}
/** Create multiple mock jobs with sequential IDs and staggered timestamps. */
export function createMockJobs(
count: number,
baseOverrides?: Partial<RawJobListItem>
): RawJobListItem[] {
const now = Date.now() / 1000
return Array.from({ length: count }, (_, i) =>
createMockJob({
id: `job-${String(i + 1).padStart(3, '0')}`,
create_time: now - i * 60,
execution_start_time: now - i * 60,
execution_end_time: now - i * 60 + 5 + i,
preview_output: {
filename: `image_${String(i + 1).padStart(3, '0')}.png`,
subfolder: '',
type: 'output',
nodeId: '1',
mediaType: 'images'
},
...baseOverrides
})
)
}
/** Create mock imported file names with various media types. */
export function createMockImportedFiles(count: number): string[] {
const extensions = ['png', 'jpg', 'mp4', 'wav', 'glb', 'txt']
return Array.from(
{ length: count },
(_, i) =>
`imported_${String(i + 1).padStart(3, '0')}.${extensions[i % extensions.length]}`
)
}
function parseLimit(url: URL, total: number): number {
const value = Number(url.searchParams.get('limit'))
if (!Number.isInteger(value) || value <= 0) {

View File

@@ -147,16 +147,6 @@ export class WorkflowHelper {
})
}
async waitForWorkflowIdle(timeout = 5000): Promise<void> {
await this.comfyPage.page.waitForFunction(
() =>
!(window.app?.extensionManager as WorkspaceStore | undefined)?.workflow
?.isBusy,
undefined,
{ timeout }
)
}
async getExportedWorkflow(options: { api: true }): Promise<ComfyApiWorkflow>
async getExportedWorkflow(options?: {
api?: false

View File

@@ -98,10 +98,6 @@ export const TestIds = {
user: {
currentUserIndicator: 'current-user-indicator'
},
queue: {
overlayToggle: 'queue-overlay-toggle',
clearHistoryAction: 'clear-history-action'
},
errors: {
imageLoadError: 'error-loading-image',
videoLoadError: 'error-loading-video'
@@ -130,5 +126,4 @@ export type TestIdValue =
(id: string) => string
>
| (typeof TestIds.user)[keyof typeof TestIds.user]
| (typeof TestIds.queue)[keyof typeof TestIds.queue]
| (typeof TestIds.errors)[keyof typeof TestIds.errors]

View File

@@ -1,49 +0,0 @@
import type { ExpectMatcherState, Locator } from '@playwright/test'
import { expect } from '@playwright/test'
import type { NodeReference } from './litegraphUtils'
function makeMatcher<T>(
getValue: (node: NodeReference) => Promise<T> | T,
type: string
) {
return async function (
this: ExpectMatcherState,
node: NodeReference,
options?: { timeout?: number; intervals?: number[] }
) {
await expect(async () => {
const value = await getValue(node)
const assertion = this.isNot
? expect(value, 'Node is ' + type).not
: expect(value, 'Node is not ' + type)
assertion.toBeTruthy()
}).toPass({ timeout: 250, ...options })
return {
pass: !this.isNot,
message: () => 'Node is ' + (this.isNot ? 'not ' : '') + type
}
}
}
export const comfyExpect = expect.extend({
toBePinned: makeMatcher((n) => n.isPinned(), 'pinned'),
toBeBypassed: makeMatcher((n) => n.isBypassed(), 'bypassed'),
toBeCollapsed: makeMatcher((n) => n.isCollapsed(), 'collapsed'),
async toHaveFocus(locator: Locator, options = { timeout: 256 }) {
await expect
.poll(
() => locator.evaluate((el) => el === document.activeElement),
options
)
.toBe(!this.isNot)
const isFocused = await locator.evaluate(
(el) => el === document.activeElement
)
return {
pass: isFocused,
message: () => `Expected element to ${isFocused ? 'not ' : ''}be focused.`
}
}
})

View File

@@ -1,3 +0,0 @@
export function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms))
}

View File

@@ -2,8 +2,8 @@ import type { Response } from '@playwright/test'
import { expect, mergeTests } from '@playwright/test'
import type { StatusWsMessage } from '../../src/schemas/apiSchema'
import { comfyPageFixture } from '@e2e/fixtures/ComfyPage'
import { webSocketFixture } from '@e2e/fixtures/ws'
import { comfyPageFixture } from '../fixtures/ComfyPage'
import { webSocketFixture } from '../fixtures/ws'
import type { WorkspaceStore } from '../types/globals'
const test = mergeTests(comfyPageFixture, webSocketFixture)

View File

@@ -1,126 +0,0 @@
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../../fixtures/ComfyPage'
test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.setup()
await comfyPage.queuePanel.overlayToggle.click()
})
test('Dialog opens from queue panel history actions menu', async ({
comfyPage
}) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
})
test('Dialog shows confirmation message with title, description, and assets note', async ({
comfyPage
}) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
await expect(
dialog.getByText('Clear your job queue history?')
).toBeVisible()
await expect(
dialog.getByText(
'All the finished or failed jobs below will be removed from this Job queue panel.'
)
).toBeVisible()
await expect(
dialog.getByText(
'Assets generated by these jobs won\u2019t be deleted and can always be viewed from the assets panel.'
)
).toBeVisible()
})
test('Cancel button closes dialog without clearing history', async ({
comfyPage
}) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
let clearCalled = false
await comfyPage.page.route('**/api/history', (route) => {
if (route.request().method() === 'POST') {
clearCalled = true
}
return route.continue()
})
await dialog.getByRole('button', { name: 'Cancel' }).click()
await expect(dialog).not.toBeVisible()
expect(clearCalled).toBe(false)
await comfyPage.page.unroute('**/api/history')
})
test('Close (X) button closes dialog without clearing history', async ({
comfyPage
}) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
let clearCalled = false
await comfyPage.page.route('**/api/history', (route) => {
if (route.request().method() === 'POST') {
clearCalled = true
}
return route.continue()
})
await dialog.getByLabel('Close').click()
await expect(dialog).not.toBeVisible()
expect(clearCalled).toBe(false)
await comfyPage.page.unroute('**/api/history')
})
test('Confirm clears queue history and closes dialog', async ({
comfyPage
}) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
const clearPromise = comfyPage.page.waitForRequest(
(req) => req.url().includes('/api/history') && req.method() === 'POST'
)
await dialog.getByRole('button', { name: 'Clear' }).click()
const request = await clearPromise
expect(request.postDataJSON()).toEqual({ clear: true })
await expect(dialog).not.toBeVisible()
})
test('Dialog state resets after close and reopen', async ({ comfyPage }) => {
await comfyPage.queuePanel.openClearHistoryDialog()
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
await dialog.getByRole('button', { name: 'Cancel' }).click()
await expect(dialog).not.toBeVisible()
await comfyPage.queuePanel.openClearHistoryDialog()
await expect(dialog).toBeVisible()
const clearButton = dialog.getByRole('button', { name: 'Clear' })
await expect(clearButton).toBeVisible()
await expect(clearButton).toBeEnabled()
})
})

View File

@@ -1,94 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { SignInDialog } from '../../fixtures/components/SignInDialog'
test.describe('Sign In dialog', { tag: '@ui' }, () => {
let dialog: SignInDialog
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
dialog = new SignInDialog(comfyPage.page)
await dialog.open()
})
test('Should open and show the sign-in form by default', async () => {
await expect(
dialog.root.getByRole('heading', { name: 'Log in to your account' })
).toBeVisible()
await expect(dialog.emailInput).toBeVisible()
await expect(dialog.passwordInput).toBeVisible()
await expect(dialog.signInButton).toBeVisible()
})
test('Should toggle from sign-in to sign-up form', async () => {
await dialog.signUpLink.click()
await expect(
dialog.root.getByRole('heading', { name: 'Create an account' })
).toBeVisible()
await expect(dialog.signUpEmailInput).toBeVisible()
await expect(dialog.signUpPasswordInput).toBeVisible()
await expect(dialog.signUpConfirmPasswordInput).toBeVisible()
await expect(dialog.signUpButton).toBeVisible()
})
test('Should toggle back from sign-up to sign-in form', async () => {
await dialog.signUpLink.click()
await expect(
dialog.root.getByRole('heading', { name: 'Create an account' })
).toBeVisible()
await dialog.signInLink.click()
await expect(
dialog.root.getByRole('heading', { name: 'Log in to your account' })
).toBeVisible()
await expect(dialog.emailInput).toBeVisible()
await expect(dialog.passwordInput).toBeVisible()
})
test('Should navigate to the API Key form and back', async () => {
await dialog.apiKeyButton.click()
await expect(dialog.apiKeyHeading).toBeVisible()
await expect(dialog.apiKeyInput).toBeVisible()
await dialog.backButton.click()
await expect(
dialog.root.getByRole('heading', { name: 'Log in to your account' })
).toBeVisible()
})
test('Should display Terms of Service and Privacy Policy links', async () => {
await expect(dialog.termsLink).toBeVisible()
await expect(dialog.termsLink).toHaveAttribute(
'href',
'https://www.comfy.org/terms-of-service'
)
await expect(dialog.privacyLink).toBeVisible()
await expect(dialog.privacyLink).toHaveAttribute(
'href',
'https://www.comfy.org/privacy'
)
})
test('Should display the "Or continue with" divider and API key button', async () => {
await expect(dialog.dividerText).toBeVisible()
await expect(dialog.apiKeyButton).toBeVisible()
})
test('Should show forgot password link on sign-in form', async () => {
await expect(dialog.forgotPasswordLink).toBeVisible()
})
test('Should close dialog via close button', async () => {
await dialog.close()
await expect(dialog.root).toBeHidden()
})
test('Should close dialog via Escape key', async ({ comfyPage }) => {
await comfyPage.page.keyboard.press('Escape')
await expect(dialog.root).toBeHidden()
})
})

View File

@@ -0,0 +1,429 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import type { ComfyPage } from '../fixtures/ComfyPage'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
// ---- Helpers ---------------------------------------------------------------
function createTestImageDataUrl(
width: number,
height: number,
color: string
): string {
const svg =
`<svg xmlns="http://www.w3.org/2000/svg" width="${width}" height="${height}">` +
`<rect width="${width}" height="${height}" fill="${color}"/>` +
`</svg>`
return `data:image/svg+xml;charset=utf-8,${encodeURIComponent(svg)}`
}
type Bounds = { x: number; y: number; width: number; height: number }
/**
* Injects an image into the node output store for the given source node,
* simulating what happens after a node executes and produces image output.
*/
async function injectSourceImage(
page: Page,
sourceNodeId: number,
dataUrl: string
): Promise<void> {
await page.evaluate(
({ nodeId, url }) => {
type NodeOutputStore = {
setNodePreviewsByNodeId: (id: number, urls: string[]) => void
}
type VueAppElement = HTMLElement & {
__vue_app__: {
config: {
globalProperties: {
$pinia: { _s: Map<string, NodeOutputStore> }
}
}
}
}
const el = document.getElementById('vue-app') as unknown as VueAppElement
const store =
el.__vue_app__.config.globalProperties.$pinia._s.get('nodeOutput')!
store.setNodePreviewsByNodeId(nodeId, [url])
},
{ nodeId: sourceNodeId, url: dataUrl }
)
}
async function setCropState(page: Page, bounds: Bounds): Promise<void> {
await page.evaluate((bounds) => {
type BoundsValue = { x: number; y: number; width: number; height: number }
const node = window.app!.graph.getNodeById(1)
const widget = node?.widgets?.find((w) => w.type === 'imagecrop')
if (widget?.value) {
const value = widget.value as unknown as BoundsValue
value.x = bounds.x
value.y = bounds.y
value.width = bounds.width
value.height = bounds.height
widget.callback?.(widget.value)
}
}, bounds)
}
async function getCropState(page: Page): Promise<Bounds> {
return page.evaluate(() => {
type BoundsValue = { x: number; y: number; width: number; height: number }
const node = window.app!.graph.getNodeById(1)
const widget = node?.widgets?.find((w) => w.type === 'imagecrop')
const v = widget?.value as unknown as BoundsValue
return { x: v.x, y: v.y, width: v.width, height: v.height }
})
}
/**
* Injects a test image into the source node and waits for the crop widget to
* be fully ready: image loaded, scale factor computed, crop box visible.
*/
async function setupWithImage(
comfyPage: ComfyPage,
imageWidth: number,
imageHeight: number,
initialBounds: Bounds
): Promise<void> {
await injectSourceImage(
comfyPage.page,
2,
createTestImageDataUrl(imageWidth, imageHeight, 'steelblue')
)
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img')).toBeVisible()
await comfyPage.page.waitForFunction(() => {
const img = document.querySelector(
'[data-node-id="1"] img'
) as HTMLImageElement | null
return (img?.complete ?? false) && (img?.naturalWidth ?? 0) > 0
})
await setCropState(comfyPage.page, initialBounds)
await comfyPage.nextFrame()
await expect(node.locator('.cursor-move')).toBeVisible()
}
/**
* Returns a locator for one of the 8 resize handles on the crop widget.
* Handles are ordered in DOM as: top, bottom, left, right, nw, ne, sw, se.
*/
function getResizeHandle(
nodeLocator: Locator,
direction: 'top' | 'bottom' | 'left' | 'right' | 'nw' | 'ne' | 'sw' | 'se'
): Locator {
switch (direction) {
case 'top':
return nodeLocator.locator('.cursor-ns-resize').first()
case 'bottom':
return nodeLocator.locator('.cursor-ns-resize').last()
case 'left':
return nodeLocator.locator('.cursor-ew-resize').first()
case 'right':
return nodeLocator.locator('.cursor-ew-resize').last()
case 'nw':
return nodeLocator.locator('.cursor-nwse-resize').first()
case 'se':
return nodeLocator.locator('.cursor-nwse-resize').last()
case 'ne':
return nodeLocator.locator('.cursor-nesw-resize').first()
case 'sw':
return nodeLocator.locator('.cursor-nesw-resize').last()
}
}
async function dragFrom(
page: Page,
locator: Locator,
deltaX: number,
deltaY: number
): Promise<void> {
const box = await locator.boundingBox()
expect(box).not.toBeNull()
const startX = box!.x + box!.width / 2
const startY = box!.y + box!.height / 2
await page.mouse.move(startX, startY)
await page.mouse.down()
await page.mouse.move(startX + deltaX, startY + deltaY, { steps: 10 })
await page.mouse.up()
}
// ---- Tests -----------------------------------------------------------------
test.describe('Image Crop', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.workflow.loadWorkflow('widgets/image_crop_widget')
await comfyPage.vueNodes.waitForNodes()
})
test(
'shows empty state when no input image is available',
{ tag: '@smoke' },
async ({ comfyPage }) => {
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node).toBeVisible()
await expect(node).toContainText('No input image connected')
await expect(node.locator('.cursor-move')).not.toBeVisible()
}
)
test(
'enforces minimum node size of 300×450',
{ tag: '@node' },
async ({ comfyPage }) => {
const size = await comfyPage.page.evaluate(() => {
const node = window.app!.graph.getNodeById(1)
return node?.size as [number, number]
})
expect(size[0]).toBeGreaterThanOrEqual(300)
expect(size[1]).toBeGreaterThanOrEqual(450)
}
)
test.describe('drag', { tag: '@widget' }, () => {
test('moves the crop box', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 200,
height: 200
})
const cropBox = comfyPage.vueNodes
.getNodeLocator('1')
.locator('.cursor-move')
await dragFrom(comfyPage.page, cropBox, 100, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x).toBeGreaterThan(100)
expect(state.y).toBe(100)
expect(state.width).toBe(200)
expect(state.height).toBe(200)
})
test('clamps to right boundary', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 550,
y: 100,
width: 200,
height: 200
})
const cropBox = comfyPage.vueNodes
.getNodeLocator('1')
.locator('.cursor-move')
await dragFrom(comfyPage.page, cropBox, 500, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x + state.width).toBeLessThanOrEqual(800)
})
test('clamps to top-left boundary', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 50,
y: 50,
width: 200,
height: 200
})
const cropBox = comfyPage.vueNodes
.getNodeLocator('1')
.locator('.cursor-move')
await dragFrom(comfyPage.page, cropBox, -500, -500)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x).toBeGreaterThanOrEqual(0)
expect(state.y).toBeGreaterThanOrEqual(0)
})
test('does nothing when no image is loaded', async ({ comfyPage }) => {
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('.cursor-move')).not.toBeVisible()
const stateBefore = await getCropState(comfyPage.page)
const nodeBox = await node.boundingBox()
if (nodeBox) {
await comfyPage.page.mouse.click(
nodeBox.x + nodeBox.width / 2,
nodeBox.y + nodeBox.height / 2
)
}
await comfyPage.nextFrame()
const stateAfter = await getCropState(comfyPage.page)
expect(stateAfter).toEqual(stateBefore)
})
})
test.describe('free resize', { tag: '@widget' }, () => {
test('right edge increases width', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'right'), 80, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.width).toBeGreaterThan(200)
expect(state.x).toBe(100)
expect(state.y).toBe(100)
expect(state.height).toBe(200)
})
test('left edge adjusts x and width', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 200,
y: 100,
width: 300,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'left'), -80, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x).toBeLessThan(200)
expect(state.width).toBeGreaterThan(300)
expect(state.y).toBe(100)
expect(state.height).toBe(200)
})
test('bottom edge increases height', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'bottom'), 0, 80)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.height).toBeGreaterThan(200)
expect(state.x).toBe(100)
expect(state.y).toBe(100)
expect(state.width).toBe(200)
})
test('top edge adjusts y and height', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 200,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'top'), 0, -80)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.y).toBeLessThan(200)
expect(state.height).toBeGreaterThan(200)
expect(state.x).toBe(100)
expect(state.width).toBe(200)
})
test('SE corner increases width and height', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'se'), 80, 80)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.width).toBeGreaterThan(200)
expect(state.height).toBeGreaterThan(200)
expect(state.x).toBe(100)
expect(state.y).toBe(100)
})
test('NW corner adjusts x, y, width, and height', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 200,
y: 200,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await dragFrom(comfyPage.page, getResizeHandle(node, 'nw'), -80, -80)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x).toBeLessThan(200)
expect(state.y).toBeLessThan(200)
expect(state.width).toBeGreaterThan(200)
expect(state.height).toBeGreaterThan(200)
})
test('enforces minimum crop size of 16px', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 50,
height: 50
})
const node = comfyPage.vueNodes.getNodeLocator('1')
// Drag right edge far left to try to collapse width below the minimum
await dragFrom(comfyPage.page, getResizeHandle(node, 'right'), -500, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.width).toBeGreaterThanOrEqual(16)
})
test('clamps resize to image boundary', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 600,
y: 100,
width: 100,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
// Drag right edge far past the image right boundary
await dragFrom(comfyPage.page, getResizeHandle(node, 'right'), 500, 0)
await comfyPage.nextFrame()
const state = await getCropState(comfyPage.page)
expect(state.x + state.width).toBeLessThanOrEqual(800)
})
test('shows 8 handles when ratio is unlocked', async ({ comfyPage }) => {
await setupWithImage(comfyPage, 800, 600, {
x: 100,
y: 100,
width: 200,
height: 200
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const handles = node.locator(
'.cursor-ns-resize, .cursor-ew-resize, .cursor-nwse-resize, .cursor-nesw-resize'
)
await expect(handles).toHaveCount(8)
})
})
})

View File

@@ -1,34 +0,0 @@
# Properties Panel E2E Tests
Tests for the right-side properties panel (`RightSidePanel.vue`).
## Structure
| File | Coverage |
| --------------------------------- | ----------------------------------------------------------- |
| `openClose.spec.ts` | Panel open/close via actionbar and close button |
| `workflowOverview.spec.ts` | No-selection state: tabs, nodes list, global settings |
| `nodeSelection.spec.ts` | Single/multi-node selection, selection changes, tab labels |
| `titleEditing.spec.ts` | Node title editing via pencil icon |
| `searchFiltering.spec.ts` | Widget search and clear |
| `nodeSettings.spec.ts` | Settings tab: node state, color, pinned (requires VueNodes) |
| `infoTab.spec.ts` | Node help content |
| `errorsTab.spec.ts` | Errors tab visibility |
| `propertiesPanelPosition.spec.ts` | Panel position relative to sidebar |
## Shared Helper
`PropertiesPanelHelper.ts` — Encapsulates panel locators and actions. Instantiated in `beforeEach`:
```typescript
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
})
```
## Conventions
- Tests requiring VueNodes rendering enable it in `beforeEach` via `comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)` and call `comfyPage.vueNodes.waitForNodes()`.
- Verify node state changes via user-facing indicators (text labels like "Bypassed"/"Muted", pin indicator test IDs) rather than internal properties.
- Color changes are verified via `page.evaluate` accessing node properties, per the guidance in `docs/guidance/playwright.md`.

View File

@@ -1,100 +0,0 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import { TestIds } from '../../fixtures/selectors'
export class PropertiesPanelHelper {
readonly root: Locator
readonly panelTitle: Locator
readonly searchBox: Locator
readonly closeButton: Locator
constructor(readonly page: Page) {
this.root = page.getByTestId(TestIds.propertiesPanel.root)
this.panelTitle = this.root.locator('h3')
this.searchBox = this.root.getByPlaceholder(/^Search/)
this.closeButton = this.root.locator('button[aria-pressed]')
}
get tabs(): Locator {
return this.root.locator('nav button')
}
getTab(label: string): Locator {
return this.root.locator('nav button', { hasText: label })
}
get titleEditIcon(): Locator {
return this.panelTitle.locator('i[class*="lucide--pencil"]')
}
get titleInput(): Locator {
return this.root.getByTestId(TestIds.node.titleInput)
}
getNodeStateButton(state: 'Normal' | 'Bypass' | 'Mute'): Locator {
return this.root.locator('button', { hasText: state })
}
getColorSwatch(colorName: string): Locator {
return this.root.locator(`[data-testid="${colorName}"]`)
}
get pinnedSwitch(): Locator {
return this.root.locator('[data-p-checked]').first()
}
get subgraphEditButton(): Locator {
return this.root.locator('button:has(i[class*="lucide--settings-2"])')
}
get contentArea(): Locator {
return this.root.locator('.scrollbar-thin')
}
get errorsTabIcon(): Locator {
return this.root.locator('nav i[class*="lucide--octagon-alert"]')
}
get viewAllSettingsButton(): Locator {
return this.root.getByRole('button', { name: /view all settings/i })
}
get collapseToggleButton(): Locator {
return this.root.locator(
'button:has(i[class*="lucide--chevrons-down-up"]), button:has(i[class*="lucide--chevrons-up-down"])'
)
}
async open(actionbar: Locator): Promise<void> {
if (!(await this.root.isVisible())) {
await actionbar.click()
await expect(this.root).toBeVisible()
}
}
async close(): Promise<void> {
if (await this.root.isVisible()) {
await this.closeButton.click()
await expect(this.root).not.toBeVisible()
}
}
async switchToTab(label: string): Promise<void> {
await this.getTab(label).click()
}
async editTitle(newTitle: string): Promise<void> {
await this.titleEditIcon.click()
await this.titleInput.fill(newTitle)
await this.titleInput.press('Enter')
}
async searchWidgets(query: string): Promise<void> {
await this.searchBox.fill(query)
}
async clearSearch(): Promise<void> {
await this.searchBox.fill('')
}
}

View File

@@ -1,31 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Errors tab', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
})
test('should show Errors tab when errors exist', async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
'Comfy.RightSidePanel.ShowErrorsTab',
true
)
await comfyPage.workflow.loadWorkflow('missing/missing_nodes')
await comfyPage.actionbar.propertiesButton.click()
await comfyPage.nextFrame()
await expect(panel.errorsTabIcon).toBeVisible()
})
test('should not show Errors tab when errors are disabled', async ({
comfyPage
}) => {
await comfyPage.actionbar.propertiesButton.click()
await expect(panel.errorsTabIcon).not.toBeVisible()
})
})

View File

@@ -1,22 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Info tab', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.actionbar.propertiesButton.click()
await comfyPage.nodeOps.selectNodes(['KSampler'])
await panel.switchToTab('Info')
})
test('should show node help content', async () => {
await expect(panel.contentArea).toBeVisible()
await expect(
panel.contentArea.getByRole('heading', { name: 'Inputs' })
).toBeVisible()
})
})

View File

@@ -1,126 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Node selection', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.actionbar.propertiesButton.click()
})
test.describe('Single node', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.nodeOps.selectNodes(['KSampler'])
})
test('should show node title in panel header', async () => {
await expect(panel.panelTitle).toContainText('KSampler')
})
test('should show Parameters, Info, and Settings tabs', async () => {
await expect(panel.getTab('Parameters')).toBeVisible()
await expect(panel.getTab('Info')).toBeVisible()
await expect(panel.getTab('Settings')).toBeVisible()
})
test('should not show Nodes tab for single node', async () => {
await expect(panel.getTab('Nodes')).not.toBeVisible()
})
test('should display node widgets in Parameters tab', async () => {
await expect(panel.contentArea.getByText('seed')).toBeVisible()
await expect(panel.contentArea.getByText('steps')).toBeVisible()
})
})
test.describe('Multi-node', () => {
test('should show item count in title', async ({ comfyPage }) => {
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(panel.panelTitle).toContainText('3 items selected')
})
test('should list all selected nodes in Parameters tab', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(panel.root.getByText('KSampler')).toHaveCount(1)
await expect(
panel.root.getByText('CLIP Text Encode (Prompt)')
).toHaveCount(2)
})
test('should not show Info tab for multi-selection', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(panel.getTab('Info')).not.toBeVisible()
})
})
test.describe('Selection changes', () => {
test('should update from no selection to node selection', async ({
comfyPage
}) => {
await expect(panel.panelTitle).toContainText('Workflow Overview')
await comfyPage.nodeOps.selectNodes(['KSampler'])
await expect(panel.panelTitle).toContainText('KSampler')
})
test('should update from node selection back to no selection', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes(['KSampler'])
await expect(panel.panelTitle).toContainText('KSampler')
await comfyPage.page.evaluate(() => {
window.app!.canvas.deselectAll()
})
await comfyPage.nextFrame()
await expect(panel.panelTitle).toContainText('Workflow Overview')
})
test('should update between different single node selections', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes(['KSampler'])
await expect(panel.panelTitle).toContainText('KSampler')
await comfyPage.page.evaluate(() => {
window.app!.canvas.deselectAll()
})
await comfyPage.nextFrame()
await comfyPage.nodeOps.selectNodes(['Empty Latent Image'])
await expect(panel.panelTitle).toContainText('Empty Latent Image')
})
})
test.describe('Tab labels', () => {
test('should show "Parameters" tab for single node', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes(['KSampler'])
await expect(panel.getTab('Parameters')).toBeVisible()
})
test('should show "Nodes" tab label for multi-selection', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(panel.getTab('Nodes')).toBeVisible()
})
})
})

View File

@@ -1,122 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Node settings', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.vueNodes.waitForNodes()
await comfyPage.actionbar.propertiesButton.click()
await comfyPage.nodeOps.selectNodes(['KSampler'])
await panel.switchToTab('Settings')
})
test.describe('Node state', () => {
test('should show Normal, Bypass, and Mute state buttons', async () => {
await expect(panel.getNodeStateButton('Normal')).toBeVisible()
await expect(panel.getNodeStateButton('Bypass')).toBeVisible()
await expect(panel.getNodeStateButton('Mute')).toBeVisible()
})
test('should set node to Bypass mode', async ({ comfyPage }) => {
await panel.getNodeStateButton('Bypass').click()
const nodeLocator = comfyPage.vueNodes.getNodeByTitle('KSampler')
await expect(nodeLocator.getByText('Bypassed')).toBeVisible()
})
test('should set node to Mute mode', async ({ comfyPage }) => {
await panel.getNodeStateButton('Mute').click()
const nodeLocator = comfyPage.vueNodes.getNodeByTitle('KSampler')
await expect(nodeLocator.getByText('Muted')).toBeVisible()
})
test('should restore node to Normal mode', async ({ comfyPage }) => {
await panel.getNodeStateButton('Bypass').click()
const nodeLocator = comfyPage.vueNodes.getNodeByTitle('KSampler')
await expect(nodeLocator.getByText('Bypassed')).toBeVisible()
await panel.getNodeStateButton('Normal').click()
await expect(nodeLocator.getByText('Bypassed')).not.toBeVisible()
await expect(nodeLocator.getByText('Muted')).not.toBeVisible()
})
})
test.describe('Node color', () => {
test('should display color swatches', async () => {
await expect(panel.getColorSwatch('noColor')).toBeVisible()
await expect(panel.getColorSwatch('red')).toBeVisible()
await expect(panel.getColorSwatch('blue')).toBeVisible()
})
test('should apply color to node', async ({ comfyPage }) => {
await panel.getColorSwatch('red').click()
await expect
.poll(() =>
comfyPage.page.evaluate(() => {
const selected = window.app!.canvas.selected_nodes
const node = Object.values(selected)[0]
return node?.color != null
})
)
.toBe(true)
})
test('should remove color with noColor swatch', async ({ comfyPage }) => {
await panel.getColorSwatch('red').click()
await expect
.poll(() =>
comfyPage.page.evaluate(() => {
const selected = window.app!.canvas.selected_nodes
const node = Object.values(selected)[0]
return node?.color != null
})
)
.toBe(true)
await panel.getColorSwatch('noColor').click()
await expect
.poll(() =>
comfyPage.page.evaluate(() => {
const selected = window.app!.canvas.selected_nodes
const node = Object.values(selected)[0]
return node?.color
})
)
.toBeFalsy()
})
})
test.describe('Pinned state', () => {
test('should display pinned toggle', async () => {
await expect(panel.pinnedSwitch).toBeVisible()
})
test('should toggle pinned state', async ({ comfyPage }) => {
await panel.pinnedSwitch.click()
const nodeLocator = comfyPage.vueNodes.getNodeByTitle('KSampler')
await expect(nodeLocator.getByTestId('node-pin-indicator')).toBeVisible()
})
test('should unpin previously pinned node', async ({ comfyPage }) => {
const nodeLocator = comfyPage.vueNodes.getNodeByTitle('KSampler')
await panel.pinnedSwitch.click()
await expect(nodeLocator.getByTestId('node-pin-indicator')).toBeVisible()
await panel.pinnedSwitch.click()
await expect(
nodeLocator.getByTestId('node-pin-indicator')
).not.toBeVisible()
})
})
})

View File

@@ -1,32 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Open and close', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
})
test('should open via actionbar toggle button', async ({ comfyPage }) => {
await expect(panel.root).not.toBeVisible()
await comfyPage.actionbar.propertiesButton.click()
await expect(panel.root).toBeVisible()
})
test('should close via panel close button', async ({ comfyPage }) => {
await comfyPage.actionbar.propertiesButton.click()
await expect(panel.root).toBeVisible()
await panel.closeButton.click()
await expect(panel.root).not.toBeVisible()
})
test('should close via close button after opening', async ({ comfyPage }) => {
await comfyPage.actionbar.propertiesButton.click()
await expect(panel.root).toBeVisible()
await panel.close()
await expect(panel.root).not.toBeVisible()
})
})

View File

@@ -0,0 +1,36 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
test.describe('Properties panel', () => {
test('opens and updates title based on selection', async ({ comfyPage }) => {
await comfyPage.actionbar.propertiesButton.click()
const { propertiesPanel } = comfyPage.menu
await expect(propertiesPanel.panelTitle).toContainText('Workflow Overview')
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(propertiesPanel.panelTitle).toContainText('3 items selected')
await expect(propertiesPanel.root.getByText('KSampler')).toHaveCount(1)
await expect(
propertiesPanel.root.getByText('CLIP Text Encode (Prompt)')
).toHaveCount(2)
await propertiesPanel.searchBox.fill('seed')
await expect(propertiesPanel.root.getByText('KSampler')).toHaveCount(1)
await expect(
propertiesPanel.root.getByText('CLIP Text Encode (Prompt)')
).toHaveCount(0)
await propertiesPanel.searchBox.fill('')
await expect(propertiesPanel.root.getByText('KSampler')).toHaveCount(1)
await expect(
propertiesPanel.root.getByText('CLIP Text Encode (Prompt)')
).toHaveCount(2)
})
})

View File

@@ -1,41 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Search filtering', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.actionbar.propertiesButton.click()
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
})
test('should filter nodes by search query', async () => {
await panel.searchWidgets('seed')
await expect(panel.root.getByText('KSampler')).toHaveCount(1)
await expect(panel.root.getByText('CLIP Text Encode (Prompt)')).toHaveCount(
0
)
})
test('should restore all nodes when search is cleared', async () => {
await panel.searchWidgets('seed')
await panel.clearSearch()
await expect(panel.root.getByText('KSampler')).toHaveCount(1)
await expect(panel.root.getByText('CLIP Text Encode (Prompt)')).toHaveCount(
2
)
})
test('should show empty state for no matches', async () => {
await panel.searchWidgets('nonexistent_widget_xyz')
await expect(
panel.contentArea.getByText(/no .* match|no results|no items/i)
).toBeVisible()
})
})

View File

@@ -1,50 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Title editing', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.actionbar.propertiesButton.click()
await comfyPage.nodeOps.selectNodes(['KSampler'])
})
test('should show pencil icon for editable title', async () => {
await expect(panel.titleEditIcon).toBeVisible()
})
test('should enter edit mode on pencil click', async () => {
await panel.titleEditIcon.click()
await expect(panel.titleInput).toBeVisible()
})
test('should update node title on edit', async () => {
const newTitle = 'My Custom Sampler'
await panel.editTitle(newTitle)
await expect(panel.panelTitle).toContainText(newTitle)
})
test('should not show pencil icon for multi-selection', async ({
comfyPage
}) => {
await comfyPage.nodeOps.selectNodes([
'KSampler',
'CLIP Text Encode (Prompt)'
])
await expect(panel.titleEditIcon).not.toBeVisible()
})
test('should not show pencil icon when nothing is selected', async ({
comfyPage
}) => {
await comfyPage.page.evaluate(() => {
window.app!.canvas.deselectAll()
})
await comfyPage.nextFrame()
await expect(panel.panelTitle).toContainText('Workflow Overview')
await expect(panel.titleEditIcon).not.toBeVisible()
})
})

View File

@@ -1,70 +0,0 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import { PropertiesPanelHelper } from './PropertiesPanelHelper'
test.describe('Properties panel - Workflow Overview', () => {
let panel: PropertiesPanelHelper
test.beforeEach(async ({ comfyPage }) => {
panel = new PropertiesPanelHelper(comfyPage.page)
await comfyPage.actionbar.propertiesButton.click()
await expect(panel.root).toBeVisible()
})
test('should show "Workflow Overview" title when nothing is selected', async () => {
await expect(panel.panelTitle).toContainText('Workflow Overview')
})
test('should show Parameters, Nodes, and Settings tabs', async () => {
await expect(panel.getTab('Parameters')).toBeVisible()
await expect(panel.getTab('Nodes')).toBeVisible()
await expect(panel.getTab('Settings')).toBeVisible()
})
test('should not show Info tab when nothing is selected', async () => {
await expect(panel.getTab('Info')).not.toBeVisible()
})
test('should switch to Nodes tab and list all workflow nodes', async ({
comfyPage
}) => {
await panel.switchToTab('Nodes')
const nodeCount = await comfyPage.nodeOps.getNodeCount()
expect(nodeCount).toBeGreaterThan(0)
await expect(panel.contentArea.locator('text=KSampler')).toBeVisible()
})
test('should filter nodes by search in Nodes tab', async () => {
await panel.switchToTab('Nodes')
await panel.searchWidgets('KSampler')
await expect(panel.contentArea.getByText('KSampler').first()).toBeVisible()
})
test('should switch to Settings tab and show global settings', async () => {
await panel.switchToTab('Settings')
await expect(panel.viewAllSettingsButton).toBeVisible()
})
test('should show "View all settings" button', async () => {
await panel.switchToTab('Settings')
await expect(panel.viewAllSettingsButton).toBeVisible()
})
test('should show Nodes section with toggles', async () => {
await panel.switchToTab('Settings')
await expect(
panel.contentArea.getByRole('button', { name: 'NODES' })
).toBeVisible()
})
test('should show Canvas section with grid settings', async () => {
await panel.switchToTab('Settings')
await expect(panel.contentArea.getByText('Canvas')).toBeVisible()
})
test('should show Connection Links section', async () => {
await panel.switchToTab('Settings')
await expect(panel.contentArea.getByText('Connection Links')).toBeVisible()
})
})

View File

@@ -1,72 +1,8 @@
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../../fixtures/ComfyPage'
import {
createMockJob,
createMockJobs
} from '../../fixtures/helpers/AssetsHelper'
import type { RawJobListItem } from '../../../src/platform/remote/comfyui/jobs/jobTypes'
// ---------------------------------------------------------------------------
// Shared fixtures
// ---------------------------------------------------------------------------
const SAMPLE_JOBS: RawJobListItem[] = [
createMockJob({
id: 'job-alpha',
create_time: 1000,
execution_start_time: 1000,
execution_end_time: 1010,
preview_output: {
filename: 'landscape.png',
subfolder: '',
type: 'output',
nodeId: '1',
mediaType: 'images'
},
outputs_count: 1
}),
createMockJob({
id: 'job-beta',
create_time: 2000,
execution_start_time: 2000,
execution_end_time: 2003,
preview_output: {
filename: 'portrait.png',
subfolder: '',
type: 'output',
nodeId: '2',
mediaType: 'images'
},
outputs_count: 1
}),
createMockJob({
id: 'job-gamma',
create_time: 3000,
execution_start_time: 3000,
execution_end_time: 3020,
preview_output: {
filename: 'abstract_art.png',
subfolder: '',
type: 'output',
nodeId: '3',
mediaType: 'images'
},
outputs_count: 2
})
]
const SAMPLE_IMPORTED_FILES = [
'reference_photo.png',
'background.jpg',
'audio_clip.wav'
]
// ==========================================================================
// 1. Empty states
// ==========================================================================
test.describe('Assets sidebar - empty states', () => {
test.describe('Assets sidebar', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockEmptyState()
await comfyPage.setup()
@@ -76,594 +12,19 @@ test.describe('Assets sidebar - empty states', () => {
await comfyPage.assets.clearMocks()
})
test('Shows empty-state copy for generated tab', async ({ comfyPage }) => {
test('Shows empty-state copy for generated and imported tabs', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await expect(tab.emptyStateTitle('No generated files found')).toBeVisible()
await expect(tab.emptyStateMessage).toBeVisible()
})
test('Shows empty-state copy for imported tab', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.switchToImported()
await tab.importedTab.click()
await expect(tab.emptyStateTitle('No imported files found')).toBeVisible()
await expect(tab.emptyStateMessage).toBeVisible()
})
test('No asset cards are rendered when empty', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await expect(tab.assetCards).toHaveCount(0)
})
})
// ==========================================================================
// 2. Tab navigation
// ==========================================================================
test.describe('Assets sidebar - tab navigation', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles(SAMPLE_IMPORTED_FILES)
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Generated tab is active by default', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await expect(tab.generatedTab).toHaveAttribute('aria-selected', 'true')
await expect(tab.importedTab).toHaveAttribute('aria-selected', 'false')
})
test('Can switch between Generated and Imported tabs', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
// Switch to Imported
await tab.switchToImported()
await expect(tab.importedTab).toHaveAttribute('aria-selected', 'true')
await expect(tab.generatedTab).toHaveAttribute('aria-selected', 'false')
// Switch back to Generated
await tab.switchToGenerated()
await expect(tab.generatedTab).toHaveAttribute('aria-selected', 'true')
})
test('Search is cleared when switching tabs', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
// Type search in Generated tab
await tab.searchInput.fill('landscape')
await expect(tab.searchInput).toHaveValue('landscape')
// Switch to Imported tab
await tab.switchToImported()
await expect(tab.searchInput).toHaveValue('')
})
})
// ==========================================================================
// 3. Asset display - grid view
// ==========================================================================
test.describe('Assets sidebar - grid view display', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles(SAMPLE_IMPORTED_FILES)
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Displays generated assets as cards in grid view', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
const count = await tab.assetCards.count()
expect(count).toBeGreaterThanOrEqual(1)
})
test('Displays imported files when switching to Imported tab', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.switchToImported()
// Wait for imported assets to render
await expect(tab.assetCards.first()).toBeVisible({ timeout: 5000 })
// Imported tab should show the mocked files
const count = await tab.assetCards.count()
expect(count).toBeGreaterThanOrEqual(1)
})
})
// ==========================================================================
// 4. View mode toggle (grid <-> list)
// ==========================================================================
test.describe('Assets sidebar - view mode toggle', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Can switch to list view via settings menu', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Open settings menu and select list view
await tab.openSettingsMenu()
await tab.listViewOption.click()
// List view items should now be visible
await expect(tab.listViewItems.first()).toBeVisible({ timeout: 5000 })
})
test('Can switch back to grid view', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Switch to list view
await tab.openSettingsMenu()
await tab.listViewOption.click()
await expect(tab.listViewItems.first()).toBeVisible({ timeout: 5000 })
// Switch back to grid view (settings popover is still open)
await tab.gridViewOption.click()
await tab.waitForAssets()
// Grid cards (with data-selected attribute) should be visible again
await expect(tab.assetCards.first()).toBeVisible({ timeout: 5000 })
})
})
// ==========================================================================
// 5. Search functionality
// ==========================================================================
test.describe('Assets sidebar - search', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Search input is visible', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await expect(tab.searchInput).toBeVisible()
})
test('Filtering assets by search query reduces displayed count', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
const initialCount = await tab.assetCards.count()
// Search for a specific filename that matches only one asset
await tab.searchInput.fill('landscape')
// Wait for filter to reduce the count
await expect(async () => {
const filteredCount = await tab.assetCards.count()
expect(filteredCount).toBeLessThan(initialCount)
}).toPass({ timeout: 5000 })
})
test('Clearing search restores all assets', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
const initialCount = await tab.assetCards.count()
// Filter then clear
await tab.searchInput.fill('landscape')
await expect(async () => {
expect(await tab.assetCards.count()).toBeLessThan(initialCount)
}).toPass({ timeout: 5000 })
await tab.searchInput.fill('')
await expect(tab.assetCards).toHaveCount(initialCount, { timeout: 5000 })
})
test('Search with no matches shows empty state', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.searchInput.fill('nonexistent_file_xyz')
await expect(tab.assetCards).toHaveCount(0, { timeout: 5000 })
})
})
// ==========================================================================
// 6. Asset selection
// ==========================================================================
test.describe('Assets sidebar - selection', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Clicking an asset card selects it', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Click first asset card
await tab.assetCards.first().click()
// Should have data-selected="true"
await expect(tab.selectedCards).toHaveCount(1)
})
test('Ctrl+click adds to selection', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
const cards = tab.assetCards
const cardCount = await cards.count()
expect(cardCount).toBeGreaterThanOrEqual(2)
// Click first card
await cards.first().click()
await expect(tab.selectedCards).toHaveCount(1)
// Ctrl+click second card
await cards.nth(1).click({ modifiers: ['ControlOrMeta'] })
await expect(tab.selectedCards).toHaveCount(2)
})
test('Selection shows footer with count and actions', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Select an asset
await tab.assetCards.first().click()
// Footer should show selection count
await expect(tab.selectionCountButton).toBeVisible({ timeout: 3000 })
})
test('Deselect all clears selection', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Select an asset
await tab.assetCards.first().click()
await expect(tab.selectedCards).toHaveCount(1)
// Hover over the selection count button to reveal "Deselect all"
await tab.selectionCountButton.hover()
await expect(tab.deselectAllButton).toBeVisible({ timeout: 3000 })
// Click "Deselect all"
await tab.deselectAllButton.click()
await expect(tab.selectedCards).toHaveCount(0)
})
test('Selection is cleared when switching tabs', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Select an asset
await tab.assetCards.first().click()
await expect(tab.selectedCards).toHaveCount(1)
// Switch to Imported tab
await tab.switchToImported()
// Switch back - selection should be cleared
await tab.switchToGenerated()
await tab.waitForAssets()
await expect(tab.selectedCards).toHaveCount(0)
})
})
// ==========================================================================
// 7. Context menu
// ==========================================================================
test.describe('Assets sidebar - context menu', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Right-clicking an asset shows context menu', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Right-click first asset
await tab.assetCards.first().click({ button: 'right' })
// Context menu should appear with standard items
const contextMenu = comfyPage.page.locator('.p-contextmenu')
await expect(contextMenu).toBeVisible({ timeout: 3000 })
})
test('Context menu contains Download action for output asset', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click({ button: 'right' })
await comfyPage.page
.locator('.p-contextmenu')
.waitFor({ state: 'visible', timeout: 3000 })
await expect(tab.contextMenuItem('Download')).toBeVisible()
})
test('Context menu contains Inspect action for image assets', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click({ button: 'right' })
await comfyPage.page
.locator('.p-contextmenu')
.waitFor({ state: 'visible', timeout: 3000 })
await expect(tab.contextMenuItem('Inspect asset')).toBeVisible()
})
test('Context menu contains Delete action for output assets', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click({ button: 'right' })
await comfyPage.page
.locator('.p-contextmenu')
.waitFor({ state: 'visible', timeout: 3000 })
await expect(tab.contextMenuItem('Delete')).toBeVisible()
})
test('Context menu contains Copy job ID for output assets', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click({ button: 'right' })
await comfyPage.page
.locator('.p-contextmenu')
.waitFor({ state: 'visible', timeout: 3000 })
await expect(tab.contextMenuItem('Copy job ID')).toBeVisible()
})
test('Context menu contains workflow actions for output assets', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click({ button: 'right' })
const contextMenu = comfyPage.page.locator('.p-contextmenu')
await expect(contextMenu).toBeVisible({ timeout: 3000 })
await expect(
tab.contextMenuItem('Open as workflow in new tab')
).toBeVisible()
await expect(tab.contextMenuItem('Export workflow')).toBeVisible()
})
test('Bulk context menu shows when multiple assets selected', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
const cards = tab.assetCards
const cardCount = await cards.count()
expect(cardCount).toBeGreaterThanOrEqual(2)
// Dismiss any toasts that appeared after asset loading
await tab.dismissToasts()
// Multi-select: use keyboard.down/up so useKeyModifier('Control') detects
// the modifier — click({ modifiers }) only sets the mouse event flag and
// does not fire a keydown event that VueUse tracks.
await cards.first().click()
await comfyPage.page.keyboard.down('Control')
await cards.nth(1).click()
await comfyPage.page.keyboard.up('Control')
// Verify multi-selection took effect and footer is stable before right-clicking
await expect(tab.selectedCards).toHaveCount(2, { timeout: 3000 })
await expect(tab.selectionFooter).toBeVisible({ timeout: 3000 })
// Use dispatchEvent instead of click({ button: 'right' }) to avoid any
// overlay intercepting the event, and assert directly without toPass.
const contextMenu = comfyPage.page.locator('.p-contextmenu')
await cards.first().dispatchEvent('contextmenu', {
bubbles: true,
cancelable: true,
button: 2
})
await expect(contextMenu).toBeVisible()
// Bulk menu should show bulk download action
await expect(tab.contextMenuItem('Download all')).toBeVisible()
})
})
// ==========================================================================
// 8. Bulk actions (footer)
// ==========================================================================
test.describe('Assets sidebar - bulk actions', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Footer shows download button when assets selected', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click()
// Download button in footer should be visible
await expect(tab.downloadSelectedButton).toBeVisible({ timeout: 3000 })
})
test('Footer shows delete button when output assets selected', async ({
comfyPage
}) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
await tab.assetCards.first().click()
// Delete button in footer should be visible
await expect(tab.deleteSelectedButton).toBeVisible({ timeout: 3000 })
})
test('Selection count displays correct number', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Select two assets
const cards = tab.assetCards
const cardCount = await cards.count()
expect(cardCount).toBeGreaterThanOrEqual(2)
await cards.first().click()
await cards.nth(1).click({ modifiers: ['ControlOrMeta'] })
// Selection count should show the count
await expect(tab.selectionCountButton).toBeVisible({ timeout: 3000 })
const text = await tab.selectionCountButton.textContent()
expect(text).toMatch(/Assets Selected: \d+/)
})
})
// ==========================================================================
// 9. Pagination
// ==========================================================================
test.describe('Assets sidebar - pagination', () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Initially loads a batch of assets with has_more pagination', async ({
comfyPage
}) => {
// Create a large set of jobs to trigger pagination
const manyJobs = createMockJobs(30)
await comfyPage.assets.mockOutputHistory(manyJobs)
await comfyPage.setup()
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.waitForAssets()
// Should load at least the first batch
const count = await tab.assetCards.count()
expect(count).toBeGreaterThanOrEqual(1)
})
})
// ==========================================================================
// 10. Settings menu visibility
// ==========================================================================
test.describe('Assets sidebar - settings menu', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.assets.mockOutputHistory(SAMPLE_JOBS)
await comfyPage.assets.mockInputFiles([])
await comfyPage.setup()
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.assets.clearMocks()
})
test('Settings menu shows view mode options', async ({ comfyPage }) => {
const tab = comfyPage.menu.assetsTab
await tab.open()
await tab.openSettingsMenu()
await expect(tab.listViewOption).toBeVisible()
await expect(tab.gridViewOption).toBeVisible()
})
})

View File

@@ -1,528 +0,0 @@
import type { Locator } from '@playwright/test'
import {
comfyExpect as expect,
comfyPageFixture as test
} from '../../../../fixtures/ComfyPage'
import type { ComfyPage } from '../../../../fixtures/ComfyPage'
const BYPASS_CLASS = /before:bg-bypass\/60/
const PIN_INDICATOR = '[data-testid="node-pin-indicator"]'
async function clickExactMenuItem(comfyPage: ComfyPage, name: string) {
await comfyPage.page.getByRole('menuitem', { name, exact: true }).click()
await comfyPage.nextFrame()
}
async function openContextMenu(comfyPage: ComfyPage, nodeTitle: string) {
const header = comfyPage.vueNodes
.getNodeByTitle(nodeTitle)
.locator('.lg-node-header')
await header.click()
await header.click({ button: 'right' })
const menu = comfyPage.page.locator('.p-contextmenu')
await menu.waitFor({ state: 'visible' })
return menu
}
async function openMultiNodeContextMenu(
comfyPage: ComfyPage,
titles: string[]
) {
// deselectAll via evaluate — clearSelection() clicks at a fixed position
// which can hit nodes or the toolbar overlay
await comfyPage.page.evaluate(() => window.app!.canvas.deselectAll())
await comfyPage.nextFrame()
for (const title of titles) {
const header = comfyPage.vueNodes
.getNodeByTitle(title)
.locator('.lg-node-header')
await header.click({ modifiers: ['ControlOrMeta'] })
}
await comfyPage.nextFrame()
const firstHeader = comfyPage.vueNodes
.getNodeByTitle(titles[0])
.locator('.lg-node-header')
const box = await firstHeader.boundingBox()
if (!box) throw new Error(`Header for "${titles[0]}" not found`)
await comfyPage.page.mouse.click(
box.x + box.width / 2,
box.y + box.height / 2,
{ button: 'right' }
)
const menu = comfyPage.page.locator('.p-contextmenu')
await menu.waitFor({ state: 'visible' })
return menu
}
function getNodeWrapper(comfyPage: ComfyPage, nodeTitle: string): Locator {
return comfyPage.page
.locator('[data-node-id]')
.filter({ hasText: nodeTitle })
.getByTestId('node-inner-wrapper')
}
async function getNodeRef(comfyPage: ComfyPage, nodeTitle: string) {
const refs = await comfyPage.nodeOps.getNodeRefsByTitle(nodeTitle)
return refs[0]
}
test.describe('Vue Node Context Menu', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.vueNodes.waitForNodes()
})
test.describe('Single Node Actions', () => {
test('should rename node via context menu', async ({ comfyPage }) => {
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Rename')
const titleInput = comfyPage.page.locator(
'.node-title-editor input[type="text"]'
)
await titleInput.waitFor({ state: 'visible' })
await titleInput.fill('My Renamed Sampler')
await titleInput.press('Enter')
await comfyPage.nextFrame()
const renamedNode =
comfyPage.vueNodes.getNodeByTitle('My Renamed Sampler')
await expect(renamedNode).toBeVisible()
})
test('should copy and paste node via context menu', async ({
comfyPage
}) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
await openContextMenu(comfyPage, 'Load Checkpoint')
await clickExactMenuItem(comfyPage, 'Copy')
// Internal clipboard paste (menu Copy uses canvas clipboard, not OS)
await comfyPage.page.evaluate(() => {
window.app!.canvas.pasteFromClipboard({ connectInputs: false })
})
await comfyPage.nextFrame()
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
initialCount + 1
)
})
test('should duplicate node via context menu', async ({ comfyPage }) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
await openContextMenu(comfyPage, 'Load Checkpoint')
await clickExactMenuItem(comfyPage, 'Duplicate')
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
initialCount + 1
)
})
test('should pin and unpin node via context menu', async ({
comfyPage
}) => {
const nodeTitle = 'Load Checkpoint'
const nodeRef = await getNodeRef(comfyPage, nodeTitle)
// Pin via context menu
await openContextMenu(comfyPage, nodeTitle)
await clickExactMenuItem(comfyPage, 'Pin')
const pinIndicator = comfyPage.vueNodes
.getNodeByTitle(nodeTitle)
.locator(PIN_INDICATOR)
await expect(pinIndicator).toBeVisible()
expect(await nodeRef.isPinned()).toBe(true)
// Verify drag blocked
const header = comfyPage.vueNodes
.getNodeByTitle(nodeTitle)
.locator('.lg-node-header')
const posBeforeDrag = await header.boundingBox()
if (!posBeforeDrag) throw new Error('Header not found')
await comfyPage.canvasOps.dragAndDrop(
{ x: posBeforeDrag.x + 10, y: posBeforeDrag.y + 10 },
{ x: posBeforeDrag.x + 256, y: posBeforeDrag.y + 256 }
)
const posAfterDrag = await header.boundingBox()
expect(posAfterDrag).toEqual(posBeforeDrag)
// Unpin via context menu
await openContextMenu(comfyPage, nodeTitle)
await clickExactMenuItem(comfyPage, 'Unpin')
await expect(pinIndicator).not.toBeVisible()
expect(await nodeRef.isPinned()).toBe(false)
})
test('should bypass node and remove bypass via context menu', async ({
comfyPage
}) => {
const nodeTitle = 'Load Checkpoint'
const nodeRef = await getNodeRef(comfyPage, nodeTitle)
await openContextMenu(comfyPage, nodeTitle)
await clickExactMenuItem(comfyPage, 'Bypass')
expect(await nodeRef.isBypassed()).toBe(true)
await expect(getNodeWrapper(comfyPage, nodeTitle)).toHaveClass(
BYPASS_CLASS
)
await openContextMenu(comfyPage, nodeTitle)
await clickExactMenuItem(comfyPage, 'Remove Bypass')
expect(await nodeRef.isBypassed()).toBe(false)
await expect(getNodeWrapper(comfyPage, nodeTitle)).not.toHaveClass(
BYPASS_CLASS
)
})
test('should minimize and expand node via context menu', async ({
comfyPage
}) => {
const fixture = await comfyPage.vueNodes.getFixtureByTitle('KSampler')
await expect(fixture.body).toBeVisible()
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Minimize Node')
await expect(fixture.body).not.toBeVisible()
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Expand Node')
await expect(fixture.body).toBeVisible()
})
test('should convert node to subgraph via context menu', async ({
comfyPage
}) => {
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Convert to Subgraph')
const subgraphNode = comfyPage.vueNodes.getNodeByTitle('New Subgraph')
await expect(subgraphNode).toBeVisible()
await expect(
comfyPage.vueNodes.getNodeByTitle('KSampler')
).not.toBeVisible()
})
})
test.describe('Image Node Actions', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.page
.context()
.grantPermissions(['clipboard-read', 'clipboard-write'])
await comfyPage.workflow.loadWorkflow('widgets/load_image_widget')
await comfyPage.vueNodes.waitForNodes(1)
})
test('should copy image to clipboard via context menu', async ({
comfyPage
}) => {
await openContextMenu(comfyPage, 'Load Image')
await clickExactMenuItem(comfyPage, 'Copy Image')
// Verify the clipboard contains an image
const hasImage = await comfyPage.page.evaluate(async () => {
const items = await navigator.clipboard.read()
return items.some((item) =>
item.types.some((t) => t.startsWith('image/'))
)
})
expect(hasImage).toBe(true)
})
test('should paste image to LoadImage node via context menu', async ({
comfyPage
}) => {
// Capture the original image src from the node's preview
const imagePreview = comfyPage.page.locator('.image-preview img')
const originalSrc = await imagePreview.getAttribute('src')
// Write a test image into the browser clipboard
await comfyPage.page.evaluate(async () => {
const resp = await fetch('/api/view?filename=example.png&type=input')
const blob = await resp.blob()
await navigator.clipboard.write([
new ClipboardItem({ [blob.type]: blob })
])
})
// Right-click and select Paste Image
await openContextMenu(comfyPage, 'Load Image')
await clickExactMenuItem(comfyPage, 'Paste Image')
// Verify the image preview src changed
await expect(imagePreview).not.toHaveAttribute('src', originalSrc!)
})
test('should open image in new tab via context menu', async ({
comfyPage
}) => {
await openContextMenu(comfyPage, 'Load Image')
const popupPromise = comfyPage.page.waitForEvent('popup')
await clickExactMenuItem(comfyPage, 'Open Image')
const popup = await popupPromise
expect(popup.url()).toContain('/api/view')
expect(popup.url()).toContain('filename=')
await popup.close()
})
test('should download image via Save Image context menu', async ({
comfyPage
}) => {
await openContextMenu(comfyPage, 'Load Image')
const downloadPromise = comfyPage.page.waitForEvent('download')
await clickExactMenuItem(comfyPage, 'Save Image')
const download = await downloadPromise
expect(download.suggestedFilename()).toBeTruthy()
})
})
test.describe('Subgraph Actions', () => {
test('should convert to subgraph and unpack back', async ({
comfyPage
}) => {
// Convert KSampler to subgraph
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Convert to Subgraph')
const subgraphNode = comfyPage.vueNodes.getNodeByTitle('New Subgraph')
await expect(subgraphNode).toBeVisible()
await expect(
comfyPage.vueNodes.getNodeByTitle('KSampler')
).not.toBeVisible()
// Unpack the subgraph
await openContextMenu(comfyPage, 'New Subgraph')
await clickExactMenuItem(comfyPage, 'Unpack Subgraph')
await expect(comfyPage.vueNodes.getNodeByTitle('KSampler')).toBeVisible()
await expect(
comfyPage.vueNodes.getNodeByTitle('New Subgraph')
).not.toBeVisible()
})
test('should open properties panel via Edit Subgraph Widgets', async ({
comfyPage
}) => {
// Convert to subgraph first
await openContextMenu(comfyPage, 'Empty Latent Image')
await clickExactMenuItem(comfyPage, 'Convert to Subgraph')
await comfyPage.nextFrame()
// Right-click subgraph and edit widgets
await openContextMenu(comfyPage, 'New Subgraph')
await clickExactMenuItem(comfyPage, 'Edit Subgraph Widgets')
await expect(comfyPage.page.getByTestId('properties-panel')).toBeVisible()
})
test('should add subgraph to library and find in node library', async ({
comfyPage
}) => {
// Convert to subgraph first
await openContextMenu(comfyPage, 'KSampler')
await clickExactMenuItem(comfyPage, 'Convert to Subgraph')
await comfyPage.nextFrame()
// Add to library
await openContextMenu(comfyPage, 'New Subgraph')
await clickExactMenuItem(comfyPage, 'Add Subgraph to Library')
// Fill the blueprint name
await comfyPage.nodeOps.promptDialogInput.waitFor({ state: 'visible' })
await comfyPage.nodeOps.fillPromptDialog('TestBlueprint')
// Open node library sidebar and search for the blueprint
await comfyPage.page.getByRole('button', { name: 'Node Library' }).click()
await comfyPage.nextFrame()
const searchBox = comfyPage.page.getByRole('combobox', {
name: 'Search'
})
await searchBox.waitFor({ state: 'visible' })
await searchBox.fill('TestBlueprint')
await comfyPage.nextFrame()
await expect(comfyPage.page.getByText('TestBlueprint')).toBeVisible()
})
})
test.describe('Multi-Node Actions', () => {
const nodeTitles = ['Load Checkpoint', 'KSampler']
test('should batch rename selected nodes via context menu', async ({
comfyPage
}) => {
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Rename')
await comfyPage.nodeOps.promptDialogInput.waitFor({ state: 'visible' })
await comfyPage.nodeOps.fillPromptDialog('MyNode')
await expect(comfyPage.vueNodes.getNodeByTitle('MyNode 1')).toBeVisible()
await expect(comfyPage.vueNodes.getNodeByTitle('MyNode 2')).toBeVisible()
})
test('should copy and paste selected nodes via context menu', async ({
comfyPage
}) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Copy')
await comfyPage.page.evaluate(() => {
window.app!.canvas.pasteFromClipboard({ connectInputs: false })
})
await comfyPage.nextFrame()
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
initialCount + nodeTitles.length
)
})
test('should duplicate selected nodes via context menu', async ({
comfyPage
}) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Duplicate')
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
initialCount + nodeTitles.length
)
})
test('should pin and unpin selected nodes via context menu', async ({
comfyPage
}) => {
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Pin')
for (const title of nodeTitles) {
const pinIndicator = comfyPage.vueNodes
.getNodeByTitle(title)
.locator(PIN_INDICATOR)
await expect(pinIndicator).toBeVisible()
}
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Unpin')
for (const title of nodeTitles) {
const pinIndicator = comfyPage.vueNodes
.getNodeByTitle(title)
.locator(PIN_INDICATOR)
await expect(pinIndicator).not.toBeVisible()
}
})
test('should bypass and remove bypass on selected nodes via context menu', async ({
comfyPage
}) => {
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Bypass')
for (const title of nodeTitles) {
const nodeRef = await getNodeRef(comfyPage, title)
expect(await nodeRef.isBypassed()).toBe(true)
await expect(getNodeWrapper(comfyPage, title)).toHaveClass(BYPASS_CLASS)
}
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Remove Bypass')
for (const title of nodeTitles) {
const nodeRef = await getNodeRef(comfyPage, title)
expect(await nodeRef.isBypassed()).toBe(false)
await expect(getNodeWrapper(comfyPage, title)).not.toHaveClass(
BYPASS_CLASS
)
}
})
test('should minimize and expand selected nodes via context menu', async ({
comfyPage
}) => {
const fixture1 =
await comfyPage.vueNodes.getFixtureByTitle('Load Checkpoint')
const fixture2 = await comfyPage.vueNodes.getFixtureByTitle('KSampler')
await expect(fixture1.body).toBeVisible()
await expect(fixture2.body).toBeVisible()
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Minimize Node')
await expect(fixture1.body).not.toBeVisible()
await expect(fixture2.body).not.toBeVisible()
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Expand Node')
await expect(fixture1.body).toBeVisible()
await expect(fixture2.body).toBeVisible()
})
test('should frame selected nodes via context menu', async ({
comfyPage
}) => {
const initialGroupCount = await comfyPage.page.evaluate(
() => window.app!.graph.groups.length
)
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Frame Nodes')
const newGroupCount = await comfyPage.page.evaluate(
() => window.app!.graph.groups.length
)
expect(newGroupCount).toBe(initialGroupCount + 1)
})
test('should convert to group node via context menu', async ({
comfyPage
}) => {
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Convert to Group Node')
await comfyPage.nodeOps.promptDialogInput.waitFor({ state: 'visible' })
await comfyPage.nodeOps.fillPromptDialog('TestGroupNode')
const groupNodes = await comfyPage.nodeOps.getNodeRefsByType(
'workflow>TestGroupNode'
)
expect(groupNodes.length).toBe(1)
})
test('should convert selected nodes to subgraph via context menu', async ({
comfyPage
}) => {
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
await openMultiNodeContextMenu(comfyPage, nodeTitles)
await clickExactMenuItem(comfyPage, 'Convert to Subgraph')
const subgraphNode = comfyPage.vueNodes.getNodeByTitle('New Subgraph')
await expect(subgraphNode).toBeVisible()
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
initialCount - nodeTitles.length + 1
)
})
})
})

View File

@@ -1,532 +0,0 @@
import { readFileSync } from 'fs'
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
test.describe('Workflow Persistence', () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
'Comfy.Workflow.WorkflowTabsPosition',
'Sidebar'
)
})
test.afterEach(async ({ comfyPage }) => {
await comfyPage.workflow.setupWorkflowsDirectory({})
})
test('Rapid tab switching does not desync workflow and graph state', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description: 'PR #9533 — desynced workflow/graph state during loading'
})
const tab = comfyPage.menu.workflowsTab
await tab.open()
await comfyPage.menu.topbar.saveWorkflow('rapid-A')
const nodeCountA = await comfyPage.nodeOps.getNodeCount()
await comfyPage.workflow.loadWorkflow('nodes/single_ksampler')
await comfyPage.menu.topbar.saveWorkflow('rapid-B')
const nodeCountB = await comfyPage.nodeOps.getNodeCount()
expect(nodeCountA).not.toBe(nodeCountB)
for (let i = 0; i < 3; i++) {
await tab.switchToWorkflow('rapid-A')
await tab.switchToWorkflow('rapid-B')
}
await comfyPage.workflow.waitForWorkflowIdle()
await comfyPage.nextFrame()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 5000 })
.toBe(nodeCountB)
await tab.switchToWorkflow('rapid-A')
await comfyPage.workflow.waitForWorkflowIdle()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 5000 })
.toBe(nodeCountA)
})
test('Node outputs are preserved when switching workflow tabs', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'PR #9380 — ChangeTracker.store() did not save nodeOutputs, losing preview images on tab switch'
})
const tab = comfyPage.menu.workflowsTab
await tab.open()
await comfyPage.menu.topbar.saveWorkflow('outputs-test')
const firstNode = await comfyPage.nodeOps.getFirstNodeRef()
expect(firstNode).toBeTruthy()
const nodeId = firstNode!.id
// Simulate node outputs as if execution completed
await comfyPage.page.evaluate((id) => {
const outputStore = window.app!.nodeOutputs
if (outputStore) {
outputStore[id] = {
images: [{ filename: 'test.png', subfolder: '', type: 'output' }]
}
}
}, String(nodeId))
// Trigger changeTracker to capture current state including outputs
await comfyPage.page.evaluate(() => {
const em = window.app!.extensionManager as unknown as Record<
string,
{ activeWorkflow?: { changeTracker?: { checkState(): void } } }
>
em.workflow?.activeWorkflow?.changeTracker?.checkState()
})
await comfyPage.nextFrame()
const outputsBefore = await comfyPage.page.evaluate((id) => {
return window.app!.nodeOutputs?.[id]
}, String(nodeId))
expect(outputsBefore).toBeTruthy()
await comfyPage.command.executeCommand('Comfy.NewBlankWorkflow')
await comfyPage.nextFrame()
await tab.switchToWorkflow('outputs-test')
await comfyPage.nextFrame()
const outputsAfter = await comfyPage.page.evaluate((id) => {
return window.app!.nodeOutputs?.[id]
}, String(nodeId))
expect(outputsAfter).toBeTruthy()
expect(outputsAfter?.images).toBeDefined()
})
test('Loading a new workflow cleanly replaces the previous graph', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'Commit 44bb6f13 — canvas graph not reset before workflow load'
})
const defaultNodeCount = await comfyPage.nodeOps.getNodeCount()
expect(defaultNodeCount).toBeGreaterThan(1)
await comfyPage.workflow.loadWorkflow('nodes/single_ksampler')
await comfyPage.nextFrame()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBe(1)
const nodes = await comfyPage.nodeOps.getNodes()
expect(nodes[0].type).toBe('KSampler')
})
test('Widget values on nodes are preserved across workflow tab switches', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description: 'PR #7648 — component widget state lost on graph change'
})
const tab = comfyPage.menu.workflowsTab
await tab.open()
await comfyPage.menu.topbar.saveWorkflow('widget-state-test')
// Read widget values via page.evaluate — these are internal LiteGraph
// state not exposed through DOM
const widgetValuesBefore = await comfyPage.page.evaluate(() => {
const nodes = window.app!.graph.nodes
const results: Record<string, unknown[]> = {}
for (const node of nodes) {
if (node.widgets && node.widgets.length > 0) {
results[node.id] = node.widgets.map((w) => ({
name: w.name,
value: w.value
}))
}
}
return results
})
expect(Object.keys(widgetValuesBefore).length).toBeGreaterThan(0)
await comfyPage.command.executeCommand('Comfy.NewBlankWorkflow')
await comfyPage.nextFrame()
await tab.switchToWorkflow('widget-state-test')
await comfyPage.nextFrame()
const widgetValuesAfter = await comfyPage.page.evaluate(() => {
const nodes = window.app!.graph.nodes
const results: Record<string, unknown[]> = {}
for (const node of nodes) {
if (node.widgets && node.widgets.length > 0) {
results[node.id] = node.widgets.map((w) => ({
name: w.name,
value: w.value
}))
}
}
return results
})
expect(widgetValuesAfter).toEqual(widgetValuesBefore)
})
test('API format workflow with missing node types partially loads', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description: 'PR #9694 — loadApiJson early-returned on missing node types'
})
const fixturePath = comfyPage.assetPath(
'nodes/api_workflow_with_missing_nodes.json'
)
const apiWorkflow = JSON.parse(readFileSync(fixturePath, 'utf-8'))
await comfyPage.page.evaluate(async (workflow) => {
await window.app!.loadApiJson(workflow, 'test-api-workflow.json')
}, apiWorkflow)
await comfyPage.nextFrame()
// Known nodes (KSampler, EmptyLatentImage) should load; unknown node skipped
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBeGreaterThanOrEqual(2)
const nodeTypes = await comfyPage.page.evaluate(() => {
return window.app!.graph.nodes.map((n: { type: string }) => n.type)
})
expect(nodeTypes).toContain('KSampler')
expect(nodeTypes).toContain('EmptyLatentImage')
expect(nodeTypes).not.toContain('NonExistentCustomNode_XYZ_12345')
})
test('Canvas has auxclick handler to prevent middle-click paste', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'PR #8259 — middle-click paste duplicates entire workflow on Linux'
})
const initialNodeCount = await comfyPage.nodeOps.getNodeCount()
await comfyPage.canvas.click({
button: 'middle',
position: { x: 100, y: 100 }
})
await comfyPage.nextFrame()
const nodeCountAfter = await comfyPage.nodeOps.getNodeCount()
expect(nodeCountAfter).toBe(initialNodeCount)
})
test('Exported workflow does not contain transient blob: URLs', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'PR #8715 — transient image URLs leaked into workflow serialization'
})
const exportedWorkflow = await comfyPage.workflow.getExportedWorkflow()
for (const node of exportedWorkflow.nodes) {
if (node.widgets_values && Array.isArray(node.widgets_values)) {
for (const value of node.widgets_values) {
if (typeof value === 'string') {
expect(value).not.toMatch(/^blob:/)
expect(value).not.toMatch(/^https?:\/\/.*\/api\/view/)
}
}
}
}
})
test('Changing locale does not break workflow operations', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description: 'PR #8963 — template workflows not reloaded on locale change'
})
const tab = comfyPage.menu.workflowsTab
await tab.open()
await comfyPage.menu.topbar.saveWorkflow('locale-test')
const initialNodeCount = await comfyPage.nodeOps.getNodeCount()
await comfyPage.settings.setSetting('Comfy.Locale', 'zh')
await comfyPage.nextFrame()
await comfyPage.settings.setSetting('Comfy.Locale', 'en')
await comfyPage.nextFrame()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount())
.toBe(initialNodeCount)
await expect.poll(() => tab.getActiveWorkflowName()).toBe('locale-test')
})
test('Node links survive save/load/switch cycles', async ({ comfyPage }) => {
test.info().annotations.push({
type: 'regression',
description: 'PR #9533 — node links must survive serialization roundtrips'
})
const tab = comfyPage.menu.workflowsTab
await tab.open()
// Link count requires internal graph state — not exposed via DOM
const linkCountBefore = await comfyPage.page.evaluate(() => {
return window.app!.graph.links
? Object.keys(window.app!.graph.links).length
: 0
})
expect(linkCountBefore).toBeGreaterThan(0)
await comfyPage.menu.topbar.saveWorkflow('links-test')
await comfyPage.command.executeCommand('Comfy.NewBlankWorkflow')
await comfyPage.nextFrame()
await tab.switchToWorkflow('links-test')
await comfyPage.workflow.waitForWorkflowIdle()
const linkCountAfter = await comfyPage.page.evaluate(() => {
return window.app!.graph.links
? Object.keys(window.app!.graph.links).length
: 0
})
expect(linkCountAfter).toBe(linkCountBefore)
})
test('Closing an inactive tab with save preserves its own content', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'PR #10745 — saveWorkflow called checkState on inactive tab, serializing the active graph instead'
})
await comfyPage.settings.setSetting(
'Comfy.Workflow.WorkflowTabsPosition',
'Topbar'
)
const suffix = Date.now().toString(36)
const nameA = `test-A-${suffix}`
const nameB = `test-B-${suffix}`
// Save the default workflow as A
await comfyPage.menu.topbar.saveWorkflow(nameA)
const nodeCountA = await comfyPage.nodeOps.getNodeCount()
// Create B: duplicate and save
await comfyPage.command.executeCommand('Comfy.DuplicateWorkflow')
await comfyPage.nextFrame()
await comfyPage.menu.topbar.saveWorkflow(nameB)
// Add a Note node in B to mark it as modified
await comfyPage.page.evaluate(() => {
window.app!.graph.add(window.LiteGraph!.createNode('Note', undefined, {}))
})
await comfyPage.nextFrame()
const nodeCountB = await comfyPage.nodeOps.getNodeCount()
expect(nodeCountB).toBe(nodeCountA + 1)
// Trigger checkState so isModified is set
await comfyPage.page.evaluate(() => {
const em = window.app!.extensionManager as unknown as Record<
string,
{ activeWorkflow?: { changeTracker?: { checkState(): void } } }
>
em.workflow?.activeWorkflow?.changeTracker?.checkState()
})
// Switch to A via topbar tab (making B inactive)
await comfyPage.menu.topbar.getWorkflowTab(nameA).click()
await comfyPage.workflow.waitForWorkflowIdle()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBe(nodeCountA)
// Close inactive B tab via middle-click — triggers "Save before closing?"
await comfyPage.menu.topbar.getWorkflowTab(nameB).click({
button: 'middle'
})
// Click "Save" in the dirty close dialog
const saveButton = comfyPage.page.getByRole('button', { name: 'Save' })
await saveButton.waitFor({ state: 'visible' })
await saveButton.click()
await comfyPage.workflow.waitForWorkflowIdle()
await comfyPage.nextFrame()
// Verify we're still on A with A's content
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBe(nodeCountA)
// Re-open B from sidebar saved list
const workflowsTab = comfyPage.menu.workflowsTab
await workflowsTab.open()
await workflowsTab.getPersistedItem(nameB).dblclick()
await comfyPage.workflow.waitForWorkflowIdle()
// B should have the extra Note node we added, not A's node count
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 5000 })
.toBe(nodeCountB)
})
test('Closing an inactive unsaved tab with save preserves its own content', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'PR #10745 — saveWorkflowAs called checkState on inactive temp tab, serializing the active graph'
})
await comfyPage.settings.setSetting(
'Comfy.Workflow.WorkflowTabsPosition',
'Topbar'
)
const suffix = Date.now().toString(36)
const nameA = `test-A-${suffix}`
const nameB = `test-B-${suffix}`
// Save the default workflow as A
await comfyPage.menu.topbar.saveWorkflow(nameA)
const nodeCountA = await comfyPage.nodeOps.getNodeCount()
// Create B as an unsaved workflow with a Note node
await comfyPage.command.executeCommand('Comfy.NewBlankWorkflow')
await comfyPage.nextFrame()
await comfyPage.page.evaluate(() => {
window.app!.graph.add(window.LiteGraph!.createNode('Note', undefined, {}))
})
await comfyPage.nextFrame()
// Trigger checkState so isModified is set
await comfyPage.page.evaluate(() => {
const em = window.app!.extensionManager as unknown as Record<
string,
{ activeWorkflow?: { changeTracker?: { checkState(): void } } }
>
em.workflow?.activeWorkflow?.changeTracker?.checkState()
})
const nodeCountB = await comfyPage.nodeOps.getNodeCount()
expect(nodeCountB).toBe(1)
expect(nodeCountA).not.toBe(nodeCountB)
// Switch to A via topbar tab (making unsaved B inactive)
await comfyPage.menu.topbar.getWorkflowTab(nameA).click()
await comfyPage.workflow.waitForWorkflowIdle()
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBe(nodeCountA)
// Close inactive unsaved B tab — triggers "Save before closing?"
await comfyPage.menu.topbar
.getWorkflowTab('Unsaved Workflow')
.click({ button: 'middle' })
// Click "Save" in the dirty close dialog (scoped to dialog)
const dialog = comfyPage.page.getByRole('dialog')
const saveButton = dialog.getByRole('button', { name: 'Save' })
await saveButton.waitFor({ state: 'visible' })
await saveButton.click()
// Fill in the filename dialog
const saveDialog = comfyPage.menu.topbar.getSaveDialog()
await saveDialog.waitFor({ state: 'visible' })
await saveDialog.fill(nameB)
await comfyPage.page.keyboard.press('Enter')
await comfyPage.workflow.waitForWorkflowIdle()
await comfyPage.nextFrame()
// Verify we're still on A with A's content
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 3000 })
.toBe(nodeCountA)
// Re-open B from sidebar saved list
const workflowsTab = comfyPage.menu.workflowsTab
await workflowsTab.open()
await workflowsTab.getPersistedItem(nameB).dblclick()
await comfyPage.workflow.waitForWorkflowIdle()
// B should have 1 node (the Note), not A's node count
await expect
.poll(() => comfyPage.nodeOps.getNodeCount(), { timeout: 5000 })
.toBe(nodeCountB)
})
test('Splitter panel sizes persist correctly in localStorage', async ({
comfyPage
}) => {
test.info().annotations.push({
type: 'regression',
description:
'Commits 91f197d9d + a1b7e57bc — splitter panel size drift on reload'
})
await comfyPage.page.evaluate(() => {
localStorage.setItem(
'Comfy.Splitter.MainSplitter',
JSON.stringify([30, 70])
)
})
await comfyPage.setup({ clearStorage: false })
await comfyPage.nextFrame()
const storedSizes = await comfyPage.page.evaluate(() => {
const raw = localStorage.getItem('Comfy.Splitter.MainSplitter')
return raw ? JSON.parse(raw) : null
})
expect(storedSizes).toBeTruthy()
expect(Array.isArray(storedSizes)).toBe(true)
for (const size of storedSizes as number[]) {
expect(typeof size).toBe('number')
expect(size).toBeGreaterThanOrEqual(0)
expect(size).not.toBeNaN()
}
const total = (storedSizes as number[]).reduce(
(a: number, b: number) => a + b,
0
)
expect(total).toBeGreaterThan(90)
expect(total).toBeLessThanOrEqual(101)
})
})

View File

@@ -363,7 +363,7 @@ Test your feature flags with different combinations:
### Example Test
```typescript
// Example from a colocated unit test
// In tests-ui/tests/api.featureFlags.test.ts
it('should handle preview metadata based on feature flag', () => {
// Mock server supports feature
api.serverFeatureFlags = { supports_preview_metadata: true }

View File

@@ -1,78 +0,0 @@
# QA Pipeline Troubleshooting
## Common Failures
### `set -euo pipefail` + grep with no match
**Symptom**: Deploy script crashes silently, badge shows FAILED.
**Cause**: `grep -oP` returns exit code 1 when no match. Under `pipefail`, this kills the entire script.
**Fix**: Always append `|| true` to grep pipelines in bash scripts.
### `__name is not defined` in page.evaluate
**Symptom**: Recording crashes with `ReferenceError: __name is not defined`.
**Cause**: tsx compiles arrow functions inside `page.evaluate()` with `__name` helpers. The browser context doesn't have these.
**Fix**: Use `page.addScriptTag({ content: '...' })` with plain JS strings instead of `page.evaluate(() => { ... })` with arrow functions.
### `Set<string>()` in page.evaluate
**Symptom**: Same `__name` error.
**Cause**: TypeScript generics like `new Set<string>()` get compiled incorrectly for browser context.
**Fix**: Use `new Set()` without type parameter.
### `zod/v4` import error
**Symptom**: `ERR_PACKAGE_PATH_NOT_EXPORTED: Package subpath './v4' is not defined`.
**Cause**: claude-agent-sdk depends on `zod/v4` internally, but the project's zod doesn't export it.
**Fix**: Import from `zod` (not `zod/v4`) in project code.
### `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`
**Symptom**: pnpm install fails with frozen lockfile mismatch.
**Cause**: Adding a new dependency changes the workspace catalog but lockfile wasn't regenerated.
**Fix**: Run `pnpm install` to regenerate lockfile, commit `pnpm-workspace.yaml` + `pnpm-lock.yaml`.
### `loadDefaultWorkflow` — "Load Default" not found
**Symptom**: Menu item "Load Default" not found, canvas stays empty.
**Cause**: The menu item name varies by version/locale. Menu navigation is fragile.
**Fix**: Use `app.resetToDefaultWorkflow()` JS API via `page.evaluate` instead of menu navigation.
### Model ID not found (Claude Agent SDK)
**Symptom**: `There's an issue with the selected model (claude-sonnet-4-6-20250514)`.
**Cause**: Dated model IDs like `claude-sonnet-4-6-20250514` don't exist.
**Fix**: Use `claude-sonnet-4-6` (no date suffix).
### Model not found (Gemini)
**Symptom**: 404 from Gemini API.
**Cause**: Preview model names like `gemini-2.5-flash-preview-05-20` expire.
**Fix**: Use `gemini-3-flash-preview` (latest stable).
## Badge Mismatches
### False REPRODUCED
**Symptom**: Badge says REPRODUCED but AI review says "could not reproduce".
**Root cause**: Grep pattern `reproduc|confirm` matches neutral words like "reproduction steps" or "could not be confirmed".
**Fix**: Use structured JSON verdict from AI (`## Verdict` section with `{"verdict": "..."}`) instead of regex matching the prose.
### INCONCLUSIVE feedback loop
**Symptom**: Once an issue gets INCONCLUSIVE, all future runs stay INCONCLUSIVE.
**Cause**: QA bot's own previous comments contain "INCONCLUSIVE", which gets fed back into pr-context.txt.
**Fix**: Filter out `github-actions[bot]` comments when building pr-context.
### pressKey with hold prevents event propagation
**Symptom**: BEFORE video doesn't show the bug (e.g., Escape doesn't close dialog).
**Cause**: `keyboard.down()` + 400ms sleep + `keyboard.up()` changes event timing. Some UI frameworks handle held keys differently than instant presses.
**Fix**: Use instant `keyboard.press()` for testing. Show key name via subtitle overlay instead.
## Cursor Not Visible
**Symptom**: No mouse cursor in recorded videos.
**Cause**: Headless Chrome doesn't render system cursor. The CSS cursor overlay relies on DOM `mousemove` events which Playwright CDP doesn't reliably trigger.
**Fix**: Monkey-patch `page.mouse.move/click/dblclick/down/up` to call `__moveCursor(x,y)` on the injected cursor div. This makes ALL mouse operations update the overlay.
## Credit Balance Too Low
**Symptom**: Research phase produces INCONCLUSIVE with 0 tool calls. Log shows "Credit balance is too low".
**Cause**: The `ANTHROPIC_API_KEY` secret in the repo has exhausted its credits.
**Fix**: Top up the Anthropic API account linked to the key, or rotate to a new key in repo Settings → Secrets.
## Agent Doesn't Perform Steps
**Symptom**: Agent opens menus and settings but never interacts with the canvas.
**Causes**:
1. `loadDefaultWorkflow` failed (no nodes on canvas)
2. Agent ran out of turn budget (30 turns / 120s)
3. Gemini Flash (old agent) ignores prompt hints
**Fix**: Use hybrid agent (Claude Sonnet 4.6 + Gemini vision). Claude's superior reasoning follows instructions precisely.

View File

@@ -1,59 +0,0 @@
# QA Pipeline Backlog
## Comparison Modes
### Type A: Same code, different settings (IMPLEMENTED)
Agent demonstrates both working (control) and broken (test) states in one session by toggling settings. E.g., Nodes 2.0 OFF → drag works, Nodes 2.0 ON → drag broken.
### Type B: Different commits
For regressions reported as "worked in vX.Y, broken in vX.Z":
- `qa-analyze-pr.ts` detects regression markers ("since v1.38", "after PR #1234")
- Pipeline checks out the old commit, records control video
- Records test video on current main
- Side-by-side comparison on report page (reuses PR before/after infra)
### Type C: Different browsers
For browser-specific bugs ("works on Chrome, broken on Firefox"):
- Run recording with different Playwright browser contexts
- Compare behavior across browsers in one report
## Agent Improvements
### TTS Narration
- OpenAI TTS (`tts-1`, nova voice) generates audio from agent reasoning
- Merged into video via ffmpeg at correct timestamps
- Currently in qa-record.ts but needs wiring into hybrid agent path
### Image/Screenshot Reading
- `qa-analyze-pr.ts` already downloads and sends images from issue bodies to Gemini
- Could also send them to the Claude agent as context ("the reporter showed this screenshot")
### Placeholder Page
- Deploy a status page immediately when CI starts
- Auto-refreshes every 30s until final report replaces it
- Shows spinner, CI link, badge
### Pre-seed Assets
- Upload test images via ComfyUI API before recording
- Enables reproduction of bugs requiring assets (#10424 zoom button)
### Environment-Dependent Issues
- #7942: needs custom TestNode — could install a test custom node pack in CI
- #9101: needs completed generation — could run with a tiny model checkpoint
## Cost Optimization
### Lazy A11y Tree
- `inspect(selector)` searches tree for specific element (~20 tokens)
- `getUIChanges()` diffs against previous snapshot (~100 tokens)
- vs dumping full tree every turn (~2000 tokens)
### Gemini Video vs Images
- 30s video clip: ~7,700 tokens (258 tok/s)
- 15 screenshots: ~19,500 tokens (1,300 tok/frame)
- Video is 2.5x cheaper and shows temporal changes
### Model Selection
- Claude Sonnet 4.6: $3/$15 per 1M in/out — best reasoning
- Gemini 2.5 Flash: $0.10/$0.40 per 1M — best vision-per-dollar
- Hybrid uses each where it's strongest

View File

@@ -1,60 +0,0 @@
# QA Pipeline Model Selection
## Current Configuration
| Script | Role | Model | Why |
| --------------------- | -------------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------- |
| `qa-analyze-pr.ts` | PR/issue analysis, QA guide generation | `gemini-3.1-pro-preview` | Needs deep reasoning over PR diffs, screenshots, and issue threads |
| `qa-record.ts` | Playwright step generation | `gemini-3.1-pro-preview` | Step quality is critical — must understand ComfyUI's canvas UI and produce precise action sequences |
| `qa-video-review.ts` | Video comparison review | `gemini-3-flash-preview` | Video analysis with structured output; flash is sufficient and faster |
| `qa-generate-test.ts` | Regression test generation | `gemini-3-flash-preview` | Code generation from QA reports; flash handles this well |
## Model Comparison
### Gemini 3.1 Pro vs GPT-5.4
| | Gemini 3.1 Pro Preview | GPT-5.4 |
| ----------------- | ---------------------- | ----------------- |
| Context window | 1M tokens | 1M tokens |
| Max output | 65K tokens | 128K tokens |
| Video input | Yes | No |
| Image input | Yes | Yes |
| Audio input | Yes | No |
| Pricing (input) | $2/1M tokens | $2.50/1M tokens |
| Pricing (output) | $12/1M tokens | $15/1M tokens |
| Function calling | Yes | Yes |
| Code execution | Yes | Yes (interpreter) |
| Structured output | Yes | Yes |
**Why Gemini over GPT for QA:**
- Native video understanding (can review recordings directly)
- Lower cost at comparable quality
- Native multimodal input (screenshots, videos, audio from issue threads)
- Better price/performance for high-volume CI usage
### Gemini 3 Flash vs GPT-5.4 Mini
| | Gemini 3 Flash Preview | GPT-5.4 Mini |
| ---------------- | ---------------------- | --------------- |
| Context window | 1M tokens | 1M tokens |
| Pricing (input) | $0.50/1M tokens | $0.40/1M tokens |
| Pricing (output) | $3/1M tokens | $1.60/1M tokens |
| Video input | Yes | No |
**Why Gemini Flash for video review:**
- Video input support is required — GPT models cannot process video files
- Good enough quality for structured comparison reports
## Upgrade History
| Date | Change | Reason |
| ---------- | ---------------------------------------------------------------- | -------------------------------------------------------------------------- |
| 2026-03-24 | `gemini-2.5-flash``gemini-3.1-pro-preview` (record) | Shallow step generation; pro model needed for complex ComfyUI interactions |
| 2026-03-24 | `gemini-2.5-pro``gemini-3.1-pro-preview` (analyze) | Keep analysis on latest pro |
| 2026-03-24 | `gemini-2.5-flash``gemini-3-flash-preview` (review, test-gen) | Latest flash for cost-efficient tasks |
## Override
All scripts accept `--model <name>` to override the default. Pass any Gemini model ID.

View File

@@ -17,7 +17,7 @@ This guide covers patterns and examples for testing Pinia stores in the ComfyUI
Basic setup for testing Pinia stores:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
import { createTestingPinia } from '@pinia/testing'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -51,7 +51,7 @@ describe('useWorkflowStore', () => {
Testing store state changes:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
it('should create a temporary workflow with a unique path', () => {
const workflow = store.createTemporary()
expect(workflow.path).toBe('workflows/Unsaved Workflow.json')
@@ -72,7 +72,7 @@ it('should create a temporary workflow not clashing with persisted workflows', a
Testing store actions:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
describe('openWorkflow', () => {
it('should load and open a temporary workflow', async () => {
// Create a test workflow
@@ -115,7 +115,7 @@ describe('openWorkflow', () => {
Testing store getters:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/modelStore.test.ts
describe('getters', () => {
beforeEach(() => {
setActivePinia(createPinia())
@@ -162,7 +162,7 @@ describe('getters', () => {
Mocking API and other dependencies:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
// Add mock for api at the top of the file
vi.mock('@/scripts/api', () => ({
api: {
@@ -205,7 +205,7 @@ describe('syncWorkflows', () => {
Testing store watchers and reactive behavior:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
import { nextTick } from 'vue'
describe('Subgraphs', () => {
@@ -253,7 +253,7 @@ describe('Subgraphs', () => {
Testing store integration with other parts of the application:
```typescript
// Example from a colocated store unit test
// Example from: tests-ui/tests/store/workflowStore.test.ts
describe('renameWorkflow', () => {
it('should rename workflow and update bookmarks', async () => {
const workflow = store.createTemporary('dir/test.json')

View File

@@ -18,7 +18,7 @@ This guide covers patterns and examples for unit testing utilities, composables,
Testing Vue composables requires handling reactivity correctly:
```typescript
// Example from a colocated composable unit test
// Example from: tests-ui/tests/composables/useServerLogs.test.ts
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { nextTick } from 'vue'
import { useServerLogs } from '@/composables/useServerLogs'
@@ -59,7 +59,7 @@ describe('useServerLogs', () => {
Testing LiteGraph-related functionality:
```typescript
// Example from a colocated LiteGraph unit test
// Example from: tests-ui/tests/litegraph.test.ts
import { LGraph, LGraphNode, LiteGraph } from '@/lib/litegraph'
import { describe, expect, it } from 'vitest'
@@ -93,7 +93,7 @@ describe('LGraph', () => {
Testing with ComfyUI workflow files:
```typescript
// Example from a colocated workflow unit test
// Example from: tests-ui/tests/comfyWorkflow.test.ts
import { describe, expect, it } from 'vitest'
import { validateComfyWorkflow } from '@/platform/workflow/validation/schemas/workflowSchema'
import { defaultGraph } from '@/scripts/defaultGraph'
@@ -125,7 +125,7 @@ describe('workflow validation', () => {
Mocking the ComfyUI API object:
```typescript
// Example from a colocated composable unit test
// Example from: tests-ui/tests/composables/useServerLogs.test.ts
import { describe, expect, it, vi } from 'vitest'
import { api } from '@/scripts/api'
@@ -183,7 +183,7 @@ describe('Function using debounce', () => {
When you need to test real debounce/throttle behavior:
```typescript
// Example from a colocated composable unit test
// Example from: tests-ui/tests/composables/useWorkflowAutoSave.test.ts
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
describe('debounced function', () => {
@@ -223,7 +223,7 @@ describe('debounced function', () => {
Creating mock node definitions for testing:
```typescript
// Example from a colocated schema unit test
// Example from: tests-ui/tests/apiTypes.test.ts
import { describe, expect, it } from 'vitest'
import {
type ComfyNodeDef,

View File

@@ -230,6 +230,15 @@ export default defineConfig([
]
}
},
{
files: ['tests-ui/**/*'],
rules: {
'@typescript-eslint/consistent-type-imports': [
'error',
{ disallowTypeAnnotations: false }
]
}
},
{
files: ['**/*.spec.ts'],
ignores: ['browser_tests/tests/**/*.spec.ts'],

View File

@@ -40,7 +40,7 @@ const config: KnipConfig = {
ignoreDependencies: ['@comfyorg/design-system', '@vercel/analytics']
}
},
ignoreBinaries: ['python3', 'wrangler'],
ignoreBinaries: ['python3'],
ignoreDependencies: [
// Weird importmap things
'@iconify-json/lucide',

View File

@@ -1,6 +1,6 @@
{
"name": "@comfyorg/comfyui-frontend",
"version": "1.43.11",
"version": "1.43.9",
"private": true,
"description": "Official front-end implementation of ComfyUI",
"homepage": "https://comfy.org",
@@ -39,14 +39,12 @@
"oxlint": "oxlint src --type-aware",
"prepare": "husky || true && git config blame.ignoreRevsFile .git-blame-ignore-revs || true",
"preview": "nx preview",
"qa:video-review": "tsx scripts/qa-video-review.ts",
"storybook": "nx storybook",
"storybook:desktop": "nx run @comfyorg/desktop-ui:storybook",
"stylelint:fix": "stylelint --cache --fix '{apps,packages,src}/**/*.{css,vue}'",
"stylelint": "stylelint --cache '{apps,packages,src}/**/*.{css,vue}'",
"test:browser": "pnpm exec nx e2e",
"test:browser:local": "cross-env PLAYWRIGHT_LOCAL=1 PLAYWRIGHT_TEST_URL=http://localhost:5173 pnpm test:browser",
"test:coverage": "vitest run --coverage",
"test:unit": "nx run test",
"typecheck": "vue-tsc --noEmit",
"typecheck:browser": "vue-tsc --project browser_tests/tsconfig.json",
@@ -59,7 +57,6 @@
"@atlaskit/pragmatic-drag-and-drop": "^1.3.1",
"@comfyorg/comfyui-electron-types": "catalog:",
"@comfyorg/design-system": "workspace:*",
"@comfyorg/ingest-types": "workspace:*",
"@comfyorg/registry-types": "workspace:*",
"@comfyorg/shared-frontend-utils": "workspace:*",
"@comfyorg/tailwind-utils": "workspace:*",
@@ -123,9 +120,7 @@
"zod-validation-error": "catalog:"
},
"devDependencies": {
"@anthropic-ai/claude-agent-sdk": "catalog:",
"@eslint/js": "catalog:",
"@google/generative-ai": "catalog:",
"@intlify/eslint-plugin-vue-i18n": "catalog:",
"@lobehub/i18n-cli": "catalog:",
"@nx/eslint": "catalog:",
@@ -143,7 +138,6 @@
"@testing-library/jest-dom": "catalog:",
"@testing-library/user-event": "catalog:",
"@testing-library/vue": "catalog:",
"@total-typescript/shoehorn": "catalog:",
"@types/fs-extra": "catalog:",
"@types/jsdom": "catalog:",
"@types/node": "catalog:",

561
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -4,12 +4,10 @@ packages:
catalog:
'@alloc/quick-lru': ^5.2.0
'@anthropic-ai/claude-agent-sdk': ^0.2.85
'@astrojs/vue': ^5.0.0
'@comfyorg/comfyui-electron-types': 0.6.2
'@eslint/js': ^9.39.1
'@formkit/auto-animate': ^0.9.0
'@google/generative-ai': ^0.24.1
'@iconify-json/lucide': ^1.1.178
'@iconify/json': ^2.2.380
'@iconify/tailwind4': ^1.2.0
@@ -48,7 +46,6 @@ catalog:
'@tiptap/extension-table-row': ^2.27.2
'@tiptap/pm': 2.27.2
'@tiptap/starter-kit': ^2.27.2
'@total-typescript/shoehorn': ^0.1.2
'@types/fs-extra': ^11.0.4
'@types/jsdom': ^21.1.7
'@types/node': ^24.1.0

View File

@@ -1,572 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
*
* Instead of driving a browser interactively, Claude:
* 1. Reads the issue + a11y snapshot of the UI
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
* 3. Runs the test → reads errors → rewrites → repeats until it works
* 4. Outputs the passing test + verdict
*
* Tools:
* - inspect(selector) — read a11y tree to understand UI state
* - writeTest(code) — write a Playwright test file
* - runTest() — execute the test and get results
* - done(verdict, summary, testCode) — finish with the working test
*/
import type { Page } from '@playwright/test'
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
import { z } from 'zod'
import { mkdirSync, readFileSync, writeFileSync } from 'fs'
import { execSync } from 'child_process'
// ── Types ──
interface ResearchOptions {
page: Page
issueContext: string
qaGuide: string
outputDir: string
serverUrl: string
anthropicApiKey?: string
maxTurns?: number
timeBudgetMs?: number
}
export type ReproMethod = 'e2e_test' | 'video' | 'both' | 'none'
export interface ResearchResult {
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
reproducedBy: ReproMethod
summary: string
evidence: string
testCode: string
log: Array<{
turn: number
timestampMs: number
toolName: string
toolInput: unknown
toolResult: string
}>
}
// ── Main research function ──
export async function runResearchPhase(
opts: ResearchOptions
): Promise<ResearchResult> {
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
opts
const maxTurns = opts.maxTurns ?? 50
let agentDone = false
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
let finalReproducedBy: ReproMethod = 'none'
let finalSummary = 'Agent did not complete'
let finalEvidence = ''
let finalTestCode = ''
let turnCount = 0
let lastPassedTurn = -1
const startTime = Date.now()
const researchLog: ResearchResult['log'] = []
const testDir = `${outputDir}/research`
mkdirSync(testDir, { recursive: true })
const testPath = `${testDir}/reproduce.spec.ts`
// Get initial a11y snapshot for context
let initialA11y = ''
try {
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
initialA11y = initialA11y.slice(0, 3000)
} catch {
initialA11y = '(could not capture initial a11y snapshot)'
}
// ── Tool: inspect ──
const inspectTool = tool(
'inspect',
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
{
selector: z
.string()
.optional()
.describe(
'Optional filter — only show elements matching this name/role. Omit for full tree.'
)
},
async (args) => {
let resultText: string
try {
const ariaText = await page
.locator('body')
.ariaSnapshot({ timeout: 5000 })
if (args.selector) {
const lines = ariaText.split('\n')
const matches = lines.filter((l: string) =>
l.toLowerCase().includes(args.selector!.toLowerCase())
)
resultText =
matches.length > 0
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
} else {
resultText = ariaText.slice(0, 3000)
}
} catch (e) {
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'inspect',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readFixture ──
const readFixtureTool = tool(
'readFixture',
'Read a fixture or helper file from browser_tests/fixtures/ to understand the API. Use this to discover available methods on comfyPage helpers before writing your test.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/fixtures/, e.g. "helpers/CanvasHelper.ts" or "components/Topbar.ts" or "ComfyPage.ts"'
)
},
async (args) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/fixtures/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
resultText = `Could not read fixture: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readFixture',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readTest ──
const readTestTool = tool(
'readTest',
'Read an existing E2E test file from browser_tests/tests/ to learn patterns and conventions used in this project.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/tests/, e.g. "workflow.spec.ts" or "subgraph.spec.ts"'
)
},
async (args) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/tests/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
// List available test files if the path doesn't exist
try {
const { readdirSync } = await import('fs')
const files = readdirSync(`${projectRoot}/browser_tests/tests/`)
.filter((f: string) => f.endsWith('.spec.ts'))
.slice(0, 30)
resultText = `File not found: ${args.path}\n\nAvailable test files:\n${files.join('\n')}`
} catch {
resultText = `Could not read test: ${e instanceof Error ? e.message : e}`
}
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readTest',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: writeTest ──
const writeTestTool = tool(
'writeTest',
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
{
code: z
.string()
.describe('Complete Playwright test file content (.spec.ts)')
},
async (args) => {
writeFileSync(testPath, args.code)
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'writeTest',
toolInput: { path: testPath, codeLength: args.code.length },
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
})
return {
content: [
{
type: 'text' as const,
text: `Test written to ${testPath}. Use runTest() to execute it.`
}
]
}
}
)
// ── Tool: runTest ──
// Place test in browser_tests/ so Playwright config finds fixtures
const projectRoot = process.cwd()
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
const runTestTool = tool(
'runTest',
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
{},
async () => {
turnCount++
// Copy the test to browser_tests/tests/ where Playwright expects it
const { copyFileSync } = await import('fs')
try {
copyFileSync(testPath, browserTestPath)
} catch {
// directory may not exist
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
copyFileSync(testPath, browserTestPath)
}
let resultText: string
try {
const output = execSync(
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
{
timeout: 90000,
encoding: 'utf-8',
env: {
...process.env,
COMFYUI_BASE_URL: serverUrl
}
}
)
resultText = `TEST PASSED:\n${output.slice(-1500)}`
} catch (e) {
const err = e as { stdout?: string; stderr?: string; message?: string }
const output = (err.stdout || '') + '\n' + (err.stderr || '')
resultText = `TEST FAILED:\n${output.slice(-2000)}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'runTest',
toolInput: { testPath },
toolResult: resultText.slice(0, 1000)
})
// Auto-save passing test code for fallback completion
if (resultText.startsWith('TEST PASSED')) {
try {
finalTestCode = readFileSync(browserTestPath, 'utf-8')
lastPassedTurn = turnCount
} catch {
// ignore
}
resultText += '\n\n⚠ Test PASSED — call done() now with verdict REPRODUCED and the test code. Do NOT write more tests.'
}
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: done ──
const doneTool = tool(
'done',
'Finish research with verdict and the final test code.',
{
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
reproducedBy: z
.enum(['e2e_test', 'video', 'both', 'none'])
.describe(
'How the bug was proven: e2e_test = Playwright assertion passed, video = visual evidence only, both = both methods, none = not reproduced'
),
summary: z.string().describe('What you found and why'),
evidence: z.string().describe('Test output that proves the verdict'),
testCode: z
.string()
.describe(
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
)
},
async (args) => {
agentDone = true
finalVerdict = args.verdict
finalReproducedBy = args.reproducedBy
finalSummary = args.summary
finalEvidence = args.evidence
finalTestCode = args.testCode
writeFileSync(testPath, args.testCode)
return {
content: [
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
]
}
}
)
// ── MCP Server ──
const server = createSdkMcpServer({
name: 'qa-research',
version: '1.0.0',
tools: [
inspectTool,
readFixtureTool,
readTestTool,
writeTestTool,
runTestTool,
doneTool
]
})
// ── System prompt ──
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
## Your tools
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
- readFixture(path) — Read fixture source code from browser_tests/fixtures/. Use to discover available methods. E.g. "helpers/CanvasHelper.ts", "components/Topbar.ts", "ComfyPage.ts"
- readTest(path) — Read an existing test from browser_tests/tests/ to learn patterns. E.g. "workflow.spec.ts". Pass any name to list available files.
- writeTest(code) — Write a Playwright test file (.spec.ts)
- runTest() — Execute the test and get results (pass/fail + errors)
- done(verdict, summary, evidence, testCode) — Finish with the final test
## Workflow
1. Read the issue description carefully
2. Use inspect() to understand the current UI state and discover element selectors
3. If unsure about the fixture API, use readFixture() to read the relevant helper source code
4. If unsure about test patterns, use readTest() to read an existing test for reference
5. Write a Playwright test that:
- Performs the exact reproduction steps from the issue
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
6. Run the test with runTest()
7. If it fails: read the error, fix the test, run again (max 5 attempts)
8. Call done() with the final verdict and test code
## Test writing guidelines
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
- Import expect: \`import { expect } from '@playwright/test'\`
- The fixture provides \`comfyPage\` which has all the helpers listed below
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
- Keep tests focused and minimal — test ONLY the reported bug
- Write ONE test, not multiple. Focus on the single clearest reproduction.
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
- Use \`comfyPage.nextFrame()\` after interactions that trigger UI updates
- NEVER use \`page.waitForTimeout()\` — use Locator actions and retrying assertions instead
- ALWAYS call done() when finished, even if the test passed — do not keep iterating after a passing test
- Use \`expect.poll()\` for async assertions: \`await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(8)\`
- CRITICAL: Your assertions must be SPECIFIC TO THE BUG. A test that asserts \`expect(count).toBeGreaterThan(0)\` proves nothing — it would pass even without the bug. Instead assert the exact broken state, e.g. \`expect(clonedWidgets).toHaveLength(0)\` (missing widgets) or \`expect(zIndex).toBeLessThan(parentZIndex)\` (wrong z-order). If a test passes trivially, it's a false positive.
- If you cannot write a bug-specific assertion, call done() with verdict NOT_REPRODUCIBLE and explain why.
## ComfyPage Fixture API Reference
### Core properties
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.canvas\` — Locator for #graph-canvas
- \`comfyPage.queueButton\` — "Queue Prompt" button
- \`comfyPage.runButton\` — "Run" button (new UI)
- \`comfyPage.confirmDialog\` — ConfirmDialog (has .confirm, .delete, .overwrite, .reject locators + .click(name) method)
- \`comfyPage.nextFrame()\` — wait for next requestAnimationFrame
- \`comfyPage.setup()\` — navigate + wait for app ready (called automatically by fixture)
### Menu (comfyPage.menu)
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.triggerTopbarCommand(['File', 'Save As'])\` — navigate menu hierarchy
- \`.openTopbarMenu()\` / \`.closeTopbarMenu()\` — open/close hamburger
- \`.openSubmenu('File')\` — hover to open submenu, returns submenu Locator
- \`.getTabNames()\` — get all open workflow tab names
- \`.getActiveTabName()\` — get active tab name
- \`.getWorkflowTab(name)\` — get tab Locator
- \`.closeWorkflowTab(name)\` — close a tab
- \`.saveWorkflow(name)\` / \`.saveWorkflowAs(name)\` / \`.exportWorkflow(name)\`
- \`.switchTheme('dark' | 'light')\`
- \`comfyPage.menu.workflowsTab\` — WorkflowsSidebarTab:
- \`.open()\` / \`.close()\` — toggle workflows sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflow names
- \`comfyPage.menu.nodeLibraryTab\` — NodeLibrarySidebarTab
- \`comfyPage.menu.assetsTab\` — AssetsSidebarTab
### Canvas (comfyPage.canvasOps)
- \`.click({x, y})\` — click at position on canvas
- \`.rightClick(x, y)\` — right-click (opens context menu)
- \`.doubleClick()\` — double-click canvas (opens node search)
- \`.clickEmptySpace()\` — click known empty area
- \`.dragAndDrop(source, target)\` — drag from source to target position
- \`.pan(offset, safeSpot?)\` — pan canvas by offset
- \`.zoom(deltaY, steps?)\` — zoom via scroll wheel
- \`.resetView()\` — reset zoom/pan to default
- \`.getScale()\` / \`.setScale(n)\` — get/set canvas zoom
- \`.getNodeCenterByTitle(title)\` — get screen coords of node center
- \`.disconnectEdge()\` / \`.connectEdge()\` — default graph edge operations
### Node Operations (comfyPage.nodeOps)
- \`.getGraphNodesCount()\` — count all nodes
- \`.getSelectedGraphNodesCount()\` — count selected nodes
- \`.getNodes()\` — get all nodes
- \`.getFirstNodeRef()\` — get NodeReference for first node
- \`.getNodeRefById(id)\` — get NodeReference by ID
- \`.getNodeRefsByType(type)\` — get all nodes of a type
- \`.waitForGraphNodes(count)\` — wait until node count matches
### Settings (comfyPage.settings)
- \`.setSetting(id, value)\` — change a ComfyUI setting
- \`.getSetting(id)\` — read current setting value
### Keyboard (comfyPage.keyboard)
- \`.undo()\` / \`.redo()\` — Ctrl+Z / Ctrl+Y
- \`.bypass()\` — Ctrl+B
- \`.selectAll()\` — Ctrl+A
- \`.ctrlSend(key)\` — send Ctrl+key
### Workflow (comfyPage.workflow)
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\`
- \`.isCurrentWorkflowModified()\` — check dirty state
### Context Menu (comfyPage.contextMenu)
- \`.openFor(locator)\` — right-click locator and wait for menu
- \`.clickMenuItem(name)\` — click a menu item by name
- \`.isVisible()\` — check if context menu is showing
- \`.assertHasItems(items)\` — assert menu contains items
### Other helpers
- \`comfyPage.settingDialog\` — SettingDialog component
- \`comfyPage.searchBox\` / \`comfyPage.searchBoxV2\` — node search
- \`comfyPage.toast\` — ToastHelper (\`.visibleToasts\`)
- \`comfyPage.subgraph\` — SubgraphHelper
- \`comfyPage.vueNodes\` — VueNodeHelpers
- \`comfyPage.bottomPanel\` — BottomPanel
- \`comfyPage.clipboard\` — ClipboardHelper
- \`comfyPage.dragDrop\` — DragDropHelper
### Available fixture files (use readFixture to explore)
- ComfyPage.ts — main fixture with all helpers
- helpers/CanvasHelper.ts, NodeOperationsHelper.ts, WorkflowHelper.ts
- helpers/KeyboardHelper.ts, SettingsHelper.ts, SubgraphHelper.ts
- components/Topbar.ts, ContextMenu.ts, SettingDialog.ts, SidebarTab.ts
## Current UI state (accessibility tree)
${initialA11y}
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
## Issue to Reproduce
${issueContext}`
// ── Run the agent ──
console.warn('Starting research phase (Claude writes E2E tests)...')
try {
for await (const message of query({
prompt:
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, readFixture() or readTest() if you need to understand the fixture API or see existing test patterns, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
options: {
model: 'claude-sonnet-4-6',
systemPrompt,
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
maxTurns,
mcpServers: { 'qa-research': server },
allowedTools: [
'mcp__qa-research__inspect',
'mcp__qa-research__readFixture',
'mcp__qa-research__readTest',
'mcp__qa-research__writeTest',
'mcp__qa-research__runTest',
'mcp__qa-research__done'
]
}
})) {
if (message.type === 'assistant' && message.message?.content) {
for (const block of message.message.content) {
if ('text' in block && block.text) {
console.warn(` Claude: ${block.text.slice(0, 200)}`)
}
if ('name' in block) {
console.warn(
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
)
}
}
}
if (agentDone) break
}
} catch (e) {
const errMsg = e instanceof Error ? e.message : String(e)
console.warn(`Research error: ${errMsg}`)
// Detect billing/auth errors and surface them clearly
if (
errMsg.includes('Credit balance is too low') ||
errMsg.includes('insufficient_quota') ||
errMsg.includes('rate_limit')
) {
finalSummary = `API error: ${errMsg.slice(0, 200)}`
finalEvidence = 'Agent could not start due to API billing/auth issue'
console.warn(
'::error::Anthropic API credits exhausted — cannot run research phase'
)
}
}
// Auto-complete: if a test passed but done() was never called, use the passing test
if (!agentDone && lastPassedTurn >= 0 && finalTestCode) {
console.warn(
`Auto-completing: test passed at turn ${lastPassedTurn} but done() was not called`
)
finalVerdict = 'REPRODUCED'
finalReproducedBy = 'e2e_test'
finalSummary = `Test passed at turn ${lastPassedTurn} (auto-completed — agent did not call done())`
finalEvidence = `Test passed with exit code 0`
}
const result: ResearchResult = {
verdict: finalVerdict,
reproducedBy: finalReproducedBy,
summary: finalSummary,
evidence: finalEvidence,
testCode: finalTestCode,
log: researchLog
}
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
console.warn(
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
)
return result
}

View File

@@ -1,84 +0,0 @@
import { describe, expect, it } from 'vitest'
import { extractMediaUrls } from './qa-analyze-pr'
describe('extractMediaUrls', () => {
it('extracts markdown image URLs', () => {
const text = '![screenshot](https://example.com/image.png)'
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
})
it('extracts multiple markdown images', () => {
const text = [
'![before](https://example.com/before.png)',
'Some text',
'![after](https://example.com/after.jpg)'
].join('\n')
expect(extractMediaUrls(text)).toEqual([
'https://example.com/before.png',
'https://example.com/after.jpg'
])
})
it('extracts raw URLs with media extensions', () => {
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
})
it('extracts GitHub user-attachments URLs', () => {
const text =
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
expect(extractMediaUrls(text)).toEqual([
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
])
})
it('extracts private-user-images URLs', () => {
const text =
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
expect(extractMediaUrls(text)).toEqual([
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
])
})
it('extracts URLs with query parameters', () => {
const text = 'https://example.com/image.png?w=800&h=600'
expect(extractMediaUrls(text)).toEqual([
'https://example.com/image.png?w=800&h=600'
])
})
it('deduplicates URLs', () => {
const text = [
'![img](https://example.com/same.png)',
'![img2](https://example.com/same.png)',
'Also https://example.com/same.png'
].join('\n')
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
})
it('returns empty array for empty input', () => {
expect(extractMediaUrls('')).toEqual([])
})
it('returns empty array for text with no media URLs', () => {
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
})
it('handles mixed media types', () => {
const text = [
'![screen](https://example.com/screenshot.png)',
'Video: https://example.com/demo.webm',
'![gif](https://example.com/animation.gif)'
].join('\n')
const urls = extractMediaUrls(text)
expect(urls).toContain('https://example.com/screenshot.png')
expect(urls).toContain('https://example.com/demo.webm')
expect(urls).toContain('https://example.com/animation.gif')
})
it('ignores non-http URLs in markdown', () => {
const text = '![local](./local-image.png)'
expect(extractMediaUrls(text)).toEqual([])
})
})

View File

@@ -1,799 +0,0 @@
#!/usr/bin/env tsx
/**
* QA PR Analysis Script
*
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
* for before/after recording sessions. Fetches PR thread, extracts media,
* and produces structured test plans.
*
* Usage:
* pnpm exec tsx scripts/qa-analyze-pr.ts \
* --pr-number 10270 \
* --repo owner/repo \
* --output-dir qa-guides/ \
* [--model gemini-3.1-pro-preview]
*
* Env: GEMINI_API_KEY (required)
*/
import { execSync } from 'node:child_process'
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
// ── Types ──
interface QaGuideStep {
action: string
description: string
expected_before?: string
expected_after?: string
}
interface QaGuide {
summary: string
test_focus: string
prerequisites: string[]
steps: QaGuideStep[]
visual_checks: string[]
}
interface PrThread {
title: string
body: string
labels: string[]
issueComments: string[]
reviewComments: string[]
reviews: string[]
diff: string
}
type TargetType = 'pr' | 'issue'
interface Options {
prNumber: string
repo: string
outputDir: string
model: string
apiKey: string
mediaBudgetBytes: number
maxVideoBytes: number
type: TargetType
}
// ── CLI parsing ──
function parseArgs(): Options {
const args = process.argv.slice(2)
const opts: Partial<Options> = {
model: 'gemini-3.1-pro-preview',
apiKey: process.env.GEMINI_API_KEY || '',
mediaBudgetBytes: 20 * 1024 * 1024,
maxVideoBytes: 10 * 1024 * 1024,
type: 'pr'
}
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--pr-number':
opts.prNumber = args[++i]
break
case '--repo':
opts.repo = args[++i]
break
case '--output-dir':
opts.outputDir = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--type':
opts.type = args[++i] as TargetType
break
case '--help':
console.warn(
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
)
process.exit(0)
}
}
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
console.error(
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
)
process.exit(1)
}
if (!opts.apiKey) {
console.error('GEMINI_API_KEY environment variable is required')
process.exit(1)
}
return opts as Options
}
// ── PR thread fetching ──
function ghExec(cmd: string): string {
try {
return execSync(cmd, {
encoding: 'utf-8',
timeout: 30_000,
stdio: ['pipe', 'pipe', 'pipe']
}).trim()
} catch (err) {
console.warn(`gh command failed: ${cmd}`)
console.warn((err as Error).message)
return ''
}
}
function fetchPrThread(prNumber: string, repo: string): PrThread {
console.warn('Fetching PR thread...')
const prView = ghExec(
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
)
const prData = prView
? JSON.parse(prView)
: { title: '', body: '', labels: [] }
const issueCommentsRaw = ghExec(
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
)
const issueComments: string[] = issueCommentsRaw
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewCommentsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
)
const reviewComments: string[] = reviewCommentsRaw
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
)
const reviews: string[] = reviewsRaw
? JSON.parse(reviewsRaw)
.filter((r: { body: string }) => r.body)
.map((r: { body: string }) => r.body)
: []
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
console.warn(
`PR #${prNumber}: "${prData.title}" | ` +
`${issueComments.length} issue comments, ` +
`${reviewComments.length} review comments, ` +
`${reviews.length} reviews, ` +
`diff: ${diff.length} chars`
)
return {
title: prData.title || '',
body: prData.body || '',
labels: (prData.labels || []).map((l: { name: string }) => l.name),
issueComments,
reviewComments,
reviews,
diff
}
}
interface IssueThread {
title: string
body: string
labels: string[]
comments: string[]
}
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
console.warn('Fetching issue thread...')
const issueView = ghExec(
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
)
const issueData = issueView
? JSON.parse(issueView)
: { title: '', body: '', labels: [] }
const commentsRaw = ghExec(
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
)
const comments: string[] = commentsRaw
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
: []
console.warn(
`Issue #${issueNumber}: "${issueData.title}" | ` +
`${comments.length} comments`
)
return {
title: issueData.title || '',
body: issueData.body || '',
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
comments
}
}
// ── Media extraction ──
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
const MEDIA_URL_PATTERNS = [
// Markdown images: ![alt](url)
/!\[[^\]]*\]\(([^)]+)\)/g,
// GitHub user-attachments
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
// Private user images
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
// Raw URLs with media extensions (standalone or in text)
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
]
export function extractMediaUrls(text: string): string[] {
if (!text) return []
const urls = new Set<string>()
for (const pattern of MEDIA_URL_PATTERNS) {
// Reset lastIndex for global patterns
pattern.lastIndex = 0
let match: RegExpExecArray | null
while ((match = pattern.exec(text)) !== null) {
// For markdown images, the URL is in capture group 1
const url = match[1] || match[0]
// Clean trailing markdown/html artifacts
const cleaned = url.replace(/[)>"'\s]+$/, '')
if (cleaned.startsWith('http')) {
urls.add(cleaned)
}
}
}
return [...urls]
}
// ── Media downloading ──
const ALLOWED_MEDIA_DOMAINS = [
'github.com',
'raw.githubusercontent.com',
'user-images.githubusercontent.com',
'private-user-images.githubusercontent.com',
'objects.githubusercontent.com',
'github.githubassets.com'
]
function isAllowedMediaDomain(url: string): boolean {
try {
const hostname = new URL(url).hostname
return ALLOWED_MEDIA_DOMAINS.some(
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
)
} catch {
return false
}
}
async function downloadMedia(
urls: string[],
outputDir: string,
budgetBytes: number,
maxVideoBytes: number
): Promise<Array<{ path: string; mimeType: string }>> {
const downloaded: Array<{ path: string; mimeType: string }> = []
let totalBytes = 0
const mediaDir = resolve(outputDir, 'media')
mkdirSync(mediaDir, { recursive: true })
for (const url of urls) {
if (totalBytes >= budgetBytes) {
console.warn(
`Media budget exhausted (${totalBytes} bytes), skipping rest`
)
break
}
if (!isAllowedMediaDomain(url)) {
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
continue
}
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(15_000),
headers: { Accept: 'image/*,video/*' },
redirect: 'follow'
})
if (!response.ok) {
console.warn(`Failed to download ${url}: ${response.status}`)
continue
}
const contentLength = response.headers.get('content-length')
if (contentLength) {
const declaredSize = Number.parseInt(contentLength, 10)
if (declaredSize > budgetBytes - totalBytes) {
console.warn(
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
)
continue
}
}
const contentType = response.headers.get('content-type') || ''
const buffer = Buffer.from(await response.arrayBuffer())
// Skip oversized videos
const isVideo =
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
if (isVideo && buffer.length > maxVideoBytes) {
console.warn(
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
)
continue
}
if (totalBytes + buffer.length > budgetBytes) {
console.warn(`Would exceed budget, skipping ${url}`)
continue
}
const ext = guessExtension(url, contentType)
const filename = `media-${downloaded.length}${ext}`
const filepath = resolve(mediaDir, filename)
writeFileSync(filepath, buffer)
totalBytes += buffer.length
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
downloaded.push({ path: filepath, mimeType })
console.warn(
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
)
} catch (err) {
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
}
}
console.warn(
`Downloaded ${downloaded.length}/${urls.length} media files ` +
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
)
return downloaded
}
function guessExtension(url: string, contentType: string): string {
const urlMatch = url.match(MEDIA_EXTENSIONS)
if (urlMatch) return urlMatch[0].toLowerCase()
const typeMap: Record<string, string> = {
'image/png': '.png',
'image/jpeg': '.jpg',
'image/gif': '.gif',
'image/webp': '.webp',
'video/mp4': '.mp4',
'video/webm': '.webm'
}
return typeMap[contentType.split(';')[0]] || '.bin'
}
function guessMimeType(ext: string): string {
const map: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime'
}
return map[ext] || 'application/octet-stream'
}
// ── Gemini analysis ──
function buildIssueAnalysisPrompt(issue: IssueThread): string {
const allText = [
`# Issue: ${issue.title}`,
'',
'## Description',
issue.body,
'',
issue.comments.length > 0
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
: ''
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
The UI has:
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
- Nodes are boxes with input/output slots connected by wires
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
- Sidebars (Workflows, Node Library, Models)
- A topbar with workflow tabs and Queue button
- The default workflow loads with these nodes (approximate center coordinates):
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
${allText}
## Available test actions
Each step must use one of these actions:
### Menu actions
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
### Element actions (by visible text)
- "click" — clicks an element by visible text (text required)
- "rightClick" — right-clicks an element to open context menu (text required)
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
### Canvas actions (by coordinates — viewport is 1280x720)
- "clickCanvas" — click at coordinates (x, y required)
- "rightClickCanvas" — right-click at coordinates (x, y required)
- "doubleClick" — double-click at coordinates to open node search (x, y)
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
### Utility
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Common ComfyUI interactions
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
- Double-click empty canvas → opens node search dialog
- Ctrl+C / Ctrl+V → copy/paste selected nodes
- Delete key → remove selected node
- Ctrl+G → group selected nodes
- Drag from output slot to input slot → create connection
- Click a node to select it, Shift+click for multi-select
## Output format
Return a JSON object with exactly one key: "reproduce", containing:
{
"summary": "One sentence: what bug this issue reports",
"test_focus": "Specific behavior to reproduce",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "clickCanvas",
"description": "Click on first node to select it",
"expected_before": "What should happen if the bug is present"
}
],
"visual_checks": ["Specific visual evidence of the bug to look for"]
}
## Rules
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
- Do NOT include login steps.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
function buildAnalysisPrompt(thread: PrThread): string {
const allText = [
`# PR: ${thread.title}`,
'',
'## Description',
thread.body,
'',
thread.issueComments.length > 0
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
: '',
thread.reviewComments.length > 0
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
: '',
thread.reviews.length > 0
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
: '',
'',
'## Diff (truncated)',
'```',
thread.diff.slice(0, 8000),
'```'
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
${allText}
## Available test actions
Each step must use one of these actions:
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required)
- "click" — clicks an element by visible text (text required)
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Output format
Return a JSON object with exactly two keys: "before" and "after", each containing:
{
"summary": "One sentence: what this PR changes",
"test_focus": "Specific behaviors to verify in this recording",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "openMenu",
"description": "Open the main menu to check file options",
"expected_before": "Old behavior description (before key only)",
"expected_after": "New behavior description (after key only)"
}
],
"visual_checks": ["Specific visual elements to look for"]
}
## Rules
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
- Use information from PR description, screenshots, and comments to understand intended behavior.
- Include at least one screenshot step in each guide.
- Do NOT include login steps.
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
async function analyzeWithGemini(
thread: PrThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<{ before: QaGuide; after: QaGuide }> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildAnalysisPrompt(thread)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
// Add media as inline data
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
// Strip markdown fences if present
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
// Handle different response shapes from Gemini
let before: QaGuide
let after: QaGuide
if (Array.isArray(parsed) && parsed.length >= 2) {
// Array format: [before, after]
before = parsed[0]
after = parsed[1]
} else if (parsed.before && parsed.after) {
// Object format: { before, after }
before = parsed.before
after = parsed.after
} else {
// Try nested wrapper keys
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
if (inner.before && inner.after) {
before = inner.before
after = inner.after
} else {
console.warn(
'Full response:',
JSON.stringify(parsed, null, 2).slice(0, 2000)
)
throw new Error(
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
)
}
}
return { before, after }
}
async function analyzeIssueWithGemini(
issue: IssueThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<QaGuide> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildIssueAnalysisPrompt(issue)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
const guide: QaGuide =
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
return guide
}
// ── Main ──
async function main() {
const opts = parseArgs()
mkdirSync(opts.outputDir, { recursive: true })
if (opts.type === 'issue') {
await analyzeIssue(opts)
} else {
await analyzePr(opts)
}
}
async function analyzeIssue(opts: Options) {
const issue = fetchIssueThread(opts.prNumber, opts.repo)
const allText = [issue.body, ...issue.comments].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guide = await analyzeIssueWithGemini(
issue,
media,
opts.model,
opts.apiKey
)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
console.warn(`Wrote QA guide:`)
console.warn(` Reproduce: ${beforePath}`)
}
async function analyzePr(opts: Options) {
const thread = fetchPrThread(opts.prNumber, opts.repo)
const allText = [
thread.body,
...thread.issueComments,
...thread.reviewComments,
...thread.reviews
].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
console.warn(`Wrote QA guides:`)
console.warn(` Before: ${beforePath}`)
console.warn(` After: ${afterPath}`)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
if (isExecutedAsScript(import.meta.url)) {
main().catch((err) => {
console.error('PR analysis failed:', err)
process.exit(1)
})
}

View File

@@ -1,176 +0,0 @@
#!/usr/bin/env bash
# Batch-trigger QA runs by creating and pushing sno-qa-* branches.
#
# Usage:
# ./scripts/qa-batch.sh 10394 10238 9996 # Trigger specific numbers
# ./scripts/qa-batch.sh --from tmp/issues.md --top 5 # From triage file
# ./scripts/qa-batch.sh --dry-run 10394 10238 # Preview only
# ./scripts/qa-batch.sh --cleanup # Delete old sno-qa-* branches
set -euo pipefail
DELAY=5
DRY_RUN=false
CLEANUP=false
FROM_FILE=""
TOP_N=0
NUMBERS=()
die() { echo "error: $*" >&2; exit 1; }
usage() {
cat <<'EOF'
Usage: qa-batch.sh [options] [numbers...]
Options:
--from <file> Extract numbers from a triage markdown file
--top <N> Take first N entries from Tier 1 (requires --from)
--dry-run Print what would happen without pushing
--cleanup Delete all sno-qa-* remote branches
--delay <secs> Seconds between pushes (default: 5)
-h, --help Show this help
EOF
exit 0
}
# --- Parse args ---
while [[ $# -gt 0 ]]; do
case "$1" in
--from) FROM_FILE="$2"; shift 2 ;;
--top) TOP_N="$2"; shift 2 ;;
--dry-run) DRY_RUN=true; shift ;;
--cleanup) CLEANUP=true; shift ;;
--delay) DELAY="$2"; shift 2 ;;
-h|--help) usage ;;
-*) die "unknown option: $1" ;;
*) NUMBERS+=("$1"); shift ;;
esac
done
# --- Cleanup mode ---
if $CLEANUP; then
echo "Fetching remote sno-qa-* branches..."
branches=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' | awk '{print $2}' | sed 's|refs/heads/||')
if [[ -z "$branches" ]]; then
echo "No sno-qa-* branches found on remote."
exit 0
fi
echo "Found branches:"
while IFS= read -r b; do echo " $b"; done <<< "$branches"
echo
if $DRY_RUN; then
echo "[dry-run] Would delete the above branches."
exit 0
fi
read -rp "Delete all of the above? [y/N] " confirm
if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then
echo "Aborted."
exit 0
fi
for branch in $branches; do
echo "Deleting origin/$branch..."
git push origin --delete "$branch"
done
echo "Done. Cleaned up $(echo "$branches" | wc -l | tr -d ' ') branches."
exit 0
fi
# --- Extract numbers from markdown ---
if [[ -n "$FROM_FILE" ]]; then
[[ -f "$FROM_FILE" ]] || die "file not found: $FROM_FILE"
[[ "$TOP_N" -gt 0 ]] || die "--top N required with --from"
# Extract Tier 1 table rows: | N | [#NNNNN](...) | ...
# Stop at the next ## heading after Tier 1
extracted=$(awk '/^## Tier 1/,/^## Tier [^1]/' "$FROM_FILE" \
| grep -oP '\[#\K\d+' \
| head -n "$TOP_N")
if [[ -z "$extracted" ]]; then
die "no numbers found in $FROM_FILE"
fi
while IFS= read -r num; do
NUMBERS+=("$num")
done <<< "$extracted"
fi
[[ ${#NUMBERS[@]} -gt 0 ]] || die "no numbers specified. Use positional args or --from/--top."
# --- Validate ---
for num in "${NUMBERS[@]}"; do
[[ "$num" =~ ^[0-9]+$ ]] || die "invalid number: $num"
done
# Deduplicate
# shellcheck disable=SC2207 # mapfile not available on macOS default bash
NUMBERS=($(printf '%s\n' "${NUMBERS[@]}" | sort -un))
# --- Push branches ---
echo "Triggering QA for: ${NUMBERS[*]}"
if $DRY_RUN; then
echo "[dry-run]"
fi
echo
pushed=()
skipped=()
# Fetch remote refs once
remote_refs=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' 2>/dev/null | awk '{print $2}' | sed 's|refs/heads/||')
for num in "${NUMBERS[@]}"; do
branch="sno-qa-$num"
# Check if already exists on remote
if echo "$remote_refs" | grep -qx "$branch"; then
echo " skip: $branch (already exists on remote)"
skipped+=("$num")
continue
fi
if $DRY_RUN; then
echo " would push: $branch"
pushed+=("$num")
continue
fi
# Create branch at current HEAD and push
git branch -f "$branch" HEAD
git push origin "$branch"
pushed+=("$num")
echo " pushed: $branch"
# Clean up local branch
git branch -D "$branch" 2>/dev/null || true
# Delay between pushes to avoid CI concurrency storm
if [[ "$num" != "${NUMBERS[-1]}" ]]; then
echo " waiting ${DELAY}s..."
sleep "$DELAY"
fi
done
# --- Summary ---
echo
echo "=== Summary ==="
echo "Triggered: ${#pushed[@]}"
echo "Skipped: ${#skipped[@]}"
if [[ ${#pushed[@]} -gt 0 ]]; then
echo
echo "Triggered numbers: ${pushed[*]}"
repo_url=$(git remote get-url origin | sed 's/\.git$//' | sed 's|git@github.com:|https://github.com/|')
echo "Actions: ${repo_url}/actions"
fi
if [[ ${#skipped[@]} -gt 0 ]]; then
echo
echo "Skipped (already exist): ${skipped[*]}"
echo "Use --cleanup first to remove old branches."
fi

View File

@@ -1,381 +0,0 @@
#!/usr/bin/env bash
# Deploy QA report to Cloudflare Pages.
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
# Writes outputs to GITHUB_OUTPUT: badge_status, url
set -euo pipefail
npm install -g wrangler@4.74.0 >/dev/null 2>&1
DEPLOY_DIR=$(mktemp -d)
mkdir -p "$DEPLOY_DIR"
for os in Linux macOS Windows; do
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
for prefix in qa qa-before; do
VID="${DIR}/${prefix}-session.mp4"
if [ -f "$VID" ]; then
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
cp "$VID" "$DEST"
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
fi
done
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
for numbered in "$DIR"/qa-session-[0-9].mp4; do
[ -f "$numbered" ] || continue
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
cp "$numbered" "$DEST"
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
done
# Generate GIF thumbnail from after video (or first pass)
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
if [ -f "$THUMB_SRC" ]; then
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| echo "GIF generation failed for ${os} (non-fatal)"
fi
done
# Build video cards and report sections
CARDS=""
# shellcheck disable=SC2034 # accessed via eval
ICONS_Linux="&#x1F427;" ICONS_macOS="&#x1F34E;" ICONS_Windows="&#x1FA9F;"
CARD_COUNT=0
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
for os in Linux macOS Windows; do
eval "ICON=\$ICONS_${os}"
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
[ "$HAS_AFTER" = "0" ] && continue
# Collect all reports for this platform (single + multi-pass)
REPORT_FILES=""
REPORT_LINK=""
REPORT_HTML=""
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
done
if [ -n "$REPORT_FILES" ]; then
# Concatenate all reports into one combined report file
COMBINED_MD=""
for rpt in $REPORT_FILES; do
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
RPT_MD=$(sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' "$rpt")
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD}&#10;&#10;---&#10;&#10;"
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
done
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
FIRST_BASENAME=$(basename "$FIRST_REPORT")
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
fi
if [ "$HAS_BEFORE" = "1" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
else
PASS_VIDEOS=""
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
[ -f "$pass_vid" ] || continue
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
done
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
fi
CARD_COUNT=$((CARD_COUNT + 1))
done
# Build commit info and target link for the report header
COMMIT_HTML=""
REPO_URL="https://github.com/${REPO}"
if [ -n "${TARGET_NUM:-}" ]; then
if [ "$TARGET_TYPE" = "issue" ]; then
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
else
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
fi
fi
if [ -n "${BEFORE_SHA:-}" ]; then
SHORT_BEFORE="${BEFORE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
fi
if [ -n "${AFTER_SHA:-}" ]; then
SHORT_AFTER="${AFTER_SHA:0:7}"
AFTER_LABEL="PR"
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
fi
if [ -n "${PIPELINE_SHA:-}" ]; then
SHORT_PIPE="${PIPELINE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
fi
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" &middot; ${COMMIT_HTML}"
RUN_LINK=""
if [ -n "${RUN_URL:-}" ]; then
RUN_LINK=" &middot; <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
fi
# Timing info
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
TIMING_HTML=""
if [ -n "${RUN_START_TIME:-}" ]; then
TIMING_HTML=" &middot; <span class=sha title='Pipeline timing'>${RUN_START_TIME} &rarr; ${DEPLOY_TIME}</span>"
fi
# Generate index.html from template
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
# Write dynamic content to temp files for safe substitution
# Cloudflare Pages _headers file — enable range requests for video seeking
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
/*.mp4
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
HEADERSEOF
# Build purpose description from pr-context.txt
PURPOSE_HTML=""
if [ -f pr-context.txt ]; then
# Extract title line and first paragraph of description
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt 2>/dev/null | sed 's/^Title: //' || true)
if [ "$TARGET_TYPE" = "issue" ]; then
PURPOSE_LABEL="Issue #${TARGET_NUM}"
PURPOSE_VERB="reports"
else
PURPOSE_LABEL="PR #${TARGET_NUM}"
PURPOSE_VERB="aims to"
fi
# Get first ~300 chars of description body (after "Description:" line)
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt 2>/dev/null | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt 2>/dev/null | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
# Build requirements from QA guide JSON
REQS_HTML=""
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1 || true)
if [ -f "$QA_GUIDE" ]; then
PREREQS=$(python3 -c "
import json, sys, html
try:
g = json.load(open(sys.argv[1]))
prereqs = g.get('prerequisites', [])
steps = g.get('steps', [])
focus = g.get('test_focus', '')
parts = []
if focus:
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
if prereqs:
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
if steps:
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
if len(steps) > 6:
parts[-1] += ' → ...'
print('<br>'.join(parts))
except: pass
" "$QA_GUIDE" 2>/dev/null)
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
fi
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
fi
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
# Badge HTML with copy button (placeholder URL filled after deploy)
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
python3 -c "
import sys, pathlib
d = pathlib.Path(sys.argv[1])
t = pathlib.Path(sys.argv[2]).read_text()
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
sys.stdout.write(t)
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
ERROREOF
# Copy research log to deploy dir if it exists
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
if [ -f "$rlog" ]; then
cp "$rlog" "$DEPLOY_DIR/research-log.json"
echo "Found research log: $rlog"
break
fi
done
# Copy generated test code to deploy dir
for tfile in qa-artifacts/*/research/reproduce.spec.ts qa-artifacts/*/*/research/reproduce.spec.ts qa-artifacts/before/*/research/reproduce.spec.ts; do
if [ -f "$tfile" ]; then
cp "$tfile" "$DEPLOY_DIR/reproduce.spec.ts"
echo "Found test code: $tfile"
break
fi
done
# Generate badge SVGs into deploy dir
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
# Try research log first (ground truth from a11y assertions)
RESEARCH_VERDICT=""
REPRO_METHOD=""
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
REPRO_METHOD=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('reproducedBy','none'))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none} (by: ${REPRO_METHOD:-none})"
if [ -n "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=1
case "$RESEARCH_VERDICT" in
REPRODUCED) REPRO_COUNT=1 ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
INCONCLUSIVE) INCONC_COUNT=1 ;;
esac
fi
fi
# Fall back to video review verdicts if no research log
if [ -z "$RESEARCH_VERDICT" ] && [ -d video-reviews ]; then
for rpt in video-reviews/*-qa-video-report.md; do
[ -f "$rpt" ] || continue
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
# Try structured JSON verdict first (from ## Verdict section)
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
RISK_JSON=$(grep -oP '"risk":\s*"[a-z]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[a-z]+$' || true)
if [ -n "$VERDICT_JSON" ]; then
case "$VERDICT_JSON" in
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
esac
else
# Fallback: grep Summary section (for older reports without ## Verdict)
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
INCONC_COUNT=$((INCONC_COUNT + 1))
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
REPRO_COUNT=$((REPRO_COUNT + 1))
fi
fi
done
fi
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
echo "Verdict: ${REPRO_COUNT}${NOT_REPRO_COUNT}${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
# Badge text:
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
if [ "$TOTAL_REPORTS" -le 1 ]; then
# Single report — simple label
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
elif [ "$FAIL_COUNT" -gt 0 ]; then
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
fi
else
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
PARTS=""
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}"
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}"
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}"
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
# Color based on best outcome
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#9f9f9f"
fi
fi
# Badge label: #NUM QA0327 (with today's date)
QA_DATE=$(date -u '+%m%d')
BADGE_LABEL="QA${QA_DATE}"
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
# For PRs, also extract fix quality from Overall Risk section
FIX_RESULT="" FIX_COLOR="#4c1"
if [ "$TARGET_TYPE" != "issue" ]; then
# Try structured JSON risk first
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
if [ -n "$ALL_RISKS" ]; then
# Use worst risk across all reports
if echo "$ALL_RISKS" | grep -q 'high'; then
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
elif echo "$ALL_RISKS" | grep -q 'medium'; then
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
elif echo "$ALL_RISKS" | grep -q 'low'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
else
# Fallback: grep Overall Risk section
RISK_TEXT=""
if [ -d video-reviews ]; then
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20 || true)
fi
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]' || true)
if [ -n "$RISK_FIRST" ]; then
case "$RISK_FIRST" in
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
esac
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
fi
fi
# Always use vertical box badge
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
"$FIX_RESULT" "$FIX_COLOR" "$REPRO_METHOD"
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
# Remove files exceeding Cloudflare Pages 25MB limit to prevent silent deploy failures
MAX_SIZE=$((25 * 1024 * 1024))
find "$DEPLOY_DIR" -type f -size +${MAX_SIZE}c | while read -r big_file; do
SIZE_MB=$(( $(stat -c%s "$big_file") / 1024 / 1024 ))
echo "Removing oversized file: $(basename "$big_file") (${SIZE_MB}MB > 25MB limit)"
rm "$big_file"
done
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
DEPLOY_OUTPUT=$(wrangler pages deploy "$DEPLOY_DIR" \
--project-name="comfy-qa" \
--branch="$BRANCH" 2>&1) || true
echo "$DEPLOY_OUTPUT" | tail -5
URL=$(echo "$DEPLOY_OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1 || true)
FALLBACK_URL="https://${BRANCH}.comfy-qa.pages.dev"
echo "url=${URL:-$FALLBACK_URL}" >> "$GITHUB_OUTPUT"
echo "Deployed to: ${URL:-$FALLBACK_URL}"

View File

@@ -1,208 +0,0 @@
#!/usr/bin/env tsx
/**
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
*
* Usage:
* pnpm exec tsx scripts/qa-generate-test.ts \
* --qa-report <path> QA video review report (markdown)
* --pr-diff <path> PR diff file
* --output <path> Output .spec.ts file path
* --model <name> Gemini model (default: gemini-3-flash-preview)
*/
import { readFile, writeFile } from 'node:fs/promises'
import { basename, resolve } from 'node:path'
import { GoogleGenerativeAI } from '@google/generative-ai'
interface CliOptions {
qaReport: string
prDiff: string
output: string
model: string
}
const DEFAULTS: CliOptions = {
qaReport: '',
prDiff: '',
output: '',
model: 'gemini-3-flash-preview'
}
// ── Fixture API reference for the prompt ────────────────────────────
const FIXTURE_API = `
## ComfyUI Playwright Test Fixture API
Import pattern:
\`\`\`typescript
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
\`\`\`
### Available helpers on \`comfyPage\`:
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
- \`.getActiveTabName(): Promise<string>\` — get active tab name
- \`.saveWorkflow(name)\` — Save via File > Save dialog
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
- \`.exportWorkflow(name)\` — Export via File > Export dialog
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
- \`.getWorkflowTab(name)\` — get a tab locator by name
- \`.closeWorkflowTab(name)\` — close a tab
- \`.openTopbarMenu()\` — open the hamburger menu
- \`.openSubmenu(label)\` — hover to open a submenu
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
- \`.open()\` / \`.close()\` — toggle sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
- \`.getPersistedItem(name)\` — get a workflow item locator
- \`comfyPage.workflow\` — WorkflowHelper:
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\` — delete a workflow
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
- \`comfyPage.settings.setSetting(key, value)\` — change settings
- \`comfyPage.keyboard\` — KeyboardHelper:
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
- \`comfyPage.nodeOps\` — NodeOperationsHelper
- \`comfyPage.canvas\` — CanvasHelper
- \`comfyPage.contextMenu\` — ContextMenu
- \`comfyPage.toast\` — ToastHelper
- \`comfyPage.confirmDialog\` — confirmation dialog
- \`comfyPage.nextFrame()\` — wait for Vue re-render
### Test patterns:
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
`
// ── Prompt builder ──────────────────────────────────────────────────
function buildPrompt(qaReport: string, prDiff: string): string {
return `You are a Playwright test generator for the ComfyUI frontend.
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
described in the QA report below. The test must:
1. Use the ComfyUI Playwright fixture API (documented below)
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
3. NOT test code implementation details
4. Be concise — only test the behavior that the PR changed
5. Follow existing test conventions (see API reference)
${FIXTURE_API}
## QA Video Review Report
${qaReport}
## PR Diff (for context on what changed)
${prDiff.slice(0, 8000)}
## Output Requirements
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
- Start with imports, end with closing brace
- Use descriptive test names that explain the expected behavior
- Add screenshot assertions where visual verification matters
- Keep it focused: 2-5 test cases covering the core behavioral change
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
`
}
// ── Gemini call ─────────────────────────────────────────────────────
async function generateTest(
qaReport: string,
prDiff: string,
model: string
): Promise<string> {
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
const genAI = new GoogleGenerativeAI(apiKey)
const genModel = genAI.getGenerativeModel({ model })
const prompt = buildPrompt(qaReport, prDiff)
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
const result = await genModel.generateContent({
contents: [{ role: 'user', parts: [{ text: prompt }] }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192
}
})
const text = result.response.text()
// Strip markdown fences if model wraps output
return text
.replace(/^```(?:typescript|ts)?\n?/, '')
.replace(/\n?```$/, '')
.trim()
}
// ── CLI ─────────────────────────────────────────────────────────────
function parseArgs(): CliOptions {
const args = process.argv.slice(2)
const opts = { ...DEFAULTS }
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--qa-report':
opts.qaReport = args[++i]
break
case '--pr-diff':
opts.prDiff = args[++i]
break
case '--output':
opts.output = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--help':
console.warn(`Usage:
pnpm exec tsx scripts/qa-generate-test.ts [options]
Options:
--qa-report <path> QA video review report (markdown) [required]
--pr-diff <path> PR diff file [required]
--output <path> Output .spec.ts path [required]
--model <name> Gemini model (default: gemini-3-flash-preview)`)
process.exit(0)
}
}
if (!opts.qaReport || !opts.prDiff || !opts.output) {
console.error('Missing required args. Run with --help for usage.')
process.exit(1)
}
return opts
}
async function main() {
const opts = parseArgs()
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
console.warn(
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
)
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
const testCode = await generateTest(qaReport, prDiff, opts.model)
const outputPath = resolve(opts.output)
await writeFile(outputPath, testCode + '\n')
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})

File diff suppressed because it is too large Load Diff

View File

@@ -1,135 +0,0 @@
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<style>
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
*{margin:0;padding:0;box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
.container{max-width:1200px;margin:0 auto}
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
.header-icon svg{color:var(--primary)}
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
.icon{font-size:1.125rem}
.links{display:flex;gap:.5rem}
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
.comp-panel{border-right:1px solid var(--border-faint)}
.comp-panel:last-child{border-right:none}
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
.report summary:hover{color:var(--fg)}
.report summary svg{flex-shrink:0;opacity:.5}
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
.report-body p{margin:.4rem 0}
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
.report-body li{margin:.25rem 0}
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
.report-body strong{color:var(--fg)}
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
.badge-img{height:20px;display:block}
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
.purpose strong{color:var(--fg);font-weight:600}
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
</style></head><body><div class=container>
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend &middot; Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
</div><script>
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[![QA Badge]('+b+')]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
document.querySelectorAll('.video-wrap video').forEach(v=>{
v.playbackRate=0.5;v.removeAttribute('autoplay');v.pause();
const c=document.createElement('div');c.className='vctrl';
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
btn('\u23EE',()=>{v.currentTime=0});
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
sep();
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===0.5)b.classList.add('active');return b});
sep();c.appendChild(time);
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
// Custom seekbar — works even without server range request support
const seekWrap=document.createElement('div');seekWrap.className='vseek';
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
let seeking=false;
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
seekBar.onchange=()=>{seeking=false};
v.closest('.video-wrap').after(seekWrap);
seekWrap.after(c);
v.ontimeupdate=()=>{
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
};
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
v.parentElement.addEventListener('keydown',e=>{
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
});
v.parentElement.setAttribute('tabindex','0');
});
</script></body></html>

View File

@@ -1,253 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Reproduce Phase — Deterministic replay of research plan with narration
*
* Takes a reproduction plan from the research phase and replays it:
* 1. Execute each action deterministically (no AI decisions)
* 2. Capture a11y snapshot before/after each action
* 3. Gemini describes what visually changed (narration for humans)
* 4. Output: narration-log.json with full evidence chain
*/
import type { Page } from '@playwright/test'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { mkdirSync, writeFileSync } from 'fs'
import type { ActionResult } from './qa-record.js'
// ── Types ──
interface ReproductionStep {
action: Record<string, unknown> & { action: string }
expectedAssertion: string
}
interface NarrationEntry {
step: number
action: string
params: Record<string, unknown>
result: ActionResult
a11yBefore: unknown
a11yAfter: unknown
assertionExpected: string
assertionPassed: boolean
assertionActual: string
geminiNarration: string
timestampMs: number
}
export interface NarrationLog {
entries: NarrationEntry[]
allAssertionsPassed: boolean
}
interface ReproduceOptions {
page: Page
plan: ReproductionStep[]
geminiApiKey: string
outputDir: string
}
// ── A11y helpers ──
interface A11yNode {
role: string
name: string
value?: string
checked?: boolean
disabled?: boolean
expanded?: boolean
children?: A11yNode[]
}
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
if (!node) return null
const sel = selector.toLowerCase()
if (
node.name?.toLowerCase().includes(sel) ||
node.role?.toLowerCase().includes(sel)
) {
return node
}
if (node.children) {
for (const child of node.children) {
const found = searchA11y(child, selector)
if (found) return found
}
}
return null
}
function summarizeA11y(node: A11yNode | null): string {
if (!node) return 'null'
const parts = [`role=${node.role}`, `name="${node.name}"`]
if (node.value !== undefined) parts.push(`value="${node.value}"`)
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
if (node.disabled) parts.push('disabled')
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
return `{${parts.join(', ')}}`
}
// ── Subtitle overlay ──
async function showSubtitle(page: Page, text: string, step: number) {
const encoded = encodeURIComponent(
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
)
await page.addScriptTag({
content: `(function(){
var id='qa-subtitle';
var el=document.getElementById(id);
if(!el){
el=document.createElement('div');
el.id=id;
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
document.body.appendChild(el);
}
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
})()`
})
}
// ── Gemini visual narration ──
async function geminiDescribe(
page: Page,
geminiApiKey: string,
focus: string
): Promise<string> {
try {
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
const genAI = new GoogleGenerativeAI(geminiApiKey)
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
const result = await model.generateContent([
{
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
},
{
inlineData: {
mimeType: 'image/jpeg',
data: screenshot.toString('base64')
}
}
])
return result.response.text().trim()
} catch (e) {
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
}
}
// ── Main reproduce function ──
export async function runReproducePhase(
opts: ReproduceOptions
): Promise<NarrationLog> {
const { page, plan, geminiApiKey, outputDir } = opts
const { executeAction } = await import('./qa-record.js')
const narrationDir = `${outputDir}/narration`
mkdirSync(narrationDir, { recursive: true })
const entries: NarrationEntry[] = []
const startMs = Date.now()
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
for (let i = 0; i < plan.length; i++) {
const step = plan[i]
const actionObj = step.action
const elapsed = Date.now() - startMs
// Show subtitle
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
// Capture a11y BEFORE
const a11yBefore = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Execute action
const result = await executeAction(
page,
actionObj as Parameters<typeof executeAction>[1],
outputDir
)
await new Promise((r) => setTimeout(r, 500))
// Capture a11y AFTER
const a11yAfter = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Check assertion
let assertionPassed = false
let assertionActual = ''
if (step.expectedAssertion) {
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
const selectorName = parts[0]
const expectedState = parts.slice(1).join(':').trim()
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
if (expectedState === 'visible' || expectedState === 'exists') {
assertionPassed = found !== null
} else if (expectedState === 'hidden' || expectedState === 'gone') {
assertionPassed = found === null
} else {
// Generic: check if the actual state contains the expected text
assertionPassed = assertionActual
.toLowerCase()
.includes(expectedState.toLowerCase())
}
console.warn(
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
)
}
// Gemini narration (visual description for humans)
const geminiNarration = await geminiDescribe(
page,
geminiApiKey,
`What changed after ${actionObj.action}?`
)
entries.push({
step: i + 1,
action: actionObj.action,
params: actionObj,
result,
a11yBefore,
a11yAfter,
assertionExpected: step.expectedAssertion,
assertionPassed,
assertionActual,
geminiNarration,
timestampMs: elapsed
})
}
// Final screenshot
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
const log: NarrationLog = {
entries,
allAssertionsPassed: entries.every((e) => e.assertionPassed)
}
writeFileSync(
`${narrationDir}/narration-log.json`,
JSON.stringify(log, null, 2)
)
console.warn(
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
)
return log
}

View File

@@ -1,150 +0,0 @@
import { describe, expect, it } from 'vitest'
import {
extractPlatformFromArtifactDirName,
pickLatestVideosByPlatform,
selectVideoCandidateByFile
} from './qa-video-review'
describe('extractPlatformFromArtifactDirName', () => {
it('extracts and normalizes known qa artifact directory names', () => {
expect(
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
).toBe('windows')
expect(
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
).toBe('macos')
expect(
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
).toBe('linux')
})
it('falls back to slugifying unknown directory names', () => {
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
'custom-platform-run'
)
})
})
describe('pickLatestVideosByPlatform', () => {
it('keeps only the latest candidate per platform', () => {
const selected = pickLatestVideosByPlatform([
{
platformName: 'windows',
videoPath: '/tmp/windows-old.mp4',
mtimeMs: 100
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
},
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
}
])
expect(selected).toEqual([
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
}
])
})
})
describe('selectVideoCandidateByFile', () => {
it('selects a single candidate by artifacts-relative path', () => {
const selected = selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-Linux-1/qa-session.mp4'
}
)
expect(selected).toEqual({
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
})
})
it('throws when basename matches multiple videos', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-session.mp4'
}
)
).toThrow('matched 2 videos')
})
it('throws when there is no matching video', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-macOS-1/qa-session.mp4'
}
)
).toThrow('No video matched')
})
it('throws when video file is missing', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: ' '
}
)
).toThrow('--video-file is required')
})
})

View File

@@ -1,765 +0,0 @@
#!/usr/bin/env tsx
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
import { basename, dirname, extname, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { globSync } from 'glob'
interface CliOptions {
artifactsDir: string
videoFile: string
beforeVideo: string
outputDir: string
model: string
requestTimeoutMs: number
dryRun: boolean
prContext: string
targetUrl: string
passLabel: string
}
interface VideoCandidate {
platformName: string
videoPath: string
mtimeMs: number
}
const DEFAULT_OPTIONS: CliOptions = {
artifactsDir: './tmp/qa-artifacts',
videoFile: '',
beforeVideo: '',
outputDir: './tmp',
model: 'gemini-3-flash-preview',
requestTimeoutMs: 300_000,
dryRun: false,
prContext: '',
targetUrl: '',
passLabel: ''
}
const USAGE = `Usage:
pnpm exec tsx scripts/qa-video-review.ts [options]
Options:
--artifacts-dir <path> Artifacts root directory
(default: ./tmp/qa-artifacts)
--video-file <name-or-path> Video file to analyze (required)
(supports basename or relative/absolute path)
--before-video <path> Before video (main branch) for comparison
When provided, sends both videos to Gemini
for comparative before/after analysis
--output-dir <path> Output directory for markdown reports
(default: ./tmp)
--model <name> Gemini model
(default: gemini-3-flash-preview)
--request-timeout-ms <n> Request timeout in milliseconds
(default: 300000)
--pr-context <file> File with PR context (title, body, diff)
for PR-aware review
--target-url <url> Issue or PR URL to include in the report
--pass-label <label> Label for multi-pass reports (e.g. pass1)
Output becomes {platform}-{label}-qa-video-report.md
--dry-run Discover videos and output targets only
--help Show this help text
Environment:
GEMINI_API_KEY Required unless --dry-run
`
function parsePositiveInteger(rawValue: string, flagName: string): number {
const parsedValue = Number.parseInt(rawValue, 10)
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
}
return parsedValue
}
function parseCliOptions(args: string[]): CliOptions {
const options: CliOptions = { ...DEFAULT_OPTIONS }
for (let index = 0; index < args.length; index += 1) {
const argument = args[index]
const nextValue = args[index + 1]
const requireValue = (flagName: string): string => {
if (!nextValue || nextValue.startsWith('--')) {
throw new Error(`Missing value for ${flagName}`)
}
index += 1
return nextValue
}
if (argument === '--help') {
process.stdout.write(USAGE)
process.exit(0)
}
if (argument === '--artifacts-dir') {
options.artifactsDir = requireValue(argument)
continue
}
if (argument === '--video-file') {
options.videoFile = requireValue(argument)
continue
}
if (argument === '--output-dir') {
options.outputDir = requireValue(argument)
continue
}
if (argument === '--model') {
options.model = requireValue(argument)
continue
}
if (argument === '--request-timeout-ms') {
options.requestTimeoutMs = parsePositiveInteger(
requireValue(argument),
argument
)
continue
}
if (argument === '--before-video') {
options.beforeVideo = requireValue(argument)
continue
}
if (argument === '--pr-context') {
options.prContext = requireValue(argument)
continue
}
if (argument === '--target-url') {
options.targetUrl = requireValue(argument)
continue
}
if (argument === '--pass-label') {
options.passLabel = requireValue(argument)
continue
}
if (argument === '--dry-run') {
options.dryRun = true
continue
}
throw new Error(`Unknown argument: ${argument}`)
}
return options
}
function normalizePlatformName(value: string): string {
const slug = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
return slug.length > 0 ? slug : 'unknown-platform'
}
export function extractPlatformFromArtifactDirName(dirName: string): string {
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
return normalizePlatformName(matchedValue ?? dirName)
}
function extractPlatformFromVideoPath(videoPath: string): string {
const artifactDirName = basename(dirname(videoPath))
return extractPlatformFromArtifactDirName(artifactDirName)
}
export function pickLatestVideosByPlatform(
candidates: VideoCandidate[]
): VideoCandidate[] {
const latestByPlatform = new Map<string, VideoCandidate>()
for (const candidate of candidates) {
const current = latestByPlatform.get(candidate.platformName)
if (!current || candidate.mtimeMs > current.mtimeMs) {
latestByPlatform.set(candidate.platformName, candidate)
}
}
return [...latestByPlatform.values()].sort((a, b) =>
a.platformName.localeCompare(b.platformName)
)
}
function toProjectRelativePath(targetPath: string): string {
const relativePath = relative(process.cwd(), targetPath)
if (relativePath.startsWith('.')) {
return relativePath
}
return `./${relativePath}`
}
function errorToString(error: unknown): string {
return error instanceof Error ? error.message : String(error)
}
function normalizePathForMatch(value: string): string {
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
}
export function selectVideoCandidateByFile(
candidates: VideoCandidate[],
options: { artifactsDir: string; videoFile: string }
): VideoCandidate {
const requestedValue = options.videoFile.trim()
if (requestedValue.length === 0) {
throw new Error('--video-file is required')
}
const artifactsRoot = resolve(options.artifactsDir)
const requestedAbsolutePath = resolve(requestedValue)
const requestedPathKey = normalizePathForMatch(requestedValue)
const matches = candidates.filter((candidate) => {
const candidateAbsolutePath = resolve(candidate.videoPath)
if (candidateAbsolutePath === requestedAbsolutePath) {
return true
}
const candidateBaseName = basename(candidate.videoPath)
if (candidateBaseName === requestedValue) {
return true
}
const relativeToCwd = normalizePathForMatch(
relative(process.cwd(), candidateAbsolutePath)
)
if (relativeToCwd === requestedPathKey) {
return true
}
const relativeToArtifacts = normalizePathForMatch(
relative(artifactsRoot, candidateAbsolutePath)
)
return relativeToArtifacts === requestedPathKey
})
if (matches.length === 1) {
return matches[0]
}
if (matches.length === 0) {
const availableVideos = candidates.map((candidate) =>
toProjectRelativePath(candidate.videoPath)
)
throw new Error(
[
`No video matched --video-file "${options.videoFile}".`,
'Available videos:',
...availableVideos.map((videoPath) => `- ${videoPath}`)
].join('\n')
)
}
throw new Error(
[
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
'Please pass a more specific path.',
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
].join('\n')
)
}
async function collectVideoCandidates(
artifactsDir: string
): Promise<VideoCandidate[]> {
const absoluteArtifactsDir = resolve(artifactsDir)
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
cwd: absoluteArtifactsDir,
absolute: true,
nodir: true
}).sort()
const candidates = await Promise.all(
videoPaths.map(async (videoPath) => {
const videoStat = await stat(videoPath)
return {
platformName: extractPlatformFromVideoPath(videoPath),
videoPath,
mtimeMs: videoStat.mtimeMs
}
})
)
return candidates
}
function getMimeType(filePath: string): string {
const ext = extname(filePath).toLowerCase()
const mimeMap: Record<string, string> = {
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime',
'.avi': 'video/x-msvideo',
'.mkv': 'video/x-matroska',
'.m4v': 'video/mp4'
}
return mimeMap[ext] || 'video/mp4'
}
function buildReviewPrompt(options: {
platformName: string
videoPath: string
prContext: string
isComparative: boolean
}): string {
const { platformName, videoPath, prContext, isComparative } = options
if (isComparative) {
return buildComparativePrompt(platformName, videoPath, prContext)
}
return buildSingleVideoPrompt(platformName, videoPath, prContext)
}
function buildComparativePrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
'',
'You are given TWO videos:',
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
'',
'Both videos show the same test steps executed on different code versions.',
''
]
if (prContext) {
lines.push('## PR Context', prContext, '')
}
lines.push(
'## Your Task',
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
'',
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
' Describe what you observe — this establishes the baseline.',
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
'3. **Comparison**: What specifically changed between before and after?',
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
' that were NOT present in the BEFORE video?',
'',
'Note: Brief black frames during page transitions are NORMAL.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible differences. Avoid speculation.',
'',
'Return markdown with these sections exactly:',
'## Summary',
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
'',
'## Behavior Changes',
'Summarize ALL behavioral differences as a markdown TABLE:',
'| Behavior | Before (main) | After (PR) | Verdict |',
'',
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
'- **Before (main)**: how it works/looks in the BEFORE video',
'- **After (PR)**: how it works/looks in the AFTER video',
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
'',
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
'that were tested, so reviewers can confirm nothing was missed.',
'',
'## Timeline Comparison',
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
'| Time | Type | Severity | Before (main) | After (PR) |',
'',
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
'- **Before (main)**: what is observed in the BEFORE video at that time',
'- **After (PR)**: what is observed in the AFTER video at that time',
'',
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
'omit that row. Focus on meaningful differences, not narrating every frame.',
'',
'## Confirmed Issues',
'For each issue, use this exact format:',
'',
'### [Short issue title]',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'',
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
'',
'**Evidence:** [What you observed at the given timestamp in which video]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'(Assess whether the PR achieves its goal based on the before/after comparison)',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
)
return lines.filter(Boolean).join('\n')
}
function buildSingleVideoPrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer reviewing a UI test session recording.',
'',
'## ANTI-HALLUCINATION RULES (READ FIRST)',
'- Describe ONLY what you can directly observe in the video frames',
'- NEVER infer or assume what "must have happened" between frames',
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
'- Your job is to be a CAMERA — report facts, not interpretations',
''
]
const isIssueContext =
prContext &&
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
prContext
)
if (prContext) {
lines.push(
'## Phase 1: Blind Observation (describe what you SEE)',
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
'- What elements does the user click/hover/type?',
'- What dialogs/menus open and close?',
'- What keyboard indicators appear? (look for subtitle overlays)',
'- What is the BEFORE state and AFTER state of each action?',
'',
'## Phase 2: Compare against expected behavior',
'Now compare your observations against the context below.',
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
''
)
if (isIssueContext) {
lines.push(
'## Issue Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video perform the reproduction steps described in the issue?',
'2. Did your Phase 1 observations show the reported bug behavior?',
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
''
)
} else {
lines.push(
'## PR Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video test the specific behavior the PR changes?',
'2. Did your Phase 1 observations show the expected before/after difference?',
'3. If the test was incomplete or inconclusive, say so honestly.',
''
)
}
}
lines.push(
`Review this QA session video for platform "${platformName}".`,
`Source video: ${toProjectRelativePath(videoPath)}.`,
'The video shows the full test session — analyze it chronologically.',
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible problems and avoid speculation.',
'If confidence is low, mark it explicitly.',
'',
'Return markdown with these sections exactly:',
'## Summary',
isIssueContext
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
: prContext
? '(Explain what the PR intended and whether the video confirms it works)'
: '',
'## Confirmed Issues',
'For each confirmed issue, use this exact format (one block per issue):',
'',
'### [Short issue title]',
'`HIGH` `01:03` `Confidence: High`',
'',
'[Description of the issue — what went wrong and what was expected]',
'',
'**Evidence:** [What you observed in the video at the given timestamp]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'Do NOT use a table for issues — use the block format above.',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the bug/behavior is clearly visible in the video',
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient'
)
return lines.filter(Boolean).join('\n')
}
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
async function readVideoFile(videoPath: string): Promise<Buffer> {
const fileStat = await stat(videoPath)
if (fileStat.size > MAX_VIDEO_BYTES) {
throw new Error(
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
)
}
return readFile(videoPath)
}
async function requestGeminiReview(options: {
apiKey: string
model: string
platformName: string
videoPath: string
beforeVideoPath: string
timeoutMs: number
prContext: string
}): Promise<string> {
const genAI = new GoogleGenerativeAI(options.apiKey)
const model = genAI.getGenerativeModel({ model: options.model })
const isComparative = options.beforeVideoPath.length > 0
const prompt = buildReviewPrompt({
platformName: options.platformName,
videoPath: options.videoPath,
prContext: options.prContext,
isComparative
})
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
if (isComparative) {
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
parts.push(
{ text: 'Video 1 — BEFORE (main branch):' },
{
inlineData: {
mimeType: getMimeType(options.beforeVideoPath),
data: beforeBuffer.toString('base64')
}
}
)
}
const afterBuffer = await readVideoFile(options.videoPath)
if (isComparative) {
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
}
parts.push({
inlineData: {
mimeType: getMimeType(options.videoPath),
data: afterBuffer.toString('base64')
}
})
const result = await model.generateContent(parts, {
timeout: options.timeoutMs
})
const response = result.response
const text = response.text()
if (!text || text.trim().length === 0) {
throw new Error('Gemini API returned no output text')
}
return text.trim()
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
function buildReportMarkdown(input: {
platformName: string
model: string
videoPath: string
videoSizeBytes: number
beforeVideoPath?: string
beforeVideoSizeBytes?: number
reviewText: string
targetUrl?: string
}): string {
const headerLines = [
`# ${input.platformName} QA Video Report`,
'',
`- Generated at: ${new Date().toISOString()}`,
`- Model: \`${input.model}\``
]
if (input.targetUrl) {
headerLines.push(`- Target: ${input.targetUrl}`)
}
if (input.beforeVideoPath) {
headerLines.push(
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
'- Mode: **Comparative (before/after)**'
)
} else {
headerLines.push(
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
`- Video size: ${formatBytes(input.videoSizeBytes)}`
)
}
headerLines.push('', '## AI Review', '')
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
}
async function reviewVideo(
video: VideoCandidate,
options: CliOptions,
apiKey: string
): Promise<void> {
let prContext = ''
if (options.prContext) {
try {
prContext = await readFile(options.prContext, 'utf-8')
process.stdout.write(
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
)
} catch {
process.stdout.write(
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
)
}
}
const beforeVideoPath = options.beforeVideo
? resolve(options.beforeVideo)
: ''
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
process.stdout.write(
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
)
}
process.stdout.write(
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
)
const reviewText = await requestGeminiReview({
apiKey,
model: options.model,
platformName: video.platformName,
videoPath: video.videoPath,
beforeVideoPath,
timeoutMs: options.requestTimeoutMs,
prContext
})
const videoStat = await stat(video.videoPath)
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
const outputPath = resolve(
options.outputDir,
`${video.platformName}${passSegment}-qa-video-report.md`
)
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
platformName: video.platformName,
model: options.model,
videoPath: video.videoPath,
videoSizeBytes: videoStat.size,
reviewText,
targetUrl: options.targetUrl || undefined
}
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
reportInput.beforeVideoPath = beforeVideoPath
reportInput.beforeVideoSizeBytes = beforeStat.size
}
const reportMarkdown = buildReportMarkdown(reportInput)
await mkdir(dirname(outputPath), { recursive: true })
await writeFile(outputPath, reportMarkdown, 'utf-8')
process.stdout.write(
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
async function main(): Promise<void> {
const options = parseCliOptions(process.argv.slice(2))
const candidates = await collectVideoCandidates(options.artifactsDir)
if (candidates.length === 0) {
process.stdout.write(
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
)
return
}
const selectedVideo = selectVideoCandidateByFile(candidates, {
artifactsDir: options.artifactsDir,
videoFile: options.videoFile
})
process.stdout.write(
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
)
if (options.dryRun) {
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
return
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
}
await reviewVideo(selectedVideo, options, apiKey)
}
if (isExecutedAsScript(import.meta.url)) {
void main().catch((error: unknown) => {
const message = errorToString(error)
process.stderr.write(`qa-video-review failed: ${message}\n`)
process.exit(1)
})
}

View File

@@ -1,4 +1,3 @@
import { fromAny, fromPartial } from '@total-typescript/shoehorn'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import {
@@ -44,12 +43,12 @@ describe('downloadUtil', () => {
createObjectURLSpy.mockClear().mockReturnValue('blob:mock-url')
revokeObjectURLSpy.mockClear().mockImplementation(() => {})
// Create a mock anchor element
mockLink = fromPartial<HTMLAnchorElement>({
mockLink = {
href: '',
download: '',
click: vi.fn(),
style: { display: '' }
})
} as unknown as HTMLAnchorElement
// Spy on DOM methods
vi.spyOn(document, 'createElement').mockReturnValue(mockLink)
@@ -173,14 +172,12 @@ describe('downloadUtil', () => {
const headersMock = {
get: vi.fn().mockReturnValue(null)
}
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
})
)
fetchMock.mockResolvedValue({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
} as unknown as Response)
downloadFile(testUrl)
@@ -201,13 +198,11 @@ describe('downloadUtil', () => {
mockIsCloud.value = true
const testUrl = 'https://storage.googleapis.com/bucket/missing.bin'
const consoleSpy = vi.spyOn(console, 'error').mockImplementation(() => {})
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: false,
status: 404,
blob: vi.fn()
})
)
fetchMock.mockResolvedValue({
ok: false,
status: 404,
blob: vi.fn()
} as Partial<Response> as Response)
downloadFile(testUrl)
@@ -229,14 +224,12 @@ describe('downloadUtil', () => {
const headersMock = {
get: vi.fn().mockReturnValue('attachment; filename="user-friendly.png"')
}
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
})
)
fetchMock.mockResolvedValue({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
} as unknown as Response)
downloadFile(testUrl)
@@ -263,14 +256,12 @@ describe('downloadUtil', () => {
'attachment; filename="fallback.png"; filename*=UTF-8\'\'%E4%B8%AD%E6%96%87.png'
)
}
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
})
)
fetchMock.mockResolvedValue({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
} as unknown as Response)
downloadFile(testUrl)
@@ -291,14 +282,12 @@ describe('downloadUtil', () => {
const headersMock = {
get: vi.fn().mockReturnValue(null)
}
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
})
)
fetchMock.mockResolvedValue({
ok: true,
status: 200,
blob: blobFn,
headers: headersMock
} as unknown as Response)
downloadFile(testUrl, 'my-fallback.png')
@@ -339,13 +328,11 @@ describe('downloadUtil', () => {
const testUrl = 'https://storage.googleapis.com/bucket/image.png'
const blob = new Blob(['test'], { type: 'image/png' })
const mockTab = { location: { href: '' }, closed: false, close: vi.fn() }
windowOpenSpy.mockReturnValue(fromAny<Window, unknown>(mockTab))
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
})
)
windowOpenSpy.mockReturnValue(mockTab as unknown as Window)
fetchMock.mockResolvedValue({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
} as unknown as Response)
await openFileInNewTab(testUrl)
@@ -359,13 +346,11 @@ describe('downloadUtil', () => {
mockIsCloud.value = true
const blob = new Blob(['test'], { type: 'image/png' })
const mockTab = { location: { href: '' }, closed: false, close: vi.fn() }
windowOpenSpy.mockReturnValue(fromAny<Window, unknown>(mockTab))
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
})
)
windowOpenSpy.mockReturnValue(mockTab as unknown as Window)
fetchMock.mockResolvedValue({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
} as unknown as Response)
await openFileInNewTab('https://example.com/image.png')
@@ -379,10 +364,11 @@ describe('downloadUtil', () => {
const testUrl = 'https://storage.googleapis.com/bucket/missing.png'
const consoleSpy = vi.spyOn(console, 'error').mockImplementation(() => {})
const mockTab = { location: { href: '' }, closed: false, close: vi.fn() }
windowOpenSpy.mockReturnValue(fromAny<Window, unknown>(mockTab))
fetchMock.mockResolvedValue(
fromPartial<Response>({ ok: false, status: 404 })
)
windowOpenSpy.mockReturnValue(mockTab as unknown as Window)
fetchMock.mockResolvedValue({
ok: false,
status: 404
} as unknown as Response)
await openFileInNewTab(testUrl)
@@ -395,13 +381,11 @@ describe('downloadUtil', () => {
mockIsCloud.value = true
const blob = new Blob(['test'], { type: 'image/png' })
const mockTab = { location: { href: '' }, closed: true, close: vi.fn() }
windowOpenSpy.mockReturnValue(fromAny<Window, unknown>(mockTab))
fetchMock.mockResolvedValue(
fromPartial<Response>({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
})
)
windowOpenSpy.mockReturnValue(mockTab as unknown as Window)
fetchMock.mockResolvedValue({
ok: true,
blob: vi.fn().mockResolvedValue(blob)
} as unknown as Response)
await openFileInNewTab('https://example.com/image.png')

View File

@@ -42,6 +42,7 @@ import type { StyleValue } from 'vue'
import { useIntersectionObserver } from '@/composables/useIntersectionObserver'
import { useMediaCache } from '@/services/mediaCacheService'
import type { ClassValue } from '@/utils/tailwindUtil'
const {
src,
@@ -53,8 +54,8 @@ const {
} = defineProps<{
src: string
alt?: string
containerClass?: string
imageClass?: string
containerClass?: ClassValue
imageClass?: ClassValue
imageStyle?: StyleValue
rootMargin?: string
}>()

View File

@@ -192,15 +192,3 @@ export function curvesToLUT(
return lut
}
export function curveDataToFloatLUT(
curve: CurveData,
size: number = 256
): Float32Array {
const lut = new Float32Array(size)
const interpolate = createInterpolator(curve.points, curve.interpolation)
for (let i = 0; i < size; i++) {
lut[i] = interpolate(i / (size - 1))
}
return lut
}

View File

@@ -1,5 +1,3 @@
import { createTestingPinia } from '@pinia/testing'
import { fromPartial } from '@total-typescript/shoehorn'
import { mount } from '@vue/test-utils'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -11,6 +9,7 @@ import type { LGraphCanvas } from '@/lib/litegraph/src/LGraphCanvas'
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import type { BaseDOMWidget } from '@/scripts/domWidget'
import { useDomWidgetStore } from '@/stores/domWidgetStore'
import { createTestingPinia } from '@pinia/testing'
type TestWidget = BaseDOMWidget<object | string>
@@ -29,7 +28,7 @@ function createNode(
}
function createWidget(id: string, node: LGraphNode, y = 12): TestWidget {
return fromPartial<TestWidget>({
return {
id,
node,
name: 'test_widget',
@@ -41,16 +40,16 @@ function createWidget(id: string, node: LGraphNode, y = 12): TestWidget {
computedHeight: 40,
margin: 10,
isVisible: () => true
})
} as unknown as TestWidget
}
function createCanvas(graph: LGraph): LGraphCanvas {
return fromPartial<LGraphCanvas>({
return {
graph,
low_quality: false,
read_only: false,
isNodeVisible: vi.fn(() => true)
})
} as unknown as LGraphCanvas
}
function drawFrame(canvas: LGraphCanvas) {

View File

@@ -1,14 +1,14 @@
import { createTestingPinia } from '@pinia/testing'
import { fromPartial } from '@total-typescript/shoehorn'
import { mount } from '@vue/test-utils'
import { createTestingPinia } from '@pinia/testing'
import { setActivePinia } from 'pinia'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { reactive } from 'vue'
import { createMockLGraphNode } from '@/utils/__tests__/litegraphTestUtils'
import type { BaseDOMWidget } from '@/scripts/domWidget'
import type { DomWidgetState } from '@/stores/domWidgetStore'
import { useDomWidgetStore } from '@/stores/domWidgetStore'
import { createMockLGraphNode } from '@/utils/__tests__/litegraphTestUtils'
import DomWidget from './DomWidget.vue'
const mockUpdatePosition = vi.fn()
@@ -63,7 +63,7 @@ function createWidgetState(overrideDisabled: boolean): DomWidgetState {
}
})
const widget = fromPartial<BaseDOMWidget<object | string>>({
const widget = {
id: 'dom-widget-id',
name: 'test_widget',
type: 'custom',
@@ -71,7 +71,7 @@ function createWidgetState(overrideDisabled: boolean): DomWidgetState {
options: {},
node,
computedDisabled: false
})
} as unknown as BaseDOMWidget<object | string>
domWidgetStore.registerWidget(widget)
domWidgetStore.setPositionOverride(widget.id, {

View File

@@ -1,7 +1,7 @@
import { fromAny } from '@total-typescript/shoehorn'
import { describe, expect, it } from 'vitest'
import { LGraph, LGraphNode } from '@/lib/litegraph/src/litegraph'
import { getDomWidgetZIndex } from './domWidgetZIndex'
describe('getDomWidgetZIndex', () => {
@@ -15,7 +15,7 @@ describe('getDomWidgetZIndex', () => {
first.order = 0
second.order = 1
const nodes = fromAny<{ _nodes: LGraphNode[] }, unknown>(graph)._nodes
const nodes = (graph as unknown as { _nodes: LGraphNode[] })._nodes
nodes.splice(nodes.indexOf(first), 1)
nodes.push(first)

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { createPinia, setActivePinia } from 'pinia'
import { nextTick, ref } from 'vue'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -160,7 +159,7 @@ describe('swapNodeGroups computed', () => {
it('excludes string nodeType entries', async () => {
const swap = getSwapNodeGroups([
fromAny<MissingNodeType, unknown>('StringGroupNode'),
'StringGroupNode' as unknown as MissingNodeType,
makeMissingNodeType('OldNode', {
nodeId: '1',
isReplaceable: true,

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { createPinia, setActivePinia } from 'pinia'
import { nextTick, ref } from 'vue'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -216,7 +215,7 @@ describe('useErrorGroups', () => {
const { groups } = createErrorGroups()
const missingNodesStore = useMissingNodesErrorStore()
missingNodesStore.setMissingNodeTypes([
fromAny<MissingNodeType, unknown>('StringGroupNode')
'StringGroupNode' as unknown as MissingNodeType
])
await nextTick()

View File

@@ -1,5 +1,4 @@
import { createTestingPinia } from '@pinia/testing'
import { fromAny } from '@total-typescript/shoehorn'
import { mount } from '@vue/test-utils'
import { setActivePinia } from 'pinia'
import type { Slots } from 'vue'
@@ -11,6 +10,7 @@ import type { LGraphNode } from '@/lib/litegraph/src/litegraph'
import type { SubgraphNode } from '@/lib/litegraph/src/subgraph/SubgraphNode'
import type { IBaseWidget } from '@/lib/litegraph/src/types/widgets'
import { usePromotionStore } from '@/stores/promotionStore'
import WidgetActions from './WidgetActions.vue'
const { mockGetInputSpecForWidget } = vi.hoisted(() => ({
@@ -93,13 +93,13 @@ describe('WidgetActions', () => {
}
function createMockNode(): LGraphNode {
return fromAny<LGraphNode, unknown>({
return {
id: 1,
type: 'TestNode',
rootGraph: { id: 'graph-test' },
computeSize: vi.fn(),
size: [200, 100]
})
} as unknown as LGraphNode
}
function mountWidgetActions(widget: IBaseWidget, node: LGraphNode) {
@@ -216,17 +216,17 @@ describe('WidgetActions', () => {
mockGetInputSpecForWidget.mockReturnValue({
type: 'CUSTOM'
})
const parentSubgraphNode = fromAny<SubgraphNode, unknown>({
const parentSubgraphNode = {
id: 4,
rootGraph: { id: 'graph-test' },
computeSize: vi.fn(),
size: [300, 150]
})
const node = fromAny<LGraphNode, unknown>({
} as unknown as SubgraphNode
const node = {
id: 4,
type: 'SubgraphNode',
rootGraph: { id: 'graph-test' }
})
} as unknown as LGraphNode
const widget = {
name: 'text',
type: 'text',

View File

@@ -1,5 +1,4 @@
import { createTestingPinia } from '@pinia/testing'
import { fromAny } from '@total-typescript/shoehorn'
import { mount } from '@vue/test-utils'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -73,13 +72,13 @@ const i18n = createI18n({
})
function createMockNode(overrides: Partial<LGraphNode> = {}): LGraphNode {
return fromAny<LGraphNode, unknown>({
return {
id: 1,
type: 'TestNode',
isSubgraphNode: () => false,
graph: { rootGraph: { id: 'test-graph-id' } },
...overrides
})
} as unknown as LGraphNode
}
function createMockWidget(overrides: Partial<IBaseWidget> = {}): IBaseWidget {
@@ -129,7 +128,7 @@ function createMockPromotedWidgetView(
return 0
}
}
return fromAny<IBaseWidget, unknown>(new MockPromotedWidgetView())
return new MockPromotedWidgetView() as unknown as IBaseWidget
}
function mountWidgetItem(

View File

@@ -143,16 +143,11 @@
<Button
v-if="shouldShowDeleteButton"
size="icon"
data-testid="assets-delete-selected"
@click="handleDeleteSelected"
>
<i class="icon-[lucide--trash-2] size-4" />
</Button>
<Button
size="icon"
data-testid="assets-download-selected"
@click="handleDownloadSelected"
>
<Button size="icon" @click="handleDownloadSelected">
<i class="icon-[lucide--download] size-4" />
</Button>
</template>
@@ -161,17 +156,12 @@
<Button
v-if="shouldShowDeleteButton"
variant="secondary"
data-testid="assets-delete-selected"
@click="handleDeleteSelected"
>
<span>{{ $t('mediaAsset.selection.deleteSelected') }}</span>
<i class="icon-[lucide--trash-2] size-4" />
</Button>
<Button
variant="secondary"
data-testid="assets-download-selected"
@click="handleDownloadSelected"
>
<Button variant="secondary" @click="handleDownloadSelected">
<span>{{ $t('mediaAsset.selection.downloadSelected') }}</span>
<i class="icon-[lucide--download] size-4" />
</Button>

View File

@@ -3,14 +3,12 @@
<LazyImage
:src="src"
:alt="alt"
:image-class="
cn(
'transform-gpu transition-transform duration-300 ease-out',
isVideoType
? 'size-full object-cover'
: 'max-h-64 max-w-full object-contain'
)
"
:image-class="[
'transform-gpu transition-transform duration-300 ease-out',
isVideoType
? 'w-full h-full object-cover'
: 'max-w-full max-h-64 object-contain'
]"
:image-style="
isHovered ? { transform: `scale(${1 + hoverZoom / 100})` } : undefined
"
@@ -21,7 +19,6 @@
<script setup lang="ts">
import LazyImage from '@/components/common/LazyImage.vue'
import BaseThumbnail from '@/components/templates/thumbnails/BaseThumbnail.vue'
import { cn } from '@/utils/tailwindUtil'
const { src, isVideo } = defineProps<{
src: string

View File

@@ -1,4 +1,3 @@
import { fromPartial } from '@total-typescript/shoehorn'
import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest'
import { useDomClipping } from './useDomClipping'
@@ -9,7 +8,7 @@ function createMockElement(rect: {
width: number
height: number
}): HTMLElement {
return fromPartial<HTMLElement>({
return {
getBoundingClientRect: vi.fn(
() =>
({
@@ -21,7 +20,7 @@ function createMockElement(rect: {
toJSON: () => ({})
}) as DOMRect
)
})
} as unknown as HTMLElement
}
function createMockCanvas(rect: {
@@ -30,7 +29,7 @@ function createMockCanvas(rect: {
width: number
height: number
}): HTMLCanvasElement {
return fromPartial<HTMLCanvasElement>({
return {
getBoundingClientRect: vi.fn(
() =>
({
@@ -42,7 +41,7 @@ function createMockCanvas(rect: {
toJSON: () => ({})
}) as DOMRect
)
})
} as unknown as HTMLCanvasElement
}
describe('useDomClipping', () => {

View File

@@ -1,6 +1,5 @@
import { createTestingPinia } from '@pinia/testing'
import { fromAny } from '@total-typescript/shoehorn'
import { setActivePinia } from 'pinia'
import { createTestingPinia } from '@pinia/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { installErrorClearingHooks } from '@/composables/graph/useErrorClearingHooks'
@@ -195,7 +194,7 @@ describe('Widget change error clearing via onWidgetChanged', () => {
const store = useExecutionErrorStore()
vi.spyOn(app, 'rootGraph', 'get').mockReturnValue(
fromAny<LGraph, unknown>(undefined)
undefined as unknown as LGraph
)
store.lastNodeErrors = {
[String(node.id)]: {

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { Rectangle } from '@/lib/litegraph/src/infrastructure/Rectangle'
@@ -9,6 +8,7 @@ import {
createMockLGraphNode,
createMockLGraphGroup
} from '@/utils/__tests__/litegraphTestUtils'
import { useGraphHierarchy } from './useGraphHierarchy'
vi.mock('@/renderer/core/canvas/canvasStore')
@@ -36,10 +36,7 @@ describe('useGraphHierarchy', () => {
mockNode = createMockNode()
mockGroups = []
mockCanvasStore = fromAny<
Partial<ReturnType<typeof useCanvasStore>>,
unknown
>({
mockCanvasStore = {
canvas: {
graph: {
groups: mockGroups
@@ -54,7 +51,7 @@ describe('useGraphHierarchy', () => {
$dispose: vi.fn(),
_customProperties: new Set(),
_p: {}
})
} as unknown as Partial<ReturnType<typeof useCanvasStore>>
vi.mocked(useCanvasStore).mockReturnValue(
mockCanvasStore as ReturnType<typeof useCanvasStore>

View File

@@ -1,6 +1,5 @@
import { createTestingPinia } from '@pinia/testing'
import { fromAny } from '@total-typescript/shoehorn'
import { setActivePinia } from 'pinia'
import { createTestingPinia } from '@pinia/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { computed, nextTick, watch } from 'vue'
@@ -12,10 +11,10 @@ import {
createTestSubgraphNode
} from '@/lib/litegraph/src/subgraph/__fixtures__/subgraphHelpers'
import { NodeSlotType } from '@/lib/litegraph/src/types/globalEnums'
import { useMissingModelStore } from '@/platform/missingModel/missingModelStore'
import { useSettingStore } from '@/platform/settings/settingStore'
import { app } from '@/scripts/app'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useMissingModelStore } from '@/platform/missingModel/missingModelStore'
import { useSettingStore } from '@/platform/settings/settingStore'
import { usePromotionStore } from '@/stores/promotionStore'
import { useWidgetValueStore } from '@/stores/widgetValueStore'
@@ -278,20 +277,18 @@ describe('Widget slotMetadata reactivity on link disconnect', () => {
const secondPromotedView = promotedViews[1]
if (!secondPromotedView) throw new Error('Expected second promoted view')
fromAny<
{
;(
secondPromotedView as unknown as {
sourceNodeId: string
sourceWidgetName: string
},
unknown
>(secondPromotedView).sourceNodeId = '9999'
fromAny<
{
}
).sourceNodeId = '9999'
;(
secondPromotedView as unknown as {
sourceNodeId: string
sourceWidgetName: string
},
unknown
>(secondPromotedView).sourceWidgetName = 'stale_widget'
}
).sourceWidgetName = 'stale_widget'
const { vueNodeData } = useGraphNodeManager(graph)
const nodeData = vueNodeData.get(String(subgraphNode.id))

View File

@@ -1,8 +1,8 @@
import { fromPartial } from '@total-typescript/shoehorn'
import { afterEach, describe, expect, it, vi } from 'vitest'
import type { LGraphNode } from '@/lib/litegraph/src/LGraphNode'
import { createMockLGraphNode } from '@/utils/__tests__/litegraphTestUtils'
import { useImageMenuOptions } from './useImageMenuOptions'
vi.mock('vue-i18n', async (importOriginal) => {
@@ -112,11 +112,9 @@ describe('useImageMenuOptions', () => {
getType: vi.fn().mockResolvedValue(mockBlob)
}
mockClipboard(
fromPartial<Clipboard>({
read: vi.fn().mockResolvedValue([mockClipboardItem])
})
)
mockClipboard({
read: vi.fn().mockResolvedValue([mockClipboardItem])
} as unknown as Clipboard)
const { getImageMenuOptions } = useImageMenuOptions()
const options = getImageMenuOptions(node)
@@ -133,7 +131,7 @@ describe('useImageMenuOptions', () => {
it('handles missing clipboard API gracefully', async () => {
const node = createImageNode()
mockClipboard(fromPartial<Clipboard>({ read: undefined }))
mockClipboard({ read: undefined } as unknown as Clipboard)
const { getImageMenuOptions } = useImageMenuOptions()
const options = getImageMenuOptions(node)
@@ -150,11 +148,9 @@ describe('useImageMenuOptions', () => {
getType: vi.fn()
}
mockClipboard(
fromPartial<Clipboard>({
read: vi.fn().mockResolvedValue([mockClipboardItem])
})
)
mockClipboard({
read: vi.fn().mockResolvedValue([mockClipboardItem])
} as unknown as Clipboard)
const { getImageMenuOptions } = useImageMenuOptions()
const options = getImageMenuOptions(node)

View File

@@ -1,11 +1,10 @@
import { createTestingPinia } from '@pinia/testing'
import { fromAny, fromPartial } from '@total-typescript/shoehorn'
import { setActivePinia } from 'pinia'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import type { LGraphNode } from '@/lib/litegraph/src/litegraph'
import { api } from '@/scripts/api'
import { app } from '@/scripts/app'
import { api } from '@/scripts/api'
import { useNodeOutputStore } from '@/stores/nodeOutputStore'
import { useMaskEditorSaver } from './useMaskEditorSaver'
@@ -22,7 +21,7 @@ vi.mock('@/stores/maskEditorDataStore', () => ({
}))
function createMockCtx(): CanvasRenderingContext2D {
return fromPartial<CanvasRenderingContext2D>({
return {
drawImage: vi.fn(),
getImageData: vi.fn(() => ({
data: new Uint8ClampedArray(4 * 4 * 4),
@@ -31,11 +30,11 @@ function createMockCtx(): CanvasRenderingContext2D {
})),
putImageData: vi.fn(),
globalCompositeOperation: 'source-over'
})
} as unknown as CanvasRenderingContext2D
}
function createMockCanvas(): HTMLCanvasElement {
return fromPartial<HTMLCanvasElement>({
return {
width: 4,
height: 4,
getContext: vi.fn(() => createMockCtx()),
@@ -43,7 +42,7 @@ function createMockCanvas(): HTMLCanvasElement {
cb(new Blob(['x'], { type: 'image/png' }))
}),
toDataURL: vi.fn(() => 'data:image/png;base64,mock')
})
} as unknown as HTMLCanvasElement
}
const mockEditorStore: Record<string, HTMLCanvasElement | null> = {
@@ -97,7 +96,7 @@ describe('useMaskEditorSaver', () => {
app.nodeOutputs = {}
app.nodePreviewImages = {}
mockNode = fromAny<LGraphNode, unknown>({
mockNode = {
id: 42,
type: 'LoadImage',
images: [],
@@ -108,7 +107,7 @@ describe('useMaskEditorSaver', () => {
widgets_values: ['original.png [input]'],
properties: { image: 'original.png [input]' },
graph: { setDirtyCanvas: vi.fn() }
})
} as unknown as LGraphNode
mockDataStore.sourceNode = mockNode
mockDataStore.inputData = {
@@ -136,7 +135,7 @@ describe('useMaskEditorSaver', () => {
vi.spyOn(document, 'createElement').mockImplementation(
(tagName: string, options?: ElementCreationOptions) => {
if (tagName === 'canvas')
return fromAny<HTMLCanvasElement, unknown>(createMockCanvas())
return createMockCanvas() as unknown as HTMLCanvasElement
return originalCreateElement(tagName, options)
}
)

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import type { LGraphNode } from '@/lib/litegraph/src/litegraph'
@@ -45,12 +44,12 @@ vi.mock('@/stores/assetsStore', () => ({
}))
function createMockNode(): LGraphNode {
return fromAny<LGraphNode, unknown>({
return {
isUploading: false,
imgs: [new Image()],
graph: { setDirtyCanvas: vi.fn() },
size: [300, 400]
})
} as unknown as LGraphNode
}
function createFile(name = 'test.png'): File {

View File

@@ -1,8 +1,8 @@
import { fromAny, fromPartial } from '@total-typescript/shoehorn'
import { ref } from 'vue'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import type { ComfyNodeDefImpl } from '@/stores/nodeDefStore'
import { useNodePreviewAndDrag } from './useNodePreviewAndDrag'
const mockStartDrag = vi.fn()
@@ -72,9 +72,9 @@ describe('useNodePreviewAndDrag', () => {
toJSON: () => ({})
})
const mockEvent = fromPartial<MouseEvent>({
const mockEvent = {
currentTarget: mockElement
})
} as Partial<MouseEvent> as MouseEvent
result.handleMouseEnter(mockEvent)
expect(result.isHovered.value).toBe(true)
@@ -85,9 +85,9 @@ describe('useNodePreviewAndDrag', () => {
const result = useNodePreviewAndDrag(nodeDef)
const mockElement = document.createElement('div')
const mockEvent = fromPartial<MouseEvent>({
const mockEvent = {
currentTarget: mockElement
})
} as Partial<MouseEvent> as MouseEvent
result.handleMouseEnter(mockEvent)
expect(result.isHovered.value).toBe(false)
@@ -116,9 +116,9 @@ describe('useNodePreviewAndDrag', () => {
setData: vi.fn(),
setDragImage: vi.fn()
}
const mockEvent = fromAny<DragEvent, unknown>({
const mockEvent = {
dataTransfer: mockDataTransfer
})
} as unknown as DragEvent
result.handleDragStart(mockEvent)
@@ -151,10 +151,10 @@ describe('useNodePreviewAndDrag', () => {
result.isDragging.value = true
const mockEvent = fromPartial<DragEvent>({
const mockEvent = {
clientX: 100,
clientY: 200
})
} as Partial<DragEvent> as DragEvent
result.handleDragEnd(mockEvent)
@@ -168,11 +168,11 @@ describe('useNodePreviewAndDrag', () => {
result.isDragging.value = true
const mockEvent = fromPartial<DragEvent>({
const mockEvent = {
dataTransfer: { dropEffect: 'none' },
clientX: 300,
clientY: 400
})
} as Partial<DragEvent> as DragEvent
result.handleDragEnd(mockEvent)

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { useEventListener } from '@vueuse/core'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { nextTick } from 'vue'
@@ -80,10 +79,10 @@ describe('useServerLogs', () => {
// Simulate receiving a log event
const mockEvent = new CustomEvent('logs', {
detail: fromAny<LogsWsMessage, unknown>({
detail: {
type: 'logs',
entries: [{ m: 'Log message 1' }, { m: 'Log message 2' }]
})
} as unknown as LogsWsMessage
}) as CustomEvent<LogsWsMessage>
eventCallback(mockEvent)
@@ -104,14 +103,14 @@ describe('useServerLogs', () => {
) => void
const mockEvent = new CustomEvent('logs', {
detail: fromAny<LogsWsMessage, unknown>({
detail: {
type: 'logs',
entries: [
{ m: 'Log message 1 dont remove me' },
{ m: 'remove me' },
{ m: '' }
]
})
} as unknown as LogsWsMessage
}) as CustomEvent<LogsWsMessage>
eventCallback(mockEvent)

View File

@@ -1,4 +1,3 @@
import { fromAny } from '@total-typescript/shoehorn'
import { ref } from 'vue'
import { afterEach, describe, expect, it, vi } from 'vitest'
@@ -81,12 +80,10 @@ describe('useWaveAudioPlayer', () => {
const mockDecodeAudioData = vi.fn(() => Promise.resolve(mockAudioBuffer))
const mockClose = vi.fn().mockResolvedValue(undefined)
globalThis.AudioContext = fromAny<typeof AudioContext, unknown>(
class {
decodeAudioData = mockDecodeAudioData
close = mockClose
}
)
globalThis.AudioContext = class {
decodeAudioData = mockDecodeAudioData
close = mockClose
} as unknown as typeof AudioContext
mockFetchApi.mockResolvedValue({
ok: true,

View File

@@ -1,7 +1,7 @@
import { fromAny } from '@total-typescript/shoehorn'
import { describe, expect, it } from 'vitest'
import type { IBaseWidget } from '@/lib/litegraph/src/types/widgets'
import { matchPromotedInput } from './matchPromotedInput'
type MockInput = {
@@ -31,13 +31,10 @@ describe(matchPromotedInput, () => {
}
const matched = matchPromotedInput(
fromAny<
Array<{
name: string
_widget?: IBaseWidget
}>,
unknown
>([aliasInput, exactInput]),
[aliasInput, exactInput] as unknown as Array<{
name: string
_widget?: IBaseWidget
}>,
targetWidget
)
@@ -51,9 +48,7 @@ describe(matchPromotedInput, () => {
}
const matched = matchPromotedInput(
fromAny<Array<{ name: string; _widget?: IBaseWidget }>, unknown>([
aliasInput
]),
[aliasInput] as unknown as Array<{ name: string; _widget?: IBaseWidget }>,
targetWidget
)
@@ -70,13 +65,10 @@ describe(matchPromotedInput, () => {
}
const matched = matchPromotedInput(
fromAny<
Array<{
name: string
_widget?: IBaseWidget
}>,
unknown
>([firstAliasInput, secondAliasInput]),
[firstAliasInput, secondAliasInput] as unknown as Array<{
name: string
_widget?: IBaseWidget
}>,
targetWidget
)

View File

@@ -1,7 +1,6 @@
import { createTestingPinia } from '@pinia/testing'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, test, vi } from 'vitest'
import { fromAny } from '@total-typescript/shoehorn'
// Barrel import must come first to avoid circular dependency
// (promotedWidgetView → widgetMap → BaseWidget → LegacyWidget → barrel)
@@ -98,12 +97,11 @@ function promotedWidgets(node: SubgraphNode): PromotedWidgetView[] {
}
function callSyncPromotions(node: SubgraphNode) {
fromAny<
{
;(
node as unknown as {
_syncPromotions: () => void
},
unknown
>(node)._syncPromotions()
}
)._syncPromotions()
}
describe(createPromotedWidgetView, () => {
@@ -158,9 +156,7 @@ describe(createPromotedWidgetView, () => {
const [subgraphNode] = setupSubgraph()
const view = createPromotedWidgetView(subgraphNode, '1', 'myWidget')
// node is defined via Object.defineProperty at runtime but not on the TS interface
expect(fromAny<Record<string, unknown>, unknown>(view).node).toBe(
subgraphNode
)
expect((view as unknown as Record<string, unknown>).node).toBe(subgraphNode)
})
test('serialize is false', () => {
@@ -293,7 +289,7 @@ describe(createPromotedWidgetView, () => {
value: 'initial',
options: {}
} satisfies Pick<IBaseWidget, 'name' | 'type' | 'value' | 'options'>
const fallbackWidget = fromAny<IBaseWidget, unknown>(fallbackWidgetShape)
const fallbackWidget = fallbackWidgetShape as unknown as IBaseWidget
innerNode.widgets = [fallbackWidget]
const widgetValueStore = useWidgetValueStore()
@@ -402,13 +398,13 @@ describe(createPromotedWidgetView, () => {
subgraphNode.pos = [10, 20]
const innerNode = firstInnerNode(innerNodes)
const mouse = vi.fn(() => true)
const legacyWidget = fromAny<IBaseWidget, unknown>({
const legacyWidget = {
name: 'legacyMouse',
type: 'mystery-legacy',
value: 'val',
options: {},
mouse
})
} as unknown as IBaseWidget
innerNode.widgets = [legacyWidget]
const view = createPromotedWidgetView(
@@ -1452,20 +1448,17 @@ describe('widgets getter caching', () => {
subgraphNode.rootGraph.primaryCanvas = fakeCanvas as LGraphCanvas
const reconcileSpy = vi.spyOn(
fromAny<
{
_buildPromotionReconcileState: (
entries: Array<{ sourceNodeId: string; sourceWidgetName: string }>,
linkedEntries: Array<{
inputName: string
inputKey: string
sourceNodeId: string
sourceWidgetName: string
}>
) => unknown
},
unknown
>(subgraphNode),
subgraphNode as unknown as {
_buildPromotionReconcileState: (
entries: Array<{ sourceNodeId: string; sourceWidgetName: string }>,
linkedEntries: Array<{
inputName: string
inputKey: string
sourceNodeId: string
sourceWidgetName: string
}>
) => unknown
},
'_buildPromotionReconcileState'
)
@@ -1485,20 +1478,17 @@ describe('widgets getter caching', () => {
subgraphNode.rootGraph.primaryCanvas = fakeCanvas as LGraphCanvas
const reconcileSpy = vi.spyOn(
fromAny<
{
_buildPromotionReconcileState: (
entries: Array<{ sourceNodeId: string; sourceWidgetName: string }>,
linkedEntries: Array<{
inputName: string
inputKey: string
sourceNodeId: string
sourceWidgetName: string
}>
) => unknown
},
unknown
>(subgraphNode),
subgraphNode as unknown as {
_buildPromotionReconcileState: (
entries: Array<{ sourceNodeId: string; sourceWidgetName: string }>,
linkedEntries: Array<{
inputName: string
inputKey: string
sourceNodeId: string
sourceWidgetName: string
}>
) => unknown
},
'_buildPromotionReconcileState'
)
@@ -1532,14 +1522,9 @@ describe('widgets getter caching', () => {
subgraph.inputNode.slots[0].connect(linkedInputB, linkedNodeB)
const resolveSpy = vi.spyOn(
fromAny<
{
_resolveLinkedPromotionBySubgraphInput: (
...args: unknown[]
) => unknown
},
unknown
>(subgraphNode),
subgraphNode as unknown as {
_resolveLinkedPromotionBySubgraphInput: (...args: unknown[]) => unknown
},
'_resolveLinkedPromotionBySubgraphInput'
)
@@ -1938,34 +1923,32 @@ function createFakeCanvasContext() {
function createInspectableCanvasContext(fillText = vi.fn()) {
const fallback = vi.fn()
return fromAny<CanvasRenderingContext2D, unknown>(
new Proxy(
{
fillText,
beginPath: vi.fn(),
roundRect: vi.fn(),
rect: vi.fn(),
fill: vi.fn(),
stroke: vi.fn(),
moveTo: vi.fn(),
lineTo: vi.fn(),
arc: vi.fn(),
measureText: (text: string) => ({ width: text.length * 8 }),
fillStyle: '#fff',
strokeStyle: '#fff',
textAlign: 'left',
globalAlpha: 1,
lineWidth: 1
} as Record<string, unknown>,
{
get(target, key) {
if (typeof key === 'string' && key in target)
return target[key as keyof typeof target]
return fallback
}
return new Proxy(
{
fillText,
beginPath: vi.fn(),
roundRect: vi.fn(),
rect: vi.fn(),
fill: vi.fn(),
stroke: vi.fn(),
moveTo: vi.fn(),
lineTo: vi.fn(),
arc: vi.fn(),
measureText: (text: string) => ({ width: text.length * 8 }),
fillStyle: '#fff',
strokeStyle: '#fff',
textAlign: 'left',
globalAlpha: 1,
lineWidth: 1
} as Record<string, unknown>,
{
get(target, key) {
if (typeof key === 'string' && key in target)
return target[key as keyof typeof target]
return fallback
}
)
)
}
) as unknown as CanvasRenderingContext2D
}
function createTwoLevelNestedSubgraph() {

Some files were not shown because too many files have changed in this diff Show More