Compare commits
2 Commits
sno-qa-103
...
sno-fix-i1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b1e593086f | ||
|
|
4e886be234 |
@@ -1,278 +0,0 @@
|
||||
---
|
||||
name: reproduce-issue
|
||||
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
|
||||
---
|
||||
|
||||
# Issue Reproduction Skill
|
||||
|
||||
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two videos are produced:
|
||||
|
||||
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
|
||||
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
|
||||
|
||||
```
|
||||
Phase 1: Research → Read issue, understand prerequisites
|
||||
Phase 2: Environment → Install custom nodes, load workflows, configure settings
|
||||
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
|
||||
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
|
||||
Phase 5: Report → Generate a structured reproduction report
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
|
||||
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
|
||||
- `gh` CLI (authenticated, for reading issues)
|
||||
- ComfyUI backend with Python environment (for installing custom nodes)
|
||||
|
||||
## Phase 1: Research the Issue
|
||||
|
||||
1. Fetch the issue details:
|
||||
|
||||
```bash
|
||||
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
|
||||
```
|
||||
|
||||
2. Extract from the issue body:
|
||||
- **Reproduction steps** (the exact sequence)
|
||||
- **Prerequisites**: specific workflows, custom nodes, settings, models
|
||||
- **Environment**: OS, browser, ComfyUI version
|
||||
- **Media**: screenshots or videos showing the bug
|
||||
|
||||
3. Search the codebase for related code:
|
||||
- Find the feature/component mentioned in the issue
|
||||
- Understand how it works currently
|
||||
- Identify what state the UI needs to be in
|
||||
|
||||
## Phase 2: Environment Setup
|
||||
|
||||
Set up everything the issue requires BEFORE attempting reproduction.
|
||||
|
||||
### Custom Nodes
|
||||
|
||||
If the issue mentions custom nodes:
|
||||
|
||||
```bash
|
||||
# Find the custom node repo
|
||||
# Clone into ComfyUI's custom_nodes directory
|
||||
cd <comfyui_path>/custom_nodes
|
||||
git clone <custom_node_repo_url>
|
||||
|
||||
# Install dependencies if needed
|
||||
cd <custom_node_name>
|
||||
pip install -r requirements.txt 2>/dev/null || true
|
||||
|
||||
# Restart ComfyUI server to load the new nodes
|
||||
```
|
||||
|
||||
### Workflows
|
||||
|
||||
If the issue references a specific workflow:
|
||||
|
||||
```bash
|
||||
# Download workflow JSON if a URL is provided
|
||||
curl -L "<workflow_url>" -o /tmp/test-workflow.json
|
||||
|
||||
# Load it via the API
|
||||
curl -X POST http://127.0.0.1:8188/api/workflow \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @/tmp/test-workflow.json
|
||||
```
|
||||
|
||||
Or load via playwright-cli:
|
||||
|
||||
```bash
|
||||
playwright-cli goto "http://127.0.0.1:8188"
|
||||
# Drag-and-drop or use File > Open to load the workflow
|
||||
```
|
||||
|
||||
### Settings
|
||||
|
||||
If the issue requires specific settings:
|
||||
|
||||
```bash
|
||||
# Use playwright-cli to open settings and change them
|
||||
playwright-cli press "Control+,"
|
||||
playwright-cli snapshot
|
||||
# Find and modify the relevant setting
|
||||
```
|
||||
|
||||
## Phase 3: Interactive Exploration — Research Video
|
||||
|
||||
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
|
||||
|
||||
```bash
|
||||
# Open browser and start video recording
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
|
||||
# Take a snapshot to see current state
|
||||
playwright-cli snapshot
|
||||
|
||||
# Interact based on what you see
|
||||
playwright-cli click <ref>
|
||||
playwright-cli fill <ref> "text"
|
||||
playwright-cli press "Control+s"
|
||||
|
||||
# Check results
|
||||
playwright-cli snapshot
|
||||
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
|
||||
```
|
||||
|
||||
### Key Principles
|
||||
|
||||
- **Observe before acting**: Always `snapshot` before interacting
|
||||
- **Retry and adapt**: If a step fails, try a different approach
|
||||
- **Document what works**: Keep notes on which steps trigger the bug
|
||||
- **Don't give up**: Try multiple approaches if the first doesn't work
|
||||
- **Establish prerequisites**: Many bugs require specific UI state:
|
||||
- Save a workflow first (File > Save)
|
||||
- Make changes to dirty the workflow
|
||||
- Open multiple tabs
|
||||
- Add specific node types
|
||||
- Change settings
|
||||
- Resize the window
|
||||
|
||||
### Common ComfyUI Interactions via playwright-cli
|
||||
|
||||
| Action | Command |
|
||||
| ------------------- | -------------------------------------------------------------- |
|
||||
| Open hamburger menu | `playwright-cli click` on the C logo button |
|
||||
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
|
||||
| Add node | Double-click canvas → type node name → select from results |
|
||||
| Connect nodes | Drag from output slot to input slot |
|
||||
| Save workflow | `playwright-cli press "Control+s"` |
|
||||
| Save As | Menu > File > Save As |
|
||||
| Select node | Click on the node |
|
||||
| Delete node | Select → `playwright-cli press "Delete"` |
|
||||
| Right-click menu | `playwright-cli click <ref> --button right` |
|
||||
| Keyboard shortcut | `playwright-cli press "Control+z"` |
|
||||
|
||||
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
|
||||
|
||||
Once the bug is confirmed, **stop the research video** and **close the research browser**:
|
||||
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
Now start a **fresh browser session** for the clean reproduce video (Video 2).
|
||||
|
||||
**IMPORTANT constraints:**
|
||||
|
||||
- **Max 5 minutes** — the reproduce video must be short and focused
|
||||
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
|
||||
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
|
||||
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
|
||||
|
||||
1. **Open browser and start recording**:
|
||||
|
||||
```bash
|
||||
playwright-cli open "http://127.0.0.1:8188"
|
||||
playwright-cli video-start
|
||||
```
|
||||
|
||||
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
|
||||
|
||||
3. **Take key screenshots** at critical moments:
|
||||
|
||||
```bash
|
||||
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
|
||||
# ... trigger the bug ...
|
||||
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
|
||||
```
|
||||
|
||||
4. **Stop recording and close** immediately after the bug is demonstrated:
|
||||
```bash
|
||||
playwright-cli video-stop
|
||||
playwright-cli close
|
||||
```
|
||||
|
||||
## Phase 5: Generate Report
|
||||
|
||||
Create a reproduction report at `tmp/qa/reproduce-report.md`:
|
||||
|
||||
```markdown
|
||||
# Issue Reproduction Report
|
||||
|
||||
- **Issue**: <issue_url>
|
||||
- **Title**: <issue_title>
|
||||
- **Date**: <today>
|
||||
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
|
||||
|
||||
## Environment
|
||||
|
||||
- ComfyUI Server: <url>
|
||||
- OS: <os>
|
||||
- Custom Nodes Installed: <list or "none">
|
||||
- Settings Changed: <list or "none">
|
||||
|
||||
## Prerequisites
|
||||
|
||||
List everything that had to be set up before the bug could be triggered:
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
## Reproduction Steps
|
||||
|
||||
Minimal steps to reproduce (the clean sequence):
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
3. ...
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
<from the issue>
|
||||
|
||||
## Actual Behavior
|
||||
|
||||
<what actually happened>
|
||||
|
||||
## Evidence
|
||||
|
||||
- Research video: `research-video/video.webm` (full exploration session)
|
||||
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
|
||||
- Screenshots: `before-bug.png`, `bug-visible.png`
|
||||
|
||||
## Root Cause Analysis (if identified)
|
||||
|
||||
<code pointers, hypothesis about what's going wrong>
|
||||
|
||||
## Notes
|
||||
|
||||
<any additional observations, workarounds discovered, related issues>
|
||||
```
|
||||
|
||||
## Handling Failures
|
||||
|
||||
If the bug **cannot be reproduced**:
|
||||
|
||||
1. Document what you tried and why it didn't work
|
||||
2. Check if the issue was already fixed (search git log for related commits)
|
||||
3. Check if it's environment-specific (OS, browser, specific version)
|
||||
4. Set report status to "Not Reproduced" with detailed notes
|
||||
5. The report is still valuable — it saves others from repeating the same investigation
|
||||
|
||||
## CI Integration
|
||||
|
||||
In CI, this skill runs as a Claude Code agent with:
|
||||
|
||||
- `ANTHROPIC_API_KEY` for Claude
|
||||
- `GEMINI_API_KEY` for initial issue analysis (optional)
|
||||
- ComfyUI server pre-started in the container
|
||||
- `playwright-cli` pre-installed
|
||||
|
||||
The CI workflow:
|
||||
|
||||
1. Gemini generates a reproduce guide (markdown) from the issue
|
||||
2. Claude agent receives the guide and runs this skill
|
||||
3. Claude explores interactively, installs dependencies, retries
|
||||
4. Claude records a clean demo once reproduced
|
||||
5. Video and report are uploaded as artifacts
|
||||
@@ -1,361 +0,0 @@
|
||||
---
|
||||
name: comfy-qa
|
||||
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
|
||||
---
|
||||
|
||||
# ComfyUI Frontend QA Skill
|
||||
|
||||
Perform comprehensive quality assurance of the ComfyUI frontend application by navigating all routes, clicking interactive elements, and testing features. Generate a structured report and submit it as a draft PR.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 22+
|
||||
- `pnpm` package manager
|
||||
- `gh` CLI (authenticated)
|
||||
- `playwright-cli` (browser automation): `npm install -g @playwright/cli@latest`
|
||||
|
||||
## Step 1: Environment Detection & Setup
|
||||
|
||||
Detect the runtime environment and ensure the app is accessible.
|
||||
|
||||
### CI Environment
|
||||
|
||||
If `CI=true` is set:
|
||||
|
||||
1. The ComfyUI backend is pre-configured in the CI container (`ghcr.io/comfy-org/comfyui-ci-container`)
|
||||
2. Frontend dist is already built and served by the backend
|
||||
3. Server runs at `http://127.0.0.1:8188`
|
||||
4. Skip user prompts — run fully automated
|
||||
|
||||
### Local Environment
|
||||
|
||||
If `CI` is not set:
|
||||
|
||||
1. **Ask the user**: "Is a ComfyUI server already running? If so, what URL? (default: http://127.0.0.1:8188)"
|
||||
- If yes: use the provided URL
|
||||
- If no: offer to start one:
|
||||
|
||||
```bash
|
||||
# Option A: Use existing ComfyUI installation
|
||||
# Ask for the path to ComfyUI, then:
|
||||
cd <comfyui_path>
|
||||
python main.py --cpu --multi-user --front-end-root <frontend_dist_path> &
|
||||
|
||||
# Option B: Build frontend and use preview server (no backend features)
|
||||
pnpm build && pnpm preview &
|
||||
```
|
||||
|
||||
2. Wait for server readiness by polling the URL (retry with 2s intervals, 60s timeout)
|
||||
|
||||
### Browser Automation Setup
|
||||
|
||||
Use **playwright-cli** for browser interaction via Bash commands:
|
||||
|
||||
```bash
|
||||
playwright-cli open http://127.0.0.1:8188 # open browser and navigate
|
||||
playwright-cli snapshot # capture snapshot with element refs
|
||||
playwright-cli click e1 # click by element ref from snapshot
|
||||
playwright-cli press Tab # keyboard shortcuts
|
||||
playwright-cli screenshot --filename=f.png # save screenshot
|
||||
```
|
||||
|
||||
playwright-cli is headless by default (CI-friendly). Each command outputs the current page snapshot with element references (`e1`, `e2`, …) that you use for subsequent `click`, `fill`, `hover` commands. Always run `snapshot` before interacting to get fresh refs.
|
||||
|
||||
For local dev servers behind proxies, adjust the URL accordingly (e.g., `https://[port].stukivx.xyz` pattern if configured).
|
||||
|
||||
## Step 2: QA Test Plan
|
||||
|
||||
Navigate to the application URL and systematically test each area below. For each test, record:
|
||||
|
||||
- **Status**: pass / fail / skip (with reason)
|
||||
- **Notes**: any issues, unexpected behavior, or visual glitches
|
||||
- **Screenshots**: take screenshots of failures or notable states
|
||||
|
||||
### 2.1 Application Load & Routes
|
||||
|
||||
| Test | Steps |
|
||||
| ----------------- | ------------------------------------------------------------ |
|
||||
| Root route loads | Navigate to `/` — GraphView should render with canvas |
|
||||
| User select route | Navigate to `/user-select` — user selection UI should appear |
|
||||
| Default redirect | If multi-user mode, `/` redirects to `/user-select` first |
|
||||
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
|
||||
|
||||
### 2.2 Canvas & Graph View
|
||||
|
||||
| Test | Steps |
|
||||
| ------------------------- | -------------------------------------------------------------- |
|
||||
| Canvas renders | The LiteGraph canvas is visible and interactive |
|
||||
| Pan canvas | Click and drag on empty canvas area |
|
||||
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
|
||||
| Fit view | Press `.` key — canvas fits to content |
|
||||
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
|
||||
| Add node via search | Open search box, find and add a node |
|
||||
| Delete node | Select a node, press Delete key |
|
||||
| Connect nodes | Drag from output slot to input slot |
|
||||
| Disconnect nodes | Right-click a link and remove, or drag from connected slot |
|
||||
| Multi-select | Shift+click or drag-select multiple nodes |
|
||||
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
|
||||
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
|
||||
| Node context menu | Right-click a node — menu appears with all expected options |
|
||||
| Canvas context menu | Right-click empty canvas — menu appears |
|
||||
|
||||
### 2.3 Node Operations
|
||||
|
||||
| Test | Steps |
|
||||
| ------------------- | ---------------------------------------------------------- |
|
||||
| Bypass node | Select node, Ctrl+B — node shows bypass state |
|
||||
| Mute node | Select node, Ctrl+M — node shows muted state |
|
||||
| Collapse node | Select node, Alt+C — node collapses |
|
||||
| Pin node | Select node, press P — node becomes pinned |
|
||||
| Rename node | Double-click node title — edit mode activates |
|
||||
| Node color | Right-click > Color — color picker works |
|
||||
| Group nodes | Select multiple nodes, Ctrl+G — group created |
|
||||
| Ungroup | Right-click group > Ungroup |
|
||||
| Widget interactions | Toggle checkboxes, adjust sliders, type in text fields |
|
||||
| Combo widget | Click dropdown widgets — options appear and are selectable |
|
||||
|
||||
### 2.4 Sidebar Tabs
|
||||
|
||||
| Test | Steps |
|
||||
| ---------------------- | ------------------------------------------------------ |
|
||||
| Workflows tab | Press W — workflows sidebar opens with saved workflows |
|
||||
| Node Library tab | Press N — node library opens with categories |
|
||||
| Model Library tab | Press M — model library opens |
|
||||
| Assets tab | Press A — assets browser opens |
|
||||
| Tab toggle | Press same key again — sidebar closes |
|
||||
| Search in sidebar | Type in search box — results filter |
|
||||
| Drag node from library | Drag a node from library onto canvas |
|
||||
|
||||
### 2.5 Topbar & Workflow Tabs
|
||||
|
||||
| Test | Steps |
|
||||
| -------------------- | ------------------------------------------------------ |
|
||||
| Workflow tab display | Current workflow name shown in tab bar |
|
||||
| New workflow | Ctrl+N — new blank workflow created |
|
||||
| Rename workflow | Double-click workflow tab |
|
||||
| Tab context menu | Right-click workflow tab — menu with Close/Rename/etc. |
|
||||
| Multiple tabs | Open multiple workflows, switch between them |
|
||||
| Queue button | Click Queue/Run button — prompt queues |
|
||||
| Batch count | Click batch count editor, change value |
|
||||
| Menu hamburger | Click hamburger menu — options appear |
|
||||
|
||||
### 2.6 Settings Dialog
|
||||
|
||||
| Test | Steps |
|
||||
| ---------------- | ---------------------------------------------------- |
|
||||
| Open settings | Press Ctrl+, or click settings button |
|
||||
| Settings tabs | Navigate through all setting categories |
|
||||
| Change a setting | Toggle a boolean setting — it persists after closing |
|
||||
| Search settings | Type in settings search box — results filter |
|
||||
| Keybindings tab | Navigate to keybindings panel |
|
||||
| About tab | Navigate to about panel — version info shown |
|
||||
| Close settings | Press Escape or click close button |
|
||||
|
||||
### 2.7 Bottom Panel
|
||||
|
||||
| Test | Steps |
|
||||
| ------------------- | -------------------------------------- |
|
||||
| Toggle panel | Press Ctrl+` — bottom panel opens |
|
||||
| Logs tab | Logs/terminal tab shows server output |
|
||||
| Shortcuts tab | Shortcuts reference is displayed |
|
||||
| Keybindings display | Press Ctrl+Shift+K — keybindings panel |
|
||||
|
||||
### 2.8 Execution & Queue
|
||||
|
||||
| Test | Steps |
|
||||
| -------------- | ----------------------------------------------------- |
|
||||
| Queue prompt | Load default workflow, click Queue — execution starts |
|
||||
| Queue progress | Progress indicator shows during execution |
|
||||
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
|
||||
| Job history | Open job history sidebar — past executions listed |
|
||||
| Clear history | Clear execution history via menu |
|
||||
|
||||
### 2.9 Workflow File Operations
|
||||
|
||||
| Test | Steps |
|
||||
| --------------- | ------------------------------------------------- |
|
||||
| Save workflow | Ctrl+S — workflow saves (check for prompt if new) |
|
||||
| Open workflow | Ctrl+O — file picker or workflow browser opens |
|
||||
| Export JSON | Menu > Export — workflow JSON downloads |
|
||||
| Import workflow | Drag a .json workflow file onto canvas |
|
||||
| Load default | Menu > Load Default — default workflow loads |
|
||||
| Clear workflow | Menu > Clear — canvas clears (after confirmation) |
|
||||
|
||||
### 2.10 Advanced Features
|
||||
|
||||
| Test | Steps |
|
||||
| --------------- | ------------------------------------------------- |
|
||||
| Minimap | Alt+M — minimap toggle |
|
||||
| Focus mode | Toggle focus mode |
|
||||
| Canvas lock | Press H to lock, V to unlock |
|
||||
| Link visibility | Ctrl+Shift+L — toggle links |
|
||||
| Subgraph | Select nodes > Ctrl+Shift+E — convert to subgraph |
|
||||
|
||||
### 2.11 Error Handling
|
||||
|
||||
| Test | Steps |
|
||||
| --------------------- | -------------------------------------------- |
|
||||
| Missing nodes dialog | Load workflow with non-existent node types |
|
||||
| Missing models dialog | Trigger missing model warning |
|
||||
| Network error | Disconnect backend, verify graceful handling |
|
||||
| Invalid workflow | Try loading malformed JSON |
|
||||
|
||||
### 2.12 Responsive & Accessibility
|
||||
|
||||
| Test | Steps |
|
||||
| ------------------- | ------------------------------------- |
|
||||
| Window resize | Resize browser window — layout adapts |
|
||||
| Keyboard navigation | Tab through interactive elements |
|
||||
| Sidebar resize | Drag sidebar edge to resize |
|
||||
|
||||
## Step 3: Generate Report
|
||||
|
||||
After completing all tests, generate a markdown report file.
|
||||
|
||||
### Report Location
|
||||
|
||||
```
|
||||
docs/qa/YYYY-MM-DD-NNN-report.md
|
||||
```
|
||||
|
||||
Where:
|
||||
|
||||
- `YYYY-MM-DD` is today's date
|
||||
- `NNN` is a zero-padded increment index (001, 002, etc.)
|
||||
|
||||
To determine the increment, check existing files:
|
||||
|
||||
```bash
|
||||
ls docs/qa/ | grep "$(date +%Y-%m-%d)" | wc -l
|
||||
```
|
||||
|
||||
### Report Template
|
||||
|
||||
```markdown
|
||||
# QA Report: ComfyUI Frontend
|
||||
|
||||
**Date**: YYYY-MM-DD
|
||||
**Environment**: CI / Local (OS, Browser)
|
||||
**Frontend Version**: (git sha or version)
|
||||
**Agent**: Claude / Codex / Other
|
||||
**Server URL**: http://...
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | Pass | Fail | Skip | Total |
|
||||
| --------------- | ---- | ---- | ---- | ----- |
|
||||
| Routes & Load | | | | |
|
||||
| Canvas | | | | |
|
||||
| Node Operations | | | | |
|
||||
| Sidebar | | | | |
|
||||
| Topbar | | | | |
|
||||
| Settings | | | | |
|
||||
| Bottom Panel | | | | |
|
||||
| Execution | | | | |
|
||||
| File Operations | | | | |
|
||||
| Advanced | | | | |
|
||||
| Error Handling | | | | |
|
||||
| Responsive | | | | |
|
||||
| **Total** | | | | |
|
||||
|
||||
## Results
|
||||
|
||||
### Routes & Load
|
||||
|
||||
- [x] Root route loads — pass
|
||||
- [ ] ...
|
||||
|
||||
### Canvas & Graph View
|
||||
|
||||
- [x] Canvas renders — pass
|
||||
- [ ] ...
|
||||
|
||||
(repeat for each category)
|
||||
|
||||
## Issues Found
|
||||
|
||||
### Issue 1: [Title]
|
||||
|
||||
- **Severity**: critical / major / minor / cosmetic
|
||||
- **Steps to reproduce**: ...
|
||||
- **Expected**: ...
|
||||
- **Actual**: ...
|
||||
- **Screenshot**: (if available)
|
||||
|
||||
## Notes
|
||||
|
||||
Any additional observations, performance notes, or suggestions.
|
||||
```
|
||||
|
||||
## Step 4: Commit and Push Report
|
||||
|
||||
### In CI (when `CI=true`)
|
||||
|
||||
Save the report directly to `$QA_ARTIFACTS` (the CI workflow uploads this as
|
||||
an artifact and posts results as a PR comment). Do **not** commit, push, or
|
||||
create a new PR.
|
||||
|
||||
### Local / interactive use
|
||||
|
||||
When running locally, create a draft PR after committing:
|
||||
|
||||
```bash
|
||||
# Ensure on a feature branch
|
||||
BRANCH_NAME="qa/$(date +%Y-%m-%d)-$(git rev-parse --short HEAD)"
|
||||
git checkout -b "$BRANCH_NAME" 2>/dev/null || git checkout "$BRANCH_NAME"
|
||||
|
||||
git add docs/qa/
|
||||
git commit -m "docs: add QA report $(date +%Y-%m-%d)
|
||||
|
||||
Automated QA report covering all frontend routes and features."
|
||||
git push -u origin "$BRANCH_NAME"
|
||||
|
||||
# Create draft PR assigned to comfy-pr-bot
|
||||
gh pr create \
|
||||
--draft \
|
||||
--title "QA Report: $(date +%Y-%m-%d)" \
|
||||
--body "## QA Report
|
||||
|
||||
Automated frontend QA run covering all routes and interactive features.
|
||||
|
||||
See \`docs/qa/\` for the full report.
|
||||
|
||||
/cc @comfy-pr-bot" \
|
||||
--assignee comfy-pr-bot
|
||||
```
|
||||
|
||||
## Execution Notes
|
||||
|
||||
### Cross-Platform Considerations
|
||||
|
||||
- **Windows**: Use `pwsh` or `cmd` equivalents for shell commands. `gh` CLI works on all platforms.
|
||||
- **macOS**: Keyboard shortcuts use Cmd instead of Ctrl in the actual app, but Playwright sends OS-appropriate keys.
|
||||
- **Linux**: Primary CI platform. Screenshot baselines are Linux-only.
|
||||
|
||||
### Agent Compatibility
|
||||
|
||||
This skill uses **playwright-cli** (`@playwright/cli`) — a token-efficient CLI designed for coding agents. Install it once with `npm install -g @playwright/cli@latest`, then use `Bash` to run commands.
|
||||
|
||||
The key operations and their playwright-cli equivalents:
|
||||
|
||||
| Action | Command |
|
||||
| ---------------- | ---------------------------------------- |
|
||||
| Navigate to URL | `playwright-cli goto <url>` |
|
||||
| Get element refs | `playwright-cli snapshot` |
|
||||
| Click element | `playwright-cli click <ref>` |
|
||||
| Type text | `playwright-cli fill <ref> <text>` |
|
||||
| Press shortcut | `playwright-cli press <key>` |
|
||||
| Take screenshot | `playwright-cli screenshot --filename=f` |
|
||||
| Hover element | `playwright-cli hover <ref>` |
|
||||
| Select dropdown | `playwright-cli select <ref> <value>` |
|
||||
|
||||
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation or major interactions to refresh refs before acting.
|
||||
|
||||
### Tips for Reliable QA
|
||||
|
||||
1. **Wait for page stability** before interacting — check that elements are visible and enabled
|
||||
2. **Take a snapshot after each major navigation** to verify state
|
||||
3. **Don't use fixed timeouts** — poll for expected conditions
|
||||
4. **Record the full page snapshot** at the start for baseline comparison
|
||||
5. **If a test fails**, document it and continue — don't abort the entire QA run
|
||||
6. **Group related tests** — complete one category before moving to the next
|
||||
11
.github/actions/setup-comfyui-server/action.yaml
vendored
@@ -44,17 +44,12 @@ runs:
|
||||
python -m pip install --upgrade pip
|
||||
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
pip install -r requirements.txt
|
||||
pip install wait-for-it
|
||||
|
||||
- name: Start ComfyUI server
|
||||
if: ${{ inputs.launch_server == 'true' }}
|
||||
shell: bash
|
||||
working-directory: ComfyUI
|
||||
env:
|
||||
EXTRA_SERVER_PARAMS: ${{ inputs.extra_server_params }}
|
||||
run: |
|
||||
python main.py --cpu --multi-user --front-end-root ../dist $EXTRA_SERVER_PARAMS &
|
||||
for i in $(seq 1 300); do
|
||||
curl -sf http://127.0.0.1:8188/api/system_stats >/dev/null 2>&1 && echo "Server ready" && exit 0
|
||||
sleep 2
|
||||
done
|
||||
echo "::error::ComfyUI server did not start within 600s" && exit 1
|
||||
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
|
||||
wait-for-it --service 127.0.0.1:8188 -t 600
|
||||
|
||||
1084
.github/workflows/pr-qa.yaml
vendored
1
.github/workflows/release-version-bump.yaml
vendored
@@ -30,7 +30,6 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
bump-version:
|
||||
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
1
.github/workflows/weekly-docs-check.yaml
vendored
@@ -18,7 +18,6 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
docs-check:
|
||||
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
|
||||
5
.gitignore
vendored
@@ -99,7 +99,4 @@ vitest.config.*.timestamp*
|
||||
# Weekly docs check output
|
||||
/output.txt
|
||||
|
||||
.amp
|
||||
.playwright-cli/
|
||||
.playwright/
|
||||
.claude/scheduled_tasks.lock
|
||||
.amp
|
||||
@@ -9,7 +9,6 @@
|
||||
"packages/registry-types/src/comfyRegistryTypes.ts",
|
||||
"public/materialdesignicons.min.css",
|
||||
"src/types/generatedManagerTypes.ts",
|
||||
"**/__fixtures__/**/*.json",
|
||||
"scripts/qa-report-template.html"
|
||||
"**/__fixtures__/**/*.json"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -51,9 +51,6 @@
|
||||
# Manager
|
||||
/src/workbench/extensions/manager/ @viva-jinyi @christian-byrne @ltdrdata
|
||||
|
||||
# Model-to-node mappings (cloud team)
|
||||
/src/platform/assets/mappings/ @deepme987
|
||||
|
||||
# LLM Instructions (blank on purpose)
|
||||
.claude/
|
||||
.cursor/
|
||||
|
||||
@@ -10,7 +10,7 @@ import { createApp } from 'vue'
|
||||
|
||||
import App from './App.vue'
|
||||
import './assets/css/style.css'
|
||||
import { i18n } from './i18n'
|
||||
import { i18n, loadLocale } from './i18n'
|
||||
import router from './router'
|
||||
|
||||
const ComfyUIPreset = definePreset(Aura, {
|
||||
@@ -20,6 +20,11 @@ const ComfyUIPreset = definePreset(Aura, {
|
||||
}
|
||||
})
|
||||
|
||||
// Load the initial locale before mounting to avoid rendering with missing translations.
|
||||
// Errors are caught to prevent a failed locale fetch from bricking startup; the app
|
||||
// falls back to English in that case.
|
||||
await loadLocale(i18n.global.locale.value).catch(() => {})
|
||||
|
||||
const app = createApp(App)
|
||||
const pinia = createPinia()
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ type ValidationState = InstallValidation['basePath']
|
||||
type IndexedUpdate = InstallValidation & Record<string, ValidationState>
|
||||
|
||||
/** State of a maintenance task, managed by the maintenance task store. */
|
||||
class MaintenanceTaskRunner {
|
||||
export class MaintenanceTaskRunner {
|
||||
constructor(readonly task: MaintenanceTask) {}
|
||||
|
||||
private _state?: MaintenanceTaskState
|
||||
|
||||
2
apps/website/.gitignore
vendored
@@ -1,2 +0,0 @@
|
||||
dist/
|
||||
.astro/
|
||||
@@ -1,24 +0,0 @@
|
||||
import { defineConfig } from 'astro/config'
|
||||
import vue from '@astrojs/vue'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
|
||||
export default defineConfig({
|
||||
site: 'https://comfy.org',
|
||||
output: 'static',
|
||||
integrations: [vue()],
|
||||
vite: {
|
||||
plugins: [tailwindcss()]
|
||||
},
|
||||
build: {
|
||||
assetsPrefix: process.env.VERCEL_URL
|
||||
? `https://${process.env.VERCEL_URL}`
|
||||
: undefined
|
||||
},
|
||||
i18n: {
|
||||
locales: ['en', 'zh-CN'],
|
||||
defaultLocale: 'en',
|
||||
routing: {
|
||||
prefixDefaultLocale: false
|
||||
}
|
||||
}
|
||||
})
|
||||
@@ -1,80 +0,0 @@
|
||||
{
|
||||
"name": "@comfyorg/website",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "astro dev",
|
||||
"build": "astro build",
|
||||
"preview": "astro preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"@comfyorg/design-system": "workspace:*",
|
||||
"@vercel/analytics": "catalog:",
|
||||
"vue": "catalog:"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@astrojs/vue": "catalog:",
|
||||
"@tailwindcss/vite": "catalog:",
|
||||
"astro": "catalog:",
|
||||
"tailwindcss": "catalog:",
|
||||
"typescript": "catalog:"
|
||||
},
|
||||
"nx": {
|
||||
"tags": [
|
||||
"scope:website",
|
||||
"type:app"
|
||||
],
|
||||
"targets": {
|
||||
"dev": {
|
||||
"executor": "nx:run-commands",
|
||||
"continuous": true,
|
||||
"options": {
|
||||
"cwd": "apps/website",
|
||||
"command": "astro dev"
|
||||
}
|
||||
},
|
||||
"serve": {
|
||||
"executor": "nx:run-commands",
|
||||
"continuous": true,
|
||||
"options": {
|
||||
"cwd": "apps/website",
|
||||
"command": "astro dev"
|
||||
}
|
||||
},
|
||||
"build": {
|
||||
"executor": "nx:run-commands",
|
||||
"cache": true,
|
||||
"dependsOn": [
|
||||
"^build"
|
||||
],
|
||||
"options": {
|
||||
"cwd": "apps/website",
|
||||
"command": "astro build"
|
||||
},
|
||||
"outputs": [
|
||||
"{projectRoot}/dist"
|
||||
]
|
||||
},
|
||||
"preview": {
|
||||
"executor": "nx:run-commands",
|
||||
"continuous": true,
|
||||
"dependsOn": [
|
||||
"build"
|
||||
],
|
||||
"options": {
|
||||
"cwd": "apps/website",
|
||||
"command": "astro preview"
|
||||
}
|
||||
},
|
||||
"typecheck": {
|
||||
"executor": "nx:run-commands",
|
||||
"cache": true,
|
||||
"options": {
|
||||
"cwd": "apps/website",
|
||||
"command": "astro check"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
1
apps/website/src/env.d.ts
vendored
@@ -1 +0,0 @@
|
||||
/// <reference types="astro/client" />
|
||||
@@ -1,2 +0,0 @@
|
||||
@import 'tailwindcss';
|
||||
@import '@comfyorg/design-system/css/base.css';
|
||||
@@ -1,9 +0,0 @@
|
||||
{
|
||||
"extends": "astro/tsconfigs/strict",
|
||||
"compilerOptions": {
|
||||
"paths": {
|
||||
"@/*": ["./src/*"]
|
||||
}
|
||||
},
|
||||
"include": ["src/**/*", "astro.config.mjs"]
|
||||
}
|
||||
@@ -1,169 +0,0 @@
|
||||
{
|
||||
"id": "f1a2b3c4-d5e6-7890-abcd-ef1234567890",
|
||||
"revision": 0,
|
||||
"last_node_id": 2,
|
||||
"last_link_id": 0,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 2,
|
||||
"type": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
|
||||
"pos": [400, 300],
|
||||
"size": [400, 200],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"properties": {},
|
||||
"widgets_values": []
|
||||
}
|
||||
],
|
||||
"links": [],
|
||||
"groups": [],
|
||||
"definitions": {
|
||||
"subgraphs": [
|
||||
{
|
||||
"id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
|
||||
"version": 1,
|
||||
"state": {
|
||||
"lastGroupId": 0,
|
||||
"lastNodeId": 120,
|
||||
"lastLinkId": 276,
|
||||
"lastRerouteId": 0
|
||||
},
|
||||
"revision": 0,
|
||||
"config": {},
|
||||
"name": "Slot Drift Duplicate Links",
|
||||
"inputNode": {
|
||||
"id": -10,
|
||||
"bounding": [0, 300, 120, 60]
|
||||
},
|
||||
"outputNode": {
|
||||
"id": -20,
|
||||
"bounding": [900, 300, 120, 60]
|
||||
},
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"widgets": [],
|
||||
"nodes": [
|
||||
{
|
||||
"id": 120,
|
||||
"type": "ComfySwitchNode",
|
||||
"title": "Switch (CFG)",
|
||||
"pos": [100, 100],
|
||||
"size": [200, 80],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [{ "name": "value", "type": "FLOAT", "link": null }],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "FLOAT",
|
||||
"type": "FLOAT",
|
||||
"links": [257, 271, 276]
|
||||
}
|
||||
],
|
||||
"properties": { "Node name for S&R": "ComfySwitchNode" },
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 85,
|
||||
"type": "KSamplerAdvanced",
|
||||
"pos": [400, 50],
|
||||
"size": [270, 262],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{ "name": "model", "type": "MODEL", "link": null },
|
||||
{ "name": "positive", "type": "CONDITIONING", "link": null },
|
||||
{ "name": "negative", "type": "CONDITIONING", "link": null },
|
||||
{ "name": "latent_image", "type": "LATENT", "link": null },
|
||||
{ "name": "steps", "type": "INT", "link": null },
|
||||
{ "name": "cfg", "type": "FLOAT", "link": 276 }
|
||||
],
|
||||
"outputs": [{ "name": "LATENT", "type": "LATENT", "links": [] }],
|
||||
"properties": { "Node name for S&R": "KSamplerAdvanced" },
|
||||
"widgets_values": [
|
||||
false,
|
||||
0,
|
||||
"randomize",
|
||||
20,
|
||||
8,
|
||||
"euler",
|
||||
"normal",
|
||||
0,
|
||||
10000,
|
||||
false
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 86,
|
||||
"type": "KSamplerAdvanced",
|
||||
"pos": [400, 350],
|
||||
"size": [270, 262],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{ "name": "model", "type": "MODEL", "link": null },
|
||||
{ "name": "positive", "type": "CONDITIONING", "link": null },
|
||||
{ "name": "negative", "type": "CONDITIONING", "link": null },
|
||||
{ "name": "latent_image", "type": "LATENT", "link": null },
|
||||
{ "name": "steps", "type": "INT", "link": null },
|
||||
{ "name": "cfg", "type": "FLOAT", "link": 271 }
|
||||
],
|
||||
"outputs": [{ "name": "LATENT", "type": "LATENT", "links": [] }],
|
||||
"properties": { "Node name for S&R": "KSamplerAdvanced" },
|
||||
"widgets_values": [
|
||||
false,
|
||||
0,
|
||||
"randomize",
|
||||
20,
|
||||
8,
|
||||
"euler",
|
||||
"normal",
|
||||
0,
|
||||
10000,
|
||||
false
|
||||
]
|
||||
}
|
||||
],
|
||||
"groups": [],
|
||||
"links": [
|
||||
{
|
||||
"id": 257,
|
||||
"origin_id": 120,
|
||||
"origin_slot": 0,
|
||||
"target_id": 85,
|
||||
"target_slot": 5,
|
||||
"type": "FLOAT"
|
||||
},
|
||||
{
|
||||
"id": 271,
|
||||
"origin_id": 120,
|
||||
"origin_slot": 0,
|
||||
"target_id": 86,
|
||||
"target_slot": 5,
|
||||
"type": "FLOAT"
|
||||
},
|
||||
{
|
||||
"id": 276,
|
||||
"origin_id": 120,
|
||||
"origin_slot": 0,
|
||||
"target_id": 85,
|
||||
"target_slot": 5,
|
||||
"type": "FLOAT"
|
||||
}
|
||||
],
|
||||
"extra": {}
|
||||
}
|
||||
]
|
||||
},
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": { "scale": 1, "offset": [0, 0] },
|
||||
"frontendVersion": "1.43.2"
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
{
|
||||
"id": "9efdcc44-6372-4b4a-b6f9-789c67f052e1",
|
||||
"revision": 0,
|
||||
"last_node_id": 4,
|
||||
"last_link_id": 0,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 4,
|
||||
"type": "f5d6b5f0-64e3-4d3e-bb28-d25d8a6c182f",
|
||||
"pos": [689.0083557128902, 467.9999999999997],
|
||||
"size": [431.8999938964844, 206.60000610351562],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"proxyWidgets": [["3", "text", "2"]]
|
||||
},
|
||||
"widgets_values": []
|
||||
}
|
||||
],
|
||||
"links": [],
|
||||
"groups": [],
|
||||
"definitions": {
|
||||
"subgraphs": [
|
||||
{
|
||||
"id": "9a3f232c-da11-4725-8927-b11e46d0cee4",
|
||||
"version": 1,
|
||||
"state": {
|
||||
"lastGroupId": 0,
|
||||
"lastNodeId": 4,
|
||||
"lastLinkId": 0,
|
||||
"lastRerouteId": 0
|
||||
},
|
||||
"revision": 0,
|
||||
"config": {},
|
||||
"name": "Inner Subgraph",
|
||||
"inputNode": {
|
||||
"id": -10,
|
||||
"bounding": [330, 367, 120, 40]
|
||||
},
|
||||
"outputNode": {
|
||||
"id": -20,
|
||||
"bounding": [983, 367, 120, 40]
|
||||
},
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"widgets": [],
|
||||
"nodes": [
|
||||
{
|
||||
"id": 1,
|
||||
"type": "CLIPTextEncode",
|
||||
"pos": [510, 166],
|
||||
"size": [400, 200],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "CONDITIONING",
|
||||
"name": "CONDITIONING",
|
||||
"type": "CONDITIONING",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
},
|
||||
"widgets_values": ["11111111111"]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "CLIPTextEncode",
|
||||
"pos": [523, 438],
|
||||
"size": [400, 200],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "CONDITIONING",
|
||||
"name": "CONDITIONING",
|
||||
"type": "CONDITIONING",
|
||||
"links": null
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
},
|
||||
"widgets_values": ["22222222222"]
|
||||
}
|
||||
],
|
||||
"groups": [],
|
||||
"links": [],
|
||||
"extra": {}
|
||||
},
|
||||
{
|
||||
"id": "f5d6b5f0-64e3-4d3e-bb28-d25d8a6c182f",
|
||||
"version": 1,
|
||||
"state": {
|
||||
"lastGroupId": 0,
|
||||
"lastNodeId": 4,
|
||||
"lastLinkId": 0,
|
||||
"lastRerouteId": 0
|
||||
},
|
||||
"revision": 0,
|
||||
"config": {},
|
||||
"name": "Outer Subgraph",
|
||||
"inputNode": {
|
||||
"id": -10,
|
||||
"bounding": [467, 446, 120, 40]
|
||||
},
|
||||
"outputNode": {
|
||||
"id": -20,
|
||||
"bounding": [932, 446, 120, 40]
|
||||
},
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"widgets": [],
|
||||
"nodes": [
|
||||
{
|
||||
"id": 3,
|
||||
"type": "9a3f232c-da11-4725-8927-b11e46d0cee4",
|
||||
"pos": [647, 389],
|
||||
"size": [400, 200],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"proxyWidgets": [
|
||||
["1", "text"],
|
||||
["2", "text"]
|
||||
]
|
||||
},
|
||||
"widgets_values": []
|
||||
}
|
||||
],
|
||||
"groups": [],
|
||||
"links": [],
|
||||
"extra": {}
|
||||
}
|
||||
]
|
||||
},
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"scale": 2.0975,
|
||||
"offset": [-581.4780189305006, -356.3000030517576]
|
||||
},
|
||||
"frontendVersion": "1.43.2"
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
@@ -1,407 +0,0 @@
|
||||
{
|
||||
"id": "0cc04f4c-d744-462d-8638-4e5f5e3947e7",
|
||||
"revision": 0,
|
||||
"last_node_id": 19,
|
||||
"last_link_id": 24,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 14,
|
||||
"type": "CLIPLoader",
|
||||
"pos": [143.16716182216328, 290.16372862874033],
|
||||
"size": [270, 117.3125],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "CLIP",
|
||||
"type": "CLIP",
|
||||
"links": [21]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPLoader"
|
||||
},
|
||||
"widgets_values": [null, "stable_diffusion", "default"]
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"type": "PreviewImage",
|
||||
"pos": [1305.1455526601603, 472.17095792625025],
|
||||
"size": [225, 48],
|
||||
"flags": {},
|
||||
"order": 4,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "images",
|
||||
"type": "IMAGE",
|
||||
"link": 24
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"properties": {
|
||||
"Node name for S&R": "PreviewImage"
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 19,
|
||||
"type": "314bbb9f-f1cc-456c-b14f-2ba92bd4a597",
|
||||
"pos": [794.198171390827, 452.45433419677147],
|
||||
"size": [225, 172],
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"label": "renamed_clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": 21
|
||||
},
|
||||
{
|
||||
"label": "renamed_seed",
|
||||
"name": "seed",
|
||||
"type": "INT",
|
||||
"widget": {
|
||||
"name": "seed"
|
||||
},
|
||||
"link": 22
|
||||
},
|
||||
{
|
||||
"label": "renamed_vae",
|
||||
"name": "vae",
|
||||
"type": "VAE",
|
||||
"link": 23
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [24]
|
||||
}
|
||||
],
|
||||
"title": "Input Test Subgraph",
|
||||
"properties": {
|
||||
"proxyWidgets": [
|
||||
["12", "seed"],
|
||||
["15", "text"]
|
||||
]
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"type": "PrimitiveInt",
|
||||
"pos": [155.04048166054417, 773.3816055422594],
|
||||
"size": [270, 82],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "INT",
|
||||
"type": "INT",
|
||||
"links": [22]
|
||||
}
|
||||
],
|
||||
"title": "Seed Int",
|
||||
"properties": {
|
||||
"Node name for S&R": "PrimitiveInt"
|
||||
},
|
||||
"widgets_values": [0, "randomize"]
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"type": "VAELoader",
|
||||
"pos": [163.6043676075426, 543.9624492717659],
|
||||
"size": [270, 82.65625],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "VAE",
|
||||
"type": "VAE",
|
||||
"links": [23]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAELoader"
|
||||
},
|
||||
"widgets_values": ["pixel_space"]
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[21, 14, 0, 19, 0, "CLIP"],
|
||||
[22, 13, 0, 19, 1, "INT"],
|
||||
[23, 17, 0, 19, 2, "VAE"],
|
||||
[24, 19, 0, 18, 0, "IMAGE"]
|
||||
],
|
||||
"groups": [],
|
||||
"definitions": {
|
||||
"subgraphs": [
|
||||
{
|
||||
"id": "314bbb9f-f1cc-456c-b14f-2ba92bd4a597",
|
||||
"version": 1,
|
||||
"state": {
|
||||
"lastGroupId": 0,
|
||||
"lastNodeId": 19,
|
||||
"lastLinkId": 24,
|
||||
"lastRerouteId": 0
|
||||
},
|
||||
"revision": 0,
|
||||
"config": {},
|
||||
"name": "Input Test Subgraph",
|
||||
"inputNode": {
|
||||
"id": -10,
|
||||
"bounding": [
|
||||
358.8694807105848, 439.23932667242485, 123.14453125,
|
||||
99.99999999999994
|
||||
]
|
||||
},
|
||||
"outputNode": {
|
||||
"id": -20,
|
||||
"bounding": [1408.5510580294986, 463.2512895126797, 120, 60]
|
||||
},
|
||||
"inputs": [
|
||||
{
|
||||
"id": "cfaad2dc-7758-412c-a4ac-dc2e6d37b28c",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"linkIds": [16],
|
||||
"localized_name": "clip",
|
||||
"label": "renamed_clip",
|
||||
"pos": [462.0140119605848, 459.23932667242485]
|
||||
},
|
||||
{
|
||||
"id": "2e4600ea-e1b1-42ca-b43a-e066fd080774",
|
||||
"name": "seed",
|
||||
"type": "INT",
|
||||
"linkIds": [15],
|
||||
"localized_name": "seed",
|
||||
"label": "renamed_seed",
|
||||
"pos": [462.0140119605848, 479.23932667242485]
|
||||
},
|
||||
{
|
||||
"id": "86ed2da7-db02-454a-9362-70a3fa3e91bf",
|
||||
"name": "vae",
|
||||
"type": "VAE",
|
||||
"linkIds": [19],
|
||||
"localized_name": "vae",
|
||||
"label": "renamed_vae",
|
||||
"pos": [462.0140119605848, 499.23932667242485]
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"id": "8670d1a7-0d44-4688-b7dd-d4b423f1aee0",
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"linkIds": [20],
|
||||
"localized_name": "IMAGE",
|
||||
"pos": [1428.5510580294986, 483.2512895126797]
|
||||
}
|
||||
],
|
||||
"widgets": [],
|
||||
"nodes": [
|
||||
{
|
||||
"id": 12,
|
||||
"type": "KSampler",
|
||||
"pos": [769.2424728654022, 512.726159169824],
|
||||
"size": [270, 262],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "model",
|
||||
"name": "model",
|
||||
"type": "MODEL",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "positive",
|
||||
"name": "positive",
|
||||
"type": "CONDITIONING",
|
||||
"link": 17
|
||||
},
|
||||
{
|
||||
"localized_name": "negative",
|
||||
"name": "negative",
|
||||
"type": "CONDITIONING",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "latent_image",
|
||||
"name": "latent_image",
|
||||
"type": "LATENT",
|
||||
"link": null
|
||||
},
|
||||
{
|
||||
"localized_name": "seed",
|
||||
"name": "seed",
|
||||
"type": "INT",
|
||||
"widget": {
|
||||
"name": "seed"
|
||||
},
|
||||
"link": 15
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "LATENT",
|
||||
"name": "LATENT",
|
||||
"type": "LATENT",
|
||||
"links": [18]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "KSampler"
|
||||
},
|
||||
"widgets_values": [0, "randomize", 20, 8, "euler", "simple", 1]
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"type": "VAEDecode",
|
||||
"pos": [1208.5510580294986, 469.21581253470083],
|
||||
"size": [140, 46],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "samples",
|
||||
"name": "samples",
|
||||
"type": "LATENT",
|
||||
"link": 18
|
||||
},
|
||||
{
|
||||
"localized_name": "vae",
|
||||
"name": "vae",
|
||||
"type": "VAE",
|
||||
"link": 19
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "IMAGE",
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [20]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "VAEDecode"
|
||||
},
|
||||
"widgets_values": []
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"type": "CLIPTextEncode",
|
||||
"pos": [681.4596332342014, 243.17567172890932],
|
||||
"size": [400, 200],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"localized_name": "clip",
|
||||
"name": "clip",
|
||||
"type": "CLIP",
|
||||
"link": 16
|
||||
},
|
||||
{
|
||||
"label": "renamed_from_sidepanel",
|
||||
"localized_name": "text",
|
||||
"name": "text",
|
||||
"type": "STRING",
|
||||
"widget": {
|
||||
"name": "text"
|
||||
},
|
||||
"link": null
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"localized_name": "CONDITIONING",
|
||||
"name": "CONDITIONING",
|
||||
"type": "CONDITIONING",
|
||||
"links": [17]
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "CLIPTextEncode"
|
||||
},
|
||||
"widgets_values": [""]
|
||||
}
|
||||
],
|
||||
"groups": [],
|
||||
"links": [
|
||||
{
|
||||
"id": 17,
|
||||
"origin_id": 15,
|
||||
"origin_slot": 0,
|
||||
"target_id": 12,
|
||||
"target_slot": 1,
|
||||
"type": "CONDITIONING"
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"origin_id": 12,
|
||||
"origin_slot": 0,
|
||||
"target_id": 16,
|
||||
"target_slot": 0,
|
||||
"type": "LATENT"
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"origin_id": -10,
|
||||
"origin_slot": 0,
|
||||
"target_id": 15,
|
||||
"target_slot": 0,
|
||||
"type": "CLIP"
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"origin_id": -10,
|
||||
"origin_slot": 1,
|
||||
"target_id": 12,
|
||||
"target_slot": 4,
|
||||
"type": "INT"
|
||||
},
|
||||
{
|
||||
"id": 19,
|
||||
"origin_id": -10,
|
||||
"origin_slot": 2,
|
||||
"target_id": 16,
|
||||
"target_slot": 1,
|
||||
"type": "VAE"
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"origin_id": 16,
|
||||
"origin_slot": 0,
|
||||
"target_id": -20,
|
||||
"target_slot": 0,
|
||||
"type": "IMAGE"
|
||||
}
|
||||
],
|
||||
"extra": {}
|
||||
}
|
||||
]
|
||||
},
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ds": {
|
||||
"scale": 0.6727925600199565,
|
||||
"offset": [446.69747171876463, 99.95078257277316]
|
||||
}
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
@@ -33,7 +33,6 @@ import { FeatureFlagHelper } from './helpers/FeatureFlagHelper'
|
||||
import { KeyboardHelper } from './helpers/KeyboardHelper'
|
||||
import { NodeOperationsHelper } from './helpers/NodeOperationsHelper'
|
||||
import { SettingsHelper } from './helpers/SettingsHelper'
|
||||
import { AppModeHelper } from './helpers/AppModeHelper'
|
||||
import { SubgraphHelper } from './helpers/SubgraphHelper'
|
||||
import { ToastHelper } from './helpers/ToastHelper'
|
||||
import { WorkflowHelper } from './helpers/WorkflowHelper'
|
||||
@@ -177,7 +176,6 @@ export class ComfyPage {
|
||||
public readonly settingDialog: SettingDialog
|
||||
public readonly confirmDialog: ConfirmDialog
|
||||
public readonly vueNodes: VueNodeHelpers
|
||||
public readonly appMode: AppModeHelper
|
||||
public readonly subgraph: SubgraphHelper
|
||||
public readonly canvasOps: CanvasHelper
|
||||
public readonly nodeOps: NodeOperationsHelper
|
||||
@@ -223,13 +221,12 @@ export class ComfyPage {
|
||||
this.settingDialog = new SettingDialog(page, this)
|
||||
this.confirmDialog = new ConfirmDialog(page)
|
||||
this.vueNodes = new VueNodeHelpers(page)
|
||||
this.appMode = new AppModeHelper(this)
|
||||
this.subgraph = new SubgraphHelper(this)
|
||||
this.canvasOps = new CanvasHelper(page, this.canvas, this.resetViewButton)
|
||||
this.nodeOps = new NodeOperationsHelper(this)
|
||||
this.settings = new SettingsHelper(page)
|
||||
this.keyboard = new KeyboardHelper(page, this.canvas)
|
||||
this.clipboard = new ClipboardHelper(this.keyboard, page)
|
||||
this.clipboard = new ClipboardHelper(this.keyboard)
|
||||
this.workflow = new WorkflowHelper(this)
|
||||
this.contextMenu = new ContextMenu(page)
|
||||
this.toast = new ToastHelper(page)
|
||||
@@ -290,7 +287,9 @@ export class ComfyPage {
|
||||
clearStorage?: boolean
|
||||
mockReleases?: boolean
|
||||
} = {}) {
|
||||
// Mock release endpoint to prevent changelog popups (before navigation)
|
||||
await this.goto()
|
||||
|
||||
// Mock release endpoint to prevent changelog popups
|
||||
if (mockReleases) {
|
||||
await this.page.route('**/releases**', async (route) => {
|
||||
const url = route.request().url()
|
||||
@@ -310,16 +309,12 @@ export class ComfyPage {
|
||||
}
|
||||
|
||||
if (clearStorage) {
|
||||
// Navigate to a lightweight same-origin endpoint to obtain a page
|
||||
// context for clearing storage without loading the full frontend app.
|
||||
await this.page.goto(`${this.url}/api/users`)
|
||||
await this.page.evaluate((id) => {
|
||||
localStorage.clear()
|
||||
sessionStorage.clear()
|
||||
localStorage.setItem('Comfy.userId', id)
|
||||
}, this.id)
|
||||
}
|
||||
|
||||
await this.goto()
|
||||
|
||||
await this.page.waitForFunction(() => document.fonts.ready)
|
||||
|
||||
@@ -1,201 +0,0 @@
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
|
||||
import type { ComfyPage } from '../ComfyPage'
|
||||
import { TestIds } from '../selectors'
|
||||
|
||||
export class AppModeHelper {
|
||||
constructor(private readonly comfyPage: ComfyPage) {}
|
||||
|
||||
private get page(): Page {
|
||||
return this.comfyPage.page
|
||||
}
|
||||
|
||||
private get builderToolbar(): Locator {
|
||||
return this.page.getByRole('navigation', { name: 'App Builder' })
|
||||
}
|
||||
|
||||
/** Enter builder mode via the "Workflow actions" dropdown → "Build app". */
|
||||
async enterBuilder() {
|
||||
await this.page
|
||||
.getByRole('button', { name: 'Workflow actions' })
|
||||
.first()
|
||||
.click()
|
||||
await this.page.getByRole('menuitem', { name: 'Build app' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Exit builder mode via the footer "Exit app builder" button. */
|
||||
async exitBuilder() {
|
||||
await this.page.getByRole('button', { name: 'Exit app builder' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Click the "Inputs" step in the builder toolbar. */
|
||||
async goToInputs() {
|
||||
await this.builderToolbar.getByRole('button', { name: 'Inputs' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Click the "Outputs" step in the builder toolbar. */
|
||||
async goToOutputs() {
|
||||
await this.builderToolbar.getByRole('button', { name: 'Outputs' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Click the "Preview" step in the builder toolbar. */
|
||||
async goToPreview() {
|
||||
await this.builderToolbar.getByRole('button', { name: 'Preview' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Click the "Next" button in the builder footer. */
|
||||
async next() {
|
||||
await this.page.getByRole('button', { name: 'Next' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Click the "Back" button in the builder footer. */
|
||||
async back() {
|
||||
await this.page.getByRole('button', { name: 'Back' }).click()
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/** Toggle app mode (linear view) on/off. */
|
||||
async toggleAppMode() {
|
||||
await this.page.evaluate(() => {
|
||||
window.app!.extensionManager.command.execute('Comfy.ToggleLinear')
|
||||
})
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/**
|
||||
* Inject linearData into the current graph and enter app mode.
|
||||
*
|
||||
* Serializes the graph, injects linearData with the given inputs and
|
||||
* auto-detected output node IDs, then reloads so the appModeStore
|
||||
* picks up the data via its activeWorkflow watcher.
|
||||
*
|
||||
* @param inputs - Widget selections as [nodeId, widgetName] tuples
|
||||
*/
|
||||
async enterAppModeWithInputs(inputs: [string, string][]) {
|
||||
await this.page.evaluate(async (inputTuples) => {
|
||||
const graph = window.app!.graph
|
||||
if (!graph) return
|
||||
|
||||
const outputNodeIds = graph.nodes
|
||||
.filter(
|
||||
(n: { type?: string }) =>
|
||||
n.type === 'SaveImage' || n.type === 'PreviewImage'
|
||||
)
|
||||
.map((n: { id: number | string }) => String(n.id))
|
||||
|
||||
const workflow = graph.serialize() as unknown as Record<string, unknown>
|
||||
const extra = (workflow.extra ?? {}) as Record<string, unknown>
|
||||
extra.linearData = { inputs: inputTuples, outputs: outputNodeIds }
|
||||
workflow.extra = extra
|
||||
await window.app!.loadGraphData(
|
||||
workflow as unknown as Parameters<
|
||||
NonNullable<typeof window.app>['loadGraphData']
|
||||
>[0]
|
||||
)
|
||||
}, inputs)
|
||||
await this.comfyPage.nextFrame()
|
||||
await this.toggleAppMode()
|
||||
}
|
||||
|
||||
/** The linear-mode widget list container (visible in app mode). */
|
||||
get linearWidgets(): Locator {
|
||||
return this.page.locator('[data-testid="linear-widgets"]')
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the actions menu trigger for a widget in the app mode widget list.
|
||||
* @param widgetName Text shown in the widget label (e.g. "seed").
|
||||
*/
|
||||
getAppModeWidgetMenu(widgetName: string): Locator {
|
||||
return this.linearWidgets
|
||||
.locator(`div:has(> div > span:text-is("${widgetName}"))`)
|
||||
.getByTestId(TestIds.builder.widgetActionsMenu)
|
||||
.first()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the actions menu trigger for a widget in the builder input-select
|
||||
* sidebar (IoItem).
|
||||
* @param title The widget title shown in the IoItem.
|
||||
*/
|
||||
getBuilderInputItemMenu(title: string): Locator {
|
||||
return this.page
|
||||
.getByTestId(TestIds.builder.ioItem)
|
||||
.filter({ hasText: title })
|
||||
.getByTestId(TestIds.builder.widgetActionsMenu)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the actions menu trigger for a widget in the builder preview/arrange
|
||||
* sidebar (AppModeWidgetList with builderMode).
|
||||
* @param ariaLabel The aria-label on the widget row, e.g. "seed — KSampler".
|
||||
*/
|
||||
getBuilderPreviewWidgetMenu(ariaLabel: string): Locator {
|
||||
return this.page
|
||||
.locator(`[aria-label="${ariaLabel}"]`)
|
||||
.getByTestId(TestIds.builder.widgetActionsMenu)
|
||||
}
|
||||
|
||||
/**
|
||||
* Rename a widget by clicking its popover trigger, selecting "Rename",
|
||||
* and filling in the dialog.
|
||||
* @param popoverTrigger The button that opens the widget's actions popover.
|
||||
* @param newName The new name to assign.
|
||||
*/
|
||||
async renameWidget(popoverTrigger: Locator, newName: string) {
|
||||
await popoverTrigger.click()
|
||||
await this.page.getByText('Rename', { exact: true }).click()
|
||||
|
||||
const dialogInput = this.page.locator(
|
||||
'.p-dialog-content input[type="text"]'
|
||||
)
|
||||
await dialogInput.fill(newName)
|
||||
await this.page.keyboard.press('Enter')
|
||||
await dialogInput.waitFor({ state: 'hidden' })
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/**
|
||||
* Rename a builder IoItem via the popover menu "Rename" action.
|
||||
* @param title The current widget title shown in the IoItem.
|
||||
* @param newName The new name to assign.
|
||||
*/
|
||||
async renameBuilderInputViaMenu(title: string, newName: string) {
|
||||
const menu = this.getBuilderInputItemMenu(title)
|
||||
await menu.click()
|
||||
await this.page.getByText('Rename', { exact: true }).click()
|
||||
|
||||
const input = this.page
|
||||
.getByTestId(TestIds.builder.ioItemTitle)
|
||||
.getByRole('textbox')
|
||||
await input.fill(newName)
|
||||
await this.page.keyboard.press('Enter')
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
/**
|
||||
* Rename a builder IoItem by double-clicking its title to trigger
|
||||
* inline editing.
|
||||
* @param title The current widget title shown in the IoItem.
|
||||
* @param newName The new name to assign.
|
||||
*/
|
||||
async renameBuilderInput(title: string, newName: string) {
|
||||
const titleEl = this.page
|
||||
.getByTestId(TestIds.builder.ioItemTitle)
|
||||
.filter({ hasText: title })
|
||||
await titleEl.dblclick()
|
||||
|
||||
const input = this.page
|
||||
.getByTestId(TestIds.builder.ioItemTitle)
|
||||
.getByRole('textbox')
|
||||
await input.fill(newName)
|
||||
await this.page.keyboard.press('Enter')
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
}
|
||||
@@ -91,12 +91,6 @@ export class CanvasHelper {
|
||||
await this.page.mouse.move(10, 10)
|
||||
}
|
||||
|
||||
async isReadOnly(): Promise<boolean> {
|
||||
return this.page.evaluate(() => {
|
||||
return window.app!.canvas.state.readOnly
|
||||
})
|
||||
}
|
||||
|
||||
async getScale(): Promise<number> {
|
||||
return this.page.evaluate(() => {
|
||||
return window.app!.canvas.ds.scale
|
||||
|
||||
@@ -1,16 +1,9 @@
|
||||
import { readFileSync } from 'fs'
|
||||
import { basename } from 'path'
|
||||
|
||||
import type { Locator, Page } from '@playwright/test'
|
||||
import type { Locator } from '@playwright/test'
|
||||
|
||||
import type { KeyboardHelper } from './KeyboardHelper'
|
||||
import { getMimeType } from './mimeTypeUtil'
|
||||
|
||||
export class ClipboardHelper {
|
||||
constructor(
|
||||
private readonly keyboard: KeyboardHelper,
|
||||
private readonly page: Page
|
||||
) {}
|
||||
constructor(private readonly keyboard: KeyboardHelper) {}
|
||||
|
||||
async copy(locator?: Locator | null): Promise<void> {
|
||||
await this.keyboard.ctrlSend('KeyC', locator ?? null)
|
||||
@@ -19,44 +12,4 @@ export class ClipboardHelper {
|
||||
async paste(locator?: Locator | null): Promise<void> {
|
||||
await this.keyboard.ctrlSend('KeyV', locator ?? null)
|
||||
}
|
||||
|
||||
async pasteFile(filePath: string): Promise<void> {
|
||||
const buffer = readFileSync(filePath)
|
||||
const bufferArray = [...new Uint8Array(buffer)]
|
||||
const fileName = basename(filePath)
|
||||
const fileType = getMimeType(fileName)
|
||||
|
||||
// Register a one-time capturing-phase listener that intercepts the next
|
||||
// paste event and injects file data onto clipboardData.
|
||||
await this.page.evaluate(
|
||||
({ bufferArray, fileName, fileType }) => {
|
||||
document.addEventListener(
|
||||
'paste',
|
||||
(e: ClipboardEvent) => {
|
||||
e.preventDefault()
|
||||
e.stopImmediatePropagation()
|
||||
|
||||
const file = new File([new Uint8Array(bufferArray)], fileName, {
|
||||
type: fileType
|
||||
})
|
||||
const dataTransfer = new DataTransfer()
|
||||
dataTransfer.items.add(file)
|
||||
|
||||
const syntheticEvent = new ClipboardEvent('paste', {
|
||||
clipboardData: dataTransfer,
|
||||
bubbles: true,
|
||||
cancelable: true
|
||||
})
|
||||
document.dispatchEvent(syntheticEvent)
|
||||
},
|
||||
{ capture: true, once: true }
|
||||
)
|
||||
},
|
||||
{ bufferArray, fileName, fileType }
|
||||
)
|
||||
|
||||
// Trigger a real Ctrl+V keystroke — the capturing listener above will
|
||||
// intercept it and re-dispatch with file data attached.
|
||||
await this.paste()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,7 +3,6 @@ import { readFileSync } from 'fs'
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import type { Position } from '../types'
|
||||
import { getMimeType } from './mimeTypeUtil'
|
||||
|
||||
export class DragDropHelper {
|
||||
constructor(
|
||||
@@ -49,8 +48,19 @@ export class DragDropHelper {
|
||||
const filePath = this.assetPath(fileName)
|
||||
const buffer = readFileSync(filePath)
|
||||
|
||||
const getFileType = (fileName: string) => {
|
||||
if (fileName.endsWith('.png')) return 'image/png'
|
||||
if (fileName.endsWith('.svg')) return 'image/svg+xml'
|
||||
if (fileName.endsWith('.webp')) return 'image/webp'
|
||||
if (fileName.endsWith('.webm')) return 'video/webm'
|
||||
if (fileName.endsWith('.json')) return 'application/json'
|
||||
if (fileName.endsWith('.glb')) return 'model/gltf-binary'
|
||||
if (fileName.endsWith('.avif')) return 'image/avif'
|
||||
return 'application/octet-stream'
|
||||
}
|
||||
|
||||
evaluateParams.fileName = fileName
|
||||
evaluateParams.fileType = getMimeType(fileName)
|
||||
evaluateParams.fileType = getFileType(fileName)
|
||||
evaluateParams.buffer = [...new Uint8Array(buffer)]
|
||||
}
|
||||
|
||||
|
||||
@@ -33,7 +33,6 @@ export class NodeOperationsHelper {
|
||||
})
|
||||
}
|
||||
|
||||
/** Reads from `window.app.graph` (the root workflow graph). */
|
||||
async getNodeCount(): Promise<number> {
|
||||
return await this.page.evaluate(() => window.app!.graph.nodes.length)
|
||||
}
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { expect } from '@playwright/test'
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import type {
|
||||
@@ -7,7 +6,6 @@ import type {
|
||||
} from '@/lib/litegraph/src/litegraph'
|
||||
|
||||
import type { ComfyPage } from '../ComfyPage'
|
||||
import { TestIds } from '../selectors'
|
||||
import type { NodeReference } from '../utils/litegraphUtils'
|
||||
import { SubgraphSlotReference } from '../utils/litegraphUtils'
|
||||
|
||||
@@ -324,93 +322,4 @@ export class SubgraphHelper {
|
||||
)
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
async isInSubgraph(): Promise<boolean> {
|
||||
return this.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
return !!graph && 'inputNode' in graph
|
||||
})
|
||||
}
|
||||
|
||||
async exitViaBreadcrumb(): Promise<void> {
|
||||
const breadcrumb = this.page.getByTestId(TestIds.breadcrumb.subgraph)
|
||||
const parentLink = breadcrumb.getByRole('link').first()
|
||||
if (await parentLink.isVisible()) {
|
||||
await parentLink.click()
|
||||
} else {
|
||||
await this.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
const graph = canvas.graph
|
||||
if (!graph) return
|
||||
canvas.setGraph(graph.rootGraph)
|
||||
})
|
||||
}
|
||||
|
||||
await this.comfyPage.nextFrame()
|
||||
await expect.poll(async () => this.isInSubgraph()).toBe(false)
|
||||
}
|
||||
|
||||
async countGraphPseudoPreviewEntries(): Promise<number> {
|
||||
return this.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
return graph.nodes.reduce((count, node) => {
|
||||
const proxyWidgets = node.properties?.proxyWidgets
|
||||
if (!Array.isArray(proxyWidgets)) return count
|
||||
|
||||
return (
|
||||
count +
|
||||
proxyWidgets.filter(
|
||||
(entry) =>
|
||||
Array.isArray(entry) &&
|
||||
entry.length >= 2 &&
|
||||
typeof entry[1] === 'string' &&
|
||||
entry[1].startsWith('$$')
|
||||
).length
|
||||
)
|
||||
}, 0)
|
||||
})
|
||||
}
|
||||
|
||||
async getHostPromotedTupleSnapshot(): Promise<
|
||||
{ hostNodeId: string; promotedWidgets: [string, string][] }[]
|
||||
> {
|
||||
return this.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
return graph._nodes
|
||||
.filter(
|
||||
(node) =>
|
||||
typeof node.isSubgraphNode === 'function' && node.isSubgraphNode()
|
||||
)
|
||||
.map((node) => {
|
||||
const proxyWidgets = Array.isArray(node.properties?.proxyWidgets)
|
||||
? node.properties.proxyWidgets
|
||||
: []
|
||||
const promotedWidgets = proxyWidgets
|
||||
.filter(
|
||||
(entry): entry is [string, string] =>
|
||||
Array.isArray(entry) &&
|
||||
entry.length >= 2 &&
|
||||
typeof entry[0] === 'string' &&
|
||||
typeof entry[1] === 'string'
|
||||
)
|
||||
.map(
|
||||
([interiorNodeId, widgetName]) =>
|
||||
[interiorNodeId, widgetName] as [string, string]
|
||||
)
|
||||
|
||||
return {
|
||||
hostNodeId: String(node.id),
|
||||
promotedWidgets
|
||||
}
|
||||
})
|
||||
.sort((a, b) => Number(a.hostNodeId) - Number(b.hostNodeId))
|
||||
})
|
||||
}
|
||||
|
||||
/** Reads from `window.app.canvas.graph` (viewed root or nested subgraph). */
|
||||
async getNodeCount(): Promise<number> {
|
||||
return this.page.evaluate(() => {
|
||||
return window.app!.canvas.graph!.nodes?.length || 0
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
export function getMimeType(fileName: string): string {
|
||||
const name = fileName.toLowerCase()
|
||||
if (name.endsWith('.png')) return 'image/png'
|
||||
if (name.endsWith('.jpg') || name.endsWith('.jpeg')) return 'image/jpeg'
|
||||
if (name.endsWith('.webp')) return 'image/webp'
|
||||
if (name.endsWith('.svg')) return 'image/svg+xml'
|
||||
if (name.endsWith('.avif')) return 'image/avif'
|
||||
if (name.endsWith('.webm')) return 'video/webm'
|
||||
if (name.endsWith('.mp4')) return 'video/mp4'
|
||||
if (name.endsWith('.json')) return 'application/json'
|
||||
if (name.endsWith('.glb')) return 'model/gltf-binary'
|
||||
return 'application/octet-stream'
|
||||
}
|
||||
@@ -28,18 +28,9 @@ export const TestIds = {
|
||||
settingsTabAbout: 'settings-tab-about',
|
||||
confirm: 'confirm-dialog',
|
||||
errorOverlay: 'error-overlay',
|
||||
errorOverlaySeeErrors: 'error-overlay-see-errors',
|
||||
runtimeErrorPanel: 'runtime-error-panel',
|
||||
missingNodeCard: 'missing-node-card',
|
||||
errorCardFindOnGithub: 'error-card-find-on-github',
|
||||
errorCardCopy: 'error-card-copy',
|
||||
about: 'about-panel',
|
||||
whatsNewSection: 'whats-new-section',
|
||||
missingNodePacksGroup: 'error-group-missing-node',
|
||||
missingModelsGroup: 'error-group-missing-model'
|
||||
},
|
||||
keybindings: {
|
||||
presetMenu: 'keybinding-preset-menu'
|
||||
whatsNewSection: 'whats-new-section'
|
||||
},
|
||||
topbar: {
|
||||
queueButton: 'queue-button',
|
||||
@@ -67,11 +58,6 @@ export const TestIds = {
|
||||
domWidgetTextarea: 'dom-widget-textarea',
|
||||
subgraphEnterButton: 'subgraph-enter-button'
|
||||
},
|
||||
builder: {
|
||||
ioItem: 'builder-io-item',
|
||||
ioItemTitle: 'builder-io-item-title',
|
||||
widgetActionsMenu: 'widget-actions-menu'
|
||||
},
|
||||
breadcrumb: {
|
||||
subgraph: 'subgraph-breadcrumb'
|
||||
},
|
||||
@@ -81,10 +67,6 @@ export const TestIds = {
|
||||
},
|
||||
user: {
|
||||
currentUserIndicator: 'current-user-indicator'
|
||||
},
|
||||
errors: {
|
||||
imageLoadError: 'error-loading-image',
|
||||
videoLoadError: 'error-loading-video'
|
||||
}
|
||||
} as const
|
||||
|
||||
@@ -96,18 +78,15 @@ export type TestIdValue =
|
||||
| (typeof TestIds.tree)[keyof typeof TestIds.tree]
|
||||
| (typeof TestIds.canvas)[keyof typeof TestIds.canvas]
|
||||
| (typeof TestIds.dialogs)[keyof typeof TestIds.dialogs]
|
||||
| (typeof TestIds.keybindings)[keyof typeof TestIds.keybindings]
|
||||
| (typeof TestIds.topbar)[keyof typeof TestIds.topbar]
|
||||
| (typeof TestIds.nodeLibrary)[keyof typeof TestIds.nodeLibrary]
|
||||
| (typeof TestIds.propertiesPanel)[keyof typeof TestIds.propertiesPanel]
|
||||
| (typeof TestIds.node)[keyof typeof TestIds.node]
|
||||
| (typeof TestIds.selectionToolbox)[keyof typeof TestIds.selectionToolbox]
|
||||
| (typeof TestIds.widgets)[keyof typeof TestIds.widgets]
|
||||
| (typeof TestIds.builder)[keyof typeof TestIds.builder]
|
||||
| (typeof TestIds.breadcrumb)[keyof typeof TestIds.breadcrumb]
|
||||
| Exclude<
|
||||
(typeof TestIds.templates)[keyof typeof TestIds.templates],
|
||||
(id: string) => string
|
||||
>
|
||||
| (typeof TestIds.user)[keyof typeof TestIds.user]
|
||||
| (typeof TestIds.errors)[keyof typeof TestIds.errors]
|
||||
|
||||
@@ -281,14 +281,6 @@ export class NodeReference {
|
||||
getType(): Promise<string> {
|
||||
return this.getProperty('type')
|
||||
}
|
||||
async centerOnNode(): Promise<void> {
|
||||
await this.comfyPage.page.evaluate((id) => {
|
||||
const node = window.app!.canvas.graph!.getNodeById(id)
|
||||
if (!node) throw new Error(`Node ${id} not found`)
|
||||
window.app!.canvas.centerOnNode(node)
|
||||
}, this.id)
|
||||
await this.comfyPage.nextFrame()
|
||||
}
|
||||
async getPosition(): Promise<Position> {
|
||||
const pos = await this.comfyPage.canvasOps.convertOffsetToCanvas(
|
||||
await this.getProperty<[number, number]>('pos')
|
||||
|
||||
@@ -29,8 +29,7 @@ export const webSocketFixture = base.extend<{
|
||||
function ([data, url]) {
|
||||
if (!url) {
|
||||
// If no URL specified, use page URL
|
||||
const u = new URL(window.location.href)
|
||||
u.hash = ''
|
||||
const u = new URL(window.location.toString())
|
||||
u.protocol = 'ws:'
|
||||
u.pathname = '/'
|
||||
url = u.toString() + 'ws'
|
||||
|
||||
@@ -75,26 +75,6 @@ export async function getPromotedWidgetCount(
|
||||
return promotedWidgets.length
|
||||
}
|
||||
|
||||
export function isPseudoPreviewEntry(entry: PromotedWidgetEntry): boolean {
|
||||
return entry[1].startsWith('$$')
|
||||
}
|
||||
|
||||
export async function getPseudoPreviewWidgets(
|
||||
comfyPage: ComfyPage,
|
||||
nodeId: string
|
||||
): Promise<PromotedWidgetEntry[]> {
|
||||
const widgets = await getPromotedWidgets(comfyPage, nodeId)
|
||||
return widgets.filter(isPseudoPreviewEntry)
|
||||
}
|
||||
|
||||
export async function getNonPreviewPromotedWidgets(
|
||||
comfyPage: ComfyPage,
|
||||
nodeId: string
|
||||
): Promise<PromotedWidgetEntry[]> {
|
||||
const widgets = await getPromotedWidgets(comfyPage, nodeId)
|
||||
return widgets.filter((entry) => !isPseudoPreviewEntry(entry))
|
||||
}
|
||||
|
||||
export async function getPromotedWidgetCountByName(
|
||||
comfyPage: ComfyPage,
|
||||
nodeId: string,
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import type { LGraph, Subgraph } from '../../src/lib/litegraph/src/litegraph'
|
||||
import { isSubgraph } from '../../src/utils/typeGuardUtil'
|
||||
|
||||
@@ -16,30 +14,3 @@ export function assertSubgraph(
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the widget-input slot Y position and the node title height
|
||||
* for the promoted "text" input on the SubgraphNode.
|
||||
*
|
||||
* The slot Y should be at the widget row, not the header. A value near
|
||||
* zero or negative indicates the slot is positioned at the header (the bug).
|
||||
*/
|
||||
export function getTextSlotPosition(page: Page, nodeId: string) {
|
||||
return page.evaluate((id) => {
|
||||
const node = window.app!.canvas.graph!.getNodeById(id)
|
||||
if (!node) return null
|
||||
|
||||
const titleHeight = window.LiteGraph!.NODE_TITLE_HEIGHT
|
||||
|
||||
for (const input of node.inputs) {
|
||||
if (!input.widget || input.type !== 'STRING') continue
|
||||
return {
|
||||
hasPos: !!input.pos,
|
||||
posY: input.pos?.[1] ?? null,
|
||||
widgetName: input.widget.name,
|
||||
titleHeight
|
||||
}
|
||||
}
|
||||
return null
|
||||
}, nodeId)
|
||||
}
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
/**
|
||||
* Default workflow widget inputs as [nodeId, widgetName] tuples.
|
||||
* All widgets from the default graph are selected so the panel scrolls,
|
||||
* pushing the last widget's dropdown to the clipping boundary.
|
||||
*/
|
||||
const DEFAULT_INPUTS: [string, string][] = [
|
||||
['4', 'ckpt_name'],
|
||||
['6', 'text'],
|
||||
['7', 'text'],
|
||||
['5', 'width'],
|
||||
['5', 'height'],
|
||||
['5', 'batch_size'],
|
||||
['3', 'seed'],
|
||||
['3', 'steps'],
|
||||
['3', 'cfg'],
|
||||
['3', 'sampler_name'],
|
||||
['3', 'scheduler'],
|
||||
['3', 'denoise'],
|
||||
['9', 'filename_prefix']
|
||||
]
|
||||
|
||||
function isClippedByAnyAncestor(el: Element): boolean {
|
||||
const child = el.getBoundingClientRect()
|
||||
let parent = el.parentElement
|
||||
|
||||
while (parent) {
|
||||
const overflow = getComputedStyle(parent).overflow
|
||||
if (overflow !== 'visible') {
|
||||
const p = parent.getBoundingClientRect()
|
||||
if (
|
||||
child.top < p.top ||
|
||||
child.bottom > p.bottom ||
|
||||
child.left < p.left ||
|
||||
child.right > p.right
|
||||
) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
parent = parent.parentElement
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/** Add a node to the graph by type and return its ID. */
|
||||
async function addNode(page: Page, nodeType: string): Promise<string> {
|
||||
return page.evaluate((type) => {
|
||||
const node = window.app!.graph.add(
|
||||
window.LiteGraph!.createNode(type, undefined, {})
|
||||
)
|
||||
return String(node!.id)
|
||||
}, nodeType)
|
||||
}
|
||||
|
||||
test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.api.serverFeatureFlags.value = {
|
||||
...window.app!.api.serverFeatureFlags.value,
|
||||
linear_toggle_enabled: true
|
||||
}
|
||||
})
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
})
|
||||
|
||||
test('Select dropdown is not clipped in app mode panel', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const saveVideoId = await addNode(comfyPage.page, 'SaveVideo')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const inputs: [string, string][] = [
|
||||
...DEFAULT_INPUTS,
|
||||
[saveVideoId, 'codec']
|
||||
]
|
||||
await comfyPage.appMode.enterAppModeWithInputs(inputs)
|
||||
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Scroll to bottom so the codec widget is at the clipping edge
|
||||
const widgetList = comfyPage.appMode.linearWidgets
|
||||
await widgetList.evaluate((el) =>
|
||||
el.scrollTo({ top: el.scrollHeight, behavior: 'instant' })
|
||||
)
|
||||
|
||||
// Click the codec select (combobox role with aria-label from WidgetSelectDefault)
|
||||
const codecSelect = widgetList.getByRole('combobox', { name: 'codec' })
|
||||
await codecSelect.click()
|
||||
|
||||
const overlay = comfyPage.page.locator('.p-select-overlay').first()
|
||||
await expect(overlay).toBeVisible({ timeout: 5000 })
|
||||
|
||||
const isInViewport = await overlay.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
})
|
||||
expect(isInViewport).toBe(true)
|
||||
|
||||
const isClipped = await overlay.evaluate(isClippedByAnyAncestor)
|
||||
expect(isClipped).toBe(false)
|
||||
})
|
||||
|
||||
test('FormDropdown popup is not clipped in app mode panel', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const loadImageId = await addNode(comfyPage.page, 'LoadImage')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const inputs: [string, string][] = [
|
||||
...DEFAULT_INPUTS,
|
||||
[loadImageId, 'image']
|
||||
]
|
||||
await comfyPage.appMode.enterAppModeWithInputs(inputs)
|
||||
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Scroll to bottom so the image widget is at the clipping edge
|
||||
const widgetList = comfyPage.appMode.linearWidgets
|
||||
await widgetList.evaluate((el) =>
|
||||
el.scrollTo({ top: el.scrollHeight, behavior: 'instant' })
|
||||
)
|
||||
|
||||
// Click the FormDropdown trigger button for the image widget.
|
||||
// The button emits 'select-click' which toggles the Popover.
|
||||
const imageRow = widgetList.locator(
|
||||
'div:has(> div > span:text-is("image"))'
|
||||
)
|
||||
const dropdownButton = imageRow.locator('button:has(> span)').first()
|
||||
await dropdownButton.click()
|
||||
|
||||
// The unstyled PrimeVue Popover renders with role="dialog".
|
||||
// Locate the one containing the image grid (filter buttons like "All", "Inputs").
|
||||
const popover = comfyPage.page
|
||||
.getByRole('dialog')
|
||||
.filter({ has: comfyPage.page.getByRole('button', { name: 'All' }) })
|
||||
.first()
|
||||
await expect(popover).toBeVisible({ timeout: 5000 })
|
||||
|
||||
const isInViewport = await popover.evaluate((el) => {
|
||||
const rect = el.getBoundingClientRect()
|
||||
return (
|
||||
rect.top >= 0 &&
|
||||
rect.left >= 0 &&
|
||||
rect.bottom <= window.innerHeight &&
|
||||
rect.right <= window.innerWidth
|
||||
)
|
||||
})
|
||||
expect(isInViewport).toBe(true)
|
||||
|
||||
const isClipped = await popover.evaluate(isClippedByAnyAncestor)
|
||||
expect(isClipped).toBe(false)
|
||||
})
|
||||
})
|
||||
@@ -1,187 +0,0 @@
|
||||
import type { ComfyPage } from '../fixtures/ComfyPage'
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
import { fitToViewInstant } from '../helpers/fitToView'
|
||||
import { getPromotedWidgetNames } from '../helpers/promotedWidgets'
|
||||
|
||||
/**
|
||||
* Convert the KSampler (id 3) in the default workflow to a subgraph,
|
||||
* enter builder, select the promoted seed widget as input and
|
||||
* SaveImage/PreviewImage as output.
|
||||
*
|
||||
* Returns the subgraph node reference for further interaction.
|
||||
*/
|
||||
async function setupSubgraphBuilder(comfyPage: ComfyPage) {
|
||||
const { page, appMode } = comfyPage
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
|
||||
const ksampler = await comfyPage.nodeOps.getNodeRefById('3')
|
||||
await ksampler.click('title')
|
||||
const subgraphNode = await ksampler.convertToSubgraph()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const subgraphNodeId = String(subgraphNode.id)
|
||||
const promotedNames = await getPromotedWidgetNames(comfyPage, subgraphNodeId)
|
||||
expect(promotedNames).toContain('seed')
|
||||
|
||||
await fitToViewInstant(comfyPage)
|
||||
await appMode.enterBuilder()
|
||||
await appMode.goToInputs()
|
||||
|
||||
// Reset zoom to 1 and center on the subgraph node so click coords are accurate
|
||||
await comfyPage.canvasOps.setScale(1)
|
||||
await subgraphNode.centerOnNode()
|
||||
|
||||
// Click the promoted seed widget on the canvas to select it
|
||||
const seedWidgetRef = await subgraphNode.getWidget(0)
|
||||
const seedPos = await seedWidgetRef.getPosition()
|
||||
const titleHeight = await page.evaluate(
|
||||
() => window.LiteGraph!['NODE_TITLE_HEIGHT'] as number
|
||||
)
|
||||
|
||||
await page.mouse.click(seedPos.x, seedPos.y + titleHeight)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Select an output node
|
||||
await appMode.goToOutputs()
|
||||
|
||||
const saveImageNodeId = await page.evaluate(() =>
|
||||
String(
|
||||
window.app!.rootGraph.nodes.find(
|
||||
(n: { type?: string }) =>
|
||||
n.type === 'SaveImage' || n.type === 'PreviewImage'
|
||||
)?.id
|
||||
)
|
||||
)
|
||||
const saveImageRef = await comfyPage.nodeOps.getNodeRefById(saveImageNodeId)
|
||||
await saveImageRef.centerOnNode()
|
||||
|
||||
// Node is centered on screen, so click the canvas center
|
||||
const canvasBox = await page.locator('#graph-canvas').boundingBox()
|
||||
if (!canvasBox) throw new Error('Canvas not found')
|
||||
await page.mouse.click(
|
||||
canvasBox.x + canvasBox.width / 2,
|
||||
canvasBox.y + canvasBox.height / 2
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
return subgraphNode
|
||||
}
|
||||
|
||||
/** Save the workflow, reopen it, and enter app mode. */
|
||||
async function saveAndReopenInAppMode(
|
||||
comfyPage: ComfyPage,
|
||||
workflowName: string
|
||||
) {
|
||||
await comfyPage.menu.topbar.saveWorkflow(workflowName)
|
||||
|
||||
const { workflowsTab } = comfyPage.menu
|
||||
await workflowsTab.open()
|
||||
await workflowsTab.getPersistedItem(workflowName).dblclick()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.appMode.toggleAppMode()
|
||||
}
|
||||
|
||||
test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.api.serverFeatureFlags.value = {
|
||||
...window.app!.api.serverFeatureFlags.value,
|
||||
linear_toggle_enabled: true
|
||||
}
|
||||
})
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.AppBuilder.VueNodeSwitchDismissed',
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
test('Rename from builder input-select sidebar via menu', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { appMode } = comfyPage
|
||||
await setupSubgraphBuilder(comfyPage)
|
||||
|
||||
// Go back to inputs step where IoItems are shown
|
||||
await appMode.goToInputs()
|
||||
|
||||
const menu = appMode.getBuilderInputItemMenu('seed')
|
||||
await expect(menu).toBeVisible({ timeout: 5000 })
|
||||
await appMode.renameBuilderInputViaMenu('seed', 'Builder Input Seed')
|
||||
|
||||
// Verify in app mode after save/reload
|
||||
await appMode.exitBuilder()
|
||||
const workflowName = `${new Date().getTime()} builder-input-menu`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(
|
||||
appMode.linearWidgets.getByText('Builder Input Seed')
|
||||
).toBeVisible()
|
||||
})
|
||||
|
||||
test('Rename from builder input-select sidebar via double-click', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { appMode } = comfyPage
|
||||
await setupSubgraphBuilder(comfyPage)
|
||||
|
||||
await appMode.goToInputs()
|
||||
|
||||
await appMode.renameBuilderInput('seed', 'Dblclick Seed')
|
||||
|
||||
await appMode.exitBuilder()
|
||||
const workflowName = `${new Date().getTime()} builder-input-dblclick`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('Dblclick Seed')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Rename from builder preview sidebar', async ({ comfyPage }) => {
|
||||
const { appMode } = comfyPage
|
||||
await setupSubgraphBuilder(comfyPage)
|
||||
|
||||
await appMode.goToPreview()
|
||||
|
||||
const menu = appMode.getBuilderPreviewWidgetMenu('seed — New Subgraph')
|
||||
await expect(menu).toBeVisible({ timeout: 5000 })
|
||||
await appMode.renameWidget(menu, 'Preview Seed')
|
||||
|
||||
// Verify in app mode after save/reload
|
||||
await appMode.exitBuilder()
|
||||
const workflowName = `${new Date().getTime()} builder-preview`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('Preview Seed')).toBeVisible()
|
||||
})
|
||||
|
||||
test('Rename from app mode', async ({ comfyPage }) => {
|
||||
const { appMode } = comfyPage
|
||||
await setupSubgraphBuilder(comfyPage)
|
||||
|
||||
// Enter app mode from builder
|
||||
await appMode.exitBuilder()
|
||||
await appMode.toggleAppMode()
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
|
||||
const menu = appMode.getAppModeWidgetMenu('seed')
|
||||
await appMode.renameWidget(menu, 'App Mode Seed')
|
||||
|
||||
await expect(appMode.linearWidgets.getByText('App Mode Seed')).toBeVisible()
|
||||
|
||||
// Verify persistence after save/reload
|
||||
await appMode.toggleAppMode()
|
||||
const workflowName = `${new Date().getTime()} app-mode`
|
||||
await saveAndReopenInAppMode(comfyPage, workflowName)
|
||||
|
||||
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
|
||||
await expect(appMode.linearWidgets.getByText('App Mode Seed')).toBeVisible()
|
||||
})
|
||||
})
|
||||
@@ -38,13 +38,16 @@ const customColorPalettes = {
|
||||
CLEAR_BACKGROUND_COLOR: '#222222',
|
||||
NODE_TITLE_COLOR: 'rgba(255,255,255,.75)',
|
||||
NODE_SELECTED_TITLE_COLOR: '#FFF',
|
||||
NODE_TEXT_SIZE: 14,
|
||||
NODE_TEXT_COLOR: '#b8b8b8',
|
||||
NODE_SUBTEXT_SIZE: 12,
|
||||
NODE_DEFAULT_COLOR: 'rgba(0,0,0,.8)',
|
||||
NODE_DEFAULT_BGCOLOR: 'rgba(22,22,22,.8)',
|
||||
NODE_DEFAULT_BOXCOLOR: 'rgba(255,255,255,.75)',
|
||||
NODE_DEFAULT_SHAPE: 'box',
|
||||
NODE_BOX_OUTLINE_COLOR: '#236692',
|
||||
DEFAULT_SHADOW_COLOR: 'rgba(0,0,0,0)',
|
||||
DEFAULT_GROUP_FONT: 24,
|
||||
WIDGET_BGCOLOR: '#242424',
|
||||
WIDGET_OUTLINE_COLOR: '#333',
|
||||
WIDGET_TEXT_COLOR: '#a3a3a8',
|
||||
@@ -99,13 +102,16 @@ const customColorPalettes = {
|
||||
CLEAR_BACKGROUND_COLOR: '#000',
|
||||
NODE_TITLE_COLOR: 'rgba(255,255,255,.75)',
|
||||
NODE_SELECTED_TITLE_COLOR: '#FFF',
|
||||
NODE_TEXT_SIZE: 14,
|
||||
NODE_TEXT_COLOR: '#b8b8b8',
|
||||
NODE_SUBTEXT_SIZE: 12,
|
||||
NODE_DEFAULT_COLOR: 'rgba(0,0,0,.8)',
|
||||
NODE_DEFAULT_BGCOLOR: 'rgba(22,22,22,.8)',
|
||||
NODE_DEFAULT_BOXCOLOR: 'rgba(255,255,255,.75)',
|
||||
NODE_DEFAULT_SHAPE: 'box',
|
||||
NODE_BOX_OUTLINE_COLOR: '#236692',
|
||||
DEFAULT_SHADOW_COLOR: 'rgba(0,0,0,0)',
|
||||
DEFAULT_GROUP_FONT: 24,
|
||||
WIDGET_BGCOLOR: '#242424',
|
||||
WIDGET_OUTLINE_COLOR: '#333',
|
||||
WIDGET_TEXT_COLOR: '#a3a3a8',
|
||||
|
||||
@@ -129,74 +129,4 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
|
||||
const undoCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(undoCount).toBe(initialCount)
|
||||
})
|
||||
|
||||
test(
|
||||
'Copy paste node, image paste onto LoadImage, image paste on empty canvas',
|
||||
{ tag: ['@node'] },
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow('nodes/load_image_with_ksampler')
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(2)
|
||||
|
||||
// Step 1: Copy a KSampler node with Ctrl+C and paste with Ctrl+V
|
||||
const ksamplerNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByType('KSampler')
|
||||
await ksamplerNodes[0].copy()
|
||||
await comfyPage.canvas.click({ position: { x: 50, y: 500 } })
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.clipboard.paste()
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5_000
|
||||
})
|
||||
.toBe(3)
|
||||
|
||||
// Step 2: Paste image onto selected LoadImage node
|
||||
const loadImageNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByType('LoadImage')
|
||||
await loadImageNodes[0].click('title')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const uploadPromise = comfyPage.page.waitForResponse(
|
||||
(resp) => resp.url().includes('/upload/') && resp.status() === 200,
|
||||
{ timeout: 10_000 }
|
||||
)
|
||||
await comfyPage.clipboard.pasteFile(
|
||||
comfyPage.assetPath('image32x32.webp')
|
||||
)
|
||||
await uploadPromise
|
||||
|
||||
await expect
|
||||
.poll(
|
||||
async () => {
|
||||
const fileWidget = await loadImageNodes[0].getWidget(0)
|
||||
return fileWidget.getValue()
|
||||
},
|
||||
{ timeout: 5_000 }
|
||||
)
|
||||
.toContain('image32x32')
|
||||
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(3)
|
||||
|
||||
// Step 3: Click empty canvas area, paste image → creates new LoadImage
|
||||
await comfyPage.canvas.click({ position: { x: 50, y: 500 } })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const uploadPromise2 = comfyPage.page.waitForResponse(
|
||||
(resp) => resp.url().includes('/upload/') && resp.status() === 200,
|
||||
{ timeout: 10_000 }
|
||||
)
|
||||
await comfyPage.clipboard.pasteFile(
|
||||
comfyPage.assetPath('image32x32.webp')
|
||||
)
|
||||
await uploadPromise2
|
||||
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5_000
|
||||
})
|
||||
.toBe(4)
|
||||
const allLoadImageNodes =
|
||||
await comfyPage.nodeOps.getNodeRefsByType('LoadImage')
|
||||
expect(allLoadImageNodes).toHaveLength(2)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
@@ -1,318 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyPage } from '../fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
|
||||
async function pressKeyAndExpectRequest(
|
||||
comfyPage: ComfyPage,
|
||||
key: string,
|
||||
urlPattern: string,
|
||||
method: string = 'POST'
|
||||
) {
|
||||
const requestPromise = comfyPage.page.waitForRequest(
|
||||
(req) => req.url().includes(urlPattern) && req.method() === method,
|
||||
{ timeout: 5000 }
|
||||
)
|
||||
await comfyPage.page.keyboard.press(key)
|
||||
return requestPromise
|
||||
}
|
||||
|
||||
test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
|
||||
test.describe('Sidebar Toggle Shortcuts', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.canvas.click({ position: { x: 400, y: 400 } })
|
||||
await comfyPage.nextFrame()
|
||||
})
|
||||
|
||||
const sidebarTabs = [
|
||||
{ key: 'KeyW', tabId: 'workflows', label: 'workflows' },
|
||||
{ key: 'KeyN', tabId: 'node-library', label: 'node library' },
|
||||
{ key: 'KeyM', tabId: 'model-library', label: 'model library' },
|
||||
{ key: 'KeyA', tabId: 'assets', label: 'assets' }
|
||||
] as const
|
||||
|
||||
for (const { key, tabId, label } of sidebarTabs) {
|
||||
test(`'${key}' toggles ${label} sidebar`, async ({ comfyPage }) => {
|
||||
const selectedButton = comfyPage.page.locator(
|
||||
`.${tabId}-tab-button.side-bar-button-selected`
|
||||
)
|
||||
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(selectedButton).toBeVisible()
|
||||
|
||||
await comfyPage.canvas.press(key)
|
||||
await comfyPage.nextFrame()
|
||||
await expect(selectedButton).not.toBeVisible()
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
test.describe('Canvas View Controls', () => {
|
||||
test("'Alt+=' zooms in", async ({ comfyPage }) => {
|
||||
const initialScale = await comfyPage.canvasOps.getScale()
|
||||
|
||||
await comfyPage.canvas.press('Alt+Equal')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const newScale = await comfyPage.canvasOps.getScale()
|
||||
expect(newScale).toBeGreaterThan(initialScale)
|
||||
})
|
||||
|
||||
test("'Alt+-' zooms out", async ({ comfyPage }) => {
|
||||
const initialScale = await comfyPage.canvasOps.getScale()
|
||||
|
||||
await comfyPage.canvas.press('Alt+Minus')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const newScale = await comfyPage.canvasOps.getScale()
|
||||
expect(newScale).toBeLessThan(initialScale)
|
||||
})
|
||||
|
||||
test("'.' fits view to nodes", async ({ comfyPage }) => {
|
||||
// Set scale very small so fit-view will zoom back to fit nodes
|
||||
await comfyPage.canvasOps.setScale(0.1)
|
||||
const scaleBefore = await comfyPage.canvasOps.getScale()
|
||||
expect(scaleBefore).toBeCloseTo(0.1, 1)
|
||||
|
||||
// Click canvas to ensure focus is within graph-canvas-container
|
||||
await comfyPage.canvas.click({ position: { x: 400, y: 400 } })
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.canvas.press('Period')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const scaleAfter = await comfyPage.canvasOps.getScale()
|
||||
expect(scaleAfter).toBeGreaterThan(scaleBefore)
|
||||
})
|
||||
|
||||
test("'h' locks canvas", async ({ comfyPage }) => {
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
|
||||
await comfyPage.canvas.press('KeyH')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
})
|
||||
|
||||
test("'v' unlocks canvas", async ({ comfyPage }) => {
|
||||
// Lock first
|
||||
await comfyPage.command.executeCommand('Comfy.Canvas.Lock')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(true)
|
||||
|
||||
await comfyPage.canvas.press('KeyV')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
expect(await comfyPage.canvasOps.isReadOnly()).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Node State Toggles', () => {
|
||||
test("'Alt+c' collapses and expands selected nodes", async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
expect(nodes.length).toBeGreaterThan(0)
|
||||
const node = nodes[0]
|
||||
|
||||
await node.click('title')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
expect(await node.isCollapsed()).toBe(false)
|
||||
|
||||
await comfyPage.canvas.press('Alt+KeyC')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await node.isCollapsed()).toBe(true)
|
||||
|
||||
await comfyPage.canvas.press('Alt+KeyC')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await node.isCollapsed()).toBe(false)
|
||||
})
|
||||
|
||||
test("'Ctrl+m' mutes and unmutes selected nodes", async ({ comfyPage }) => {
|
||||
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
|
||||
expect(nodes.length).toBeGreaterThan(0)
|
||||
const node = nodes[0]
|
||||
|
||||
await node.click('title')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Normal mode is ALWAYS (0)
|
||||
const getMode = () =>
|
||||
comfyPage.page.evaluate((nodeId) => {
|
||||
return window.app!.canvas.graph!.getNodeById(nodeId)!.mode
|
||||
}, node.id)
|
||||
|
||||
expect(await getMode()).toBe(0)
|
||||
|
||||
await comfyPage.canvas.press('Control+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
// NEVER (2) = muted
|
||||
expect(await getMode()).toBe(2)
|
||||
|
||||
await comfyPage.canvas.press('Control+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
expect(await getMode()).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Mode and Panel Toggles', () => {
|
||||
test("'Alt+m' toggles app mode", async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
// Set up linearData so app mode has something to show
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
|
||||
// Toggle off with Alt+m
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.appMode.linearWidgets).not.toBeVisible()
|
||||
|
||||
// Toggle on again
|
||||
await comfyPage.page.keyboard.press('Alt+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
|
||||
})
|
||||
|
||||
test("'Alt+Shift+m' toggles minimap", async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.settings.setSetting('Comfy.Minimap.Visible', true)
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.CanvasMenu', true)
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
|
||||
const minimap = comfyPage.page.locator('.litegraph-minimap')
|
||||
await expect(minimap).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(minimap).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(minimap).toBeVisible()
|
||||
})
|
||||
|
||||
test("'Ctrl+`' toggles terminal/logs panel", async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.bottomPanel.root).toBeVisible()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Backquote')
|
||||
await comfyPage.nextFrame()
|
||||
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Queue and Execution', () => {
|
||||
test("'Ctrl+Enter' queues prompt", async ({ comfyPage }) => {
|
||||
const request = await pressKeyAndExpectRequest(
|
||||
comfyPage,
|
||||
'Control+Enter',
|
||||
'/prompt',
|
||||
'POST'
|
||||
)
|
||||
expect(request.url()).toContain('/prompt')
|
||||
})
|
||||
|
||||
test("'Ctrl+Shift+Enter' queues prompt to front", async ({ comfyPage }) => {
|
||||
const request = await pressKeyAndExpectRequest(
|
||||
comfyPage,
|
||||
'Control+Shift+Enter',
|
||||
'/prompt',
|
||||
'POST'
|
||||
)
|
||||
const body = request.postDataJSON()
|
||||
expect(body.front).toBe(true)
|
||||
})
|
||||
|
||||
test("'Ctrl+Alt+Enter' interrupts execution", async ({ comfyPage }) => {
|
||||
const request = await pressKeyAndExpectRequest(
|
||||
comfyPage,
|
||||
'Control+Alt+Enter',
|
||||
'/interrupt',
|
||||
'POST'
|
||||
)
|
||||
expect(request.url()).toContain('/interrupt')
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('File Operations', () => {
|
||||
test("'Ctrl+s' triggers save workflow", async ({ comfyPage }) => {
|
||||
// On a new unsaved workflow, Ctrl+s triggers Save As dialog.
|
||||
// The dialog appearing proves the keybinding was intercepted by the app.
|
||||
await comfyPage.page.keyboard.press('Control+s')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// The Save As dialog should appear (p-dialog overlay)
|
||||
const dialogOverlay = comfyPage.page.locator('.p-dialog-mask')
|
||||
await expect(dialogOverlay).toBeVisible({ timeout: 3000 })
|
||||
|
||||
// Dismiss the dialog
|
||||
await comfyPage.page.keyboard.press('Escape')
|
||||
await comfyPage.nextFrame()
|
||||
})
|
||||
|
||||
test("'Ctrl+o' triggers open workflow", async ({ comfyPage }) => {
|
||||
// Ctrl+o calls app.ui.loadFile() which clicks a hidden file input.
|
||||
// Detect the file input click via an event listener.
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.TestCommand = false
|
||||
const fileInputs =
|
||||
document.querySelectorAll<HTMLInputElement>('input[type="file"]')
|
||||
for (const input of fileInputs) {
|
||||
input.addEventListener('click', () => {
|
||||
window.TestCommand = true
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+o')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
expect(await comfyPage.page.evaluate(() => window.TestCommand)).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Graph Operations', () => {
|
||||
test("'Ctrl+Shift+e' converts selection to subgraph", async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const initialCount = await comfyPage.nodeOps.getGraphNodesCount()
|
||||
expect(initialCount).toBeGreaterThan(1)
|
||||
|
||||
// Select all nodes
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.page.keyboard.press('Control+Shift+KeyE')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// After conversion, node count should decrease
|
||||
// (multiple nodes replaced by single subgraph node)
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
|
||||
timeout: 5000
|
||||
})
|
||||
.toBeLessThan(initialCount)
|
||||
})
|
||||
|
||||
test("'r' refreshes node definitions", async ({ comfyPage }) => {
|
||||
const request = await pressKeyAndExpectRequest(
|
||||
comfyPage,
|
||||
'KeyR',
|
||||
'/object_info',
|
||||
'GET'
|
||||
)
|
||||
expect(request.url()).toContain('/object_info')
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -28,7 +28,7 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
|
||||
)
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
|
||||
const missingNodesTitle = errorOverlay.getByText(/Missing Node Packs/)
|
||||
const missingNodesTitle = comfyPage.page.getByText(/Missing Node Packs/)
|
||||
await expect(missingNodesTitle).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -42,13 +42,11 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
|
||||
)
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
|
||||
const missingNodesTitle = errorOverlay.getByText(/Missing Node Packs/)
|
||||
const missingNodesTitle = comfyPage.page.getByText(/Missing Node Packs/)
|
||||
await expect(missingNodesTitle).toBeVisible()
|
||||
|
||||
// Click "See Errors" to open the errors tab and verify subgraph node content
|
||||
await errorOverlay
|
||||
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
|
||||
.click()
|
||||
await errorOverlay.getByRole('button', { name: 'See Errors' }).click()
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
|
||||
const missingNodeCard = comfyPage.page.getByTestId(
|
||||
@@ -77,9 +75,7 @@ test.describe('Missing nodes in Error Overlay', { tag: '@ui' }, () => {
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
|
||||
// Click "See Errors" to open the right side panel errors tab
|
||||
await errorOverlay
|
||||
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
|
||||
.click()
|
||||
await errorOverlay.getByRole('button', { name: 'See Errors' }).click()
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
|
||||
// Verify MissingNodeCard is rendered in the errors tab
|
||||
@@ -148,44 +144,6 @@ test.describe('Execution error', () => {
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Error actions in Errors Tab', { tag: '@ui' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.RightSidePanel.ShowErrorsTab',
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
test('Should show Find on GitHub and Copy buttons in error card after execution error', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('nodes/execution_error')
|
||||
await comfyPage.command.executeCommand('Comfy.QueuePrompt')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Wait for error overlay and click "See Errors"
|
||||
const errorOverlay = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.errorOverlay
|
||||
)
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
await errorOverlay
|
||||
.getByTestId(TestIds.dialogs.errorOverlaySeeErrors)
|
||||
.click()
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
|
||||
// Verify Find on GitHub button is present in the error card
|
||||
const findOnGithubButton = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.errorCardFindOnGithub
|
||||
)
|
||||
await expect(findOnGithubButton).toBeVisible()
|
||||
|
||||
// Verify Copy button is present in the error card
|
||||
const copyButton = comfyPage.page.getByTestId(TestIds.dialogs.errorCardCopy)
|
||||
await expect(copyButton).toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Missing models in Error Tab', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
@@ -210,7 +168,7 @@ test.describe('Missing models in Error Tab', () => {
|
||||
)
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
|
||||
const missingModelsTitle = errorOverlay.getByText(/Missing Models/)
|
||||
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
|
||||
await expect(missingModelsTitle).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -226,7 +184,7 @@ test.describe('Missing models in Error Tab', () => {
|
||||
)
|
||||
await expect(errorOverlay).toBeVisible()
|
||||
|
||||
const missingModelsTitle = errorOverlay.getByText(/Missing Models/)
|
||||
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
|
||||
await expect(missingModelsTitle).toBeVisible()
|
||||
})
|
||||
|
||||
@@ -237,10 +195,13 @@ test.describe('Missing models in Error Tab', () => {
|
||||
'missing/model_metadata_widget_mismatch'
|
||||
)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
|
||||
).not.toBeVisible()
|
||||
await expect(comfyPage.page.getByText(/Missing Models/)).not.toBeVisible()
|
||||
const missingModelsTitle = comfyPage.page.getByText(/Missing Models/)
|
||||
await expect(missingModelsTitle).not.toBeVisible()
|
||||
|
||||
const errorOverlay = comfyPage.page.getByTestId(
|
||||
TestIds.dialogs.errorOverlay
|
||||
)
|
||||
await expect(errorOverlay).not.toBeVisible()
|
||||
})
|
||||
|
||||
// Flaky test after parallelization
|
||||
|
||||
@@ -23,85 +23,4 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
|
||||
await comfyPage.workflow.loadWorkflow('links/bad_link')
|
||||
await expect.poll(() => comfyPage.toast.getVisibleToastCount()).toBe(2)
|
||||
})
|
||||
|
||||
// Regression: duplicate links with shifted target_slot (widget-to-input
|
||||
// conversion) caused the wrong link to survive during deduplication.
|
||||
// Switch(CFG) node 120 connects to both KSamplerAdvanced 85 and 86 (2 links).
|
||||
// Links 257 and 276 shared the same tuple (origin=120 → target=85 slot=5).
|
||||
// Node 85's input.link was 276 (valid), but the bug kept 257 (stale) and
|
||||
// removed 276, breaking the cfg connection on KSamplerAdvanced 85.
|
||||
// Ref: https://github.com/Comfy-Org/ComfyUI_frontend/issues/10291
|
||||
test('Deduplicates links without breaking connections on slot-drift workflow', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('links/duplicate_links_slot_drift')
|
||||
|
||||
const result = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
|
||||
const subgraph = graph.subgraphs.values().next().value
|
||||
if (!subgraph) return { error: 'No subgraph found' }
|
||||
|
||||
// Node 120 = Switch (CFG), connects to both KSamplerAdvanced 85 and 86
|
||||
const switchCfg = subgraph.getNodeById(120)
|
||||
const ksampler85 = subgraph.getNodeById(85)
|
||||
const ksampler86 = subgraph.getNodeById(86)
|
||||
if (!switchCfg || !ksampler85 || !ksampler86)
|
||||
return { error: 'Required nodes not found' }
|
||||
|
||||
// Find cfg inputs by name (slot indices shift due to widget-to-input)
|
||||
const cfgInput85 = ksampler85.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
const cfgInput86 = ksampler86.inputs.find(
|
||||
(i: { name: string }) => i.name === 'cfg'
|
||||
)
|
||||
const cfg85Linked = cfgInput85?.link != null
|
||||
const cfg86Linked = cfgInput86?.link != null
|
||||
|
||||
// Verify the surviving links exist in the subgraph link map
|
||||
const cfg85LinkValid =
|
||||
cfg85Linked && subgraph.links.has(cfgInput85!.link!)
|
||||
const cfg86LinkValid =
|
||||
cfg86Linked && subgraph.links.has(cfgInput86!.link!)
|
||||
|
||||
// Switch(CFG) output should have exactly 2 links (one to each KSampler)
|
||||
const switchOutputLinkCount = switchCfg.outputs[0]?.links?.length ?? 0
|
||||
|
||||
// Count links from Switch(CFG) to node 85 cfg (should be 1, not 2)
|
||||
let cfgLinkToNode85Count = 0
|
||||
for (const link of subgraph.links.values()) {
|
||||
if (link.origin_id === 120 && link.target_id === 85)
|
||||
cfgLinkToNode85Count++
|
||||
}
|
||||
|
||||
return {
|
||||
cfg85Linked,
|
||||
cfg86Linked,
|
||||
cfg85LinkValid,
|
||||
cfg86LinkValid,
|
||||
cfg85LinkId: cfgInput85?.link ?? null,
|
||||
cfg86LinkId: cfgInput86?.link ?? null,
|
||||
switchOutputLinkIds: [...(switchCfg.outputs[0]?.links ?? [])],
|
||||
switchOutputLinkCount,
|
||||
cfgLinkToNode85Count
|
||||
}
|
||||
})
|
||||
|
||||
expect(result).not.toHaveProperty('error')
|
||||
// Both KSamplerAdvanced nodes must have their cfg input connected
|
||||
expect(result.cfg85Linked).toBe(true)
|
||||
expect(result.cfg86Linked).toBe(true)
|
||||
// Links must exist in the subgraph link map
|
||||
expect(result.cfg85LinkValid).toBe(true)
|
||||
expect(result.cfg86LinkValid).toBe(true)
|
||||
// Switch(CFG) output has exactly 2 links (one per KSamplerAdvanced)
|
||||
expect(result.switchOutputLinkCount).toBe(2)
|
||||
// Only 1 link from Switch(CFG) to node 85 (duplicate removed)
|
||||
expect(result.cfgLinkToNode85Count).toBe(1)
|
||||
// Output link IDs must match the input link IDs (source/target integrity)
|
||||
expect(result.switchOutputLinkIds).toEqual(
|
||||
expect.arrayContaining([result.cfg85LinkId, result.cfg86LinkId])
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -764,13 +764,13 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
)
|
||||
})
|
||||
|
||||
const generateUniqueFilename = (extension = '') =>
|
||||
`${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}${extension}`
|
||||
|
||||
test.describe('Restore all open workflows on reload', () => {
|
||||
let workflowA: string
|
||||
let workflowB: string
|
||||
|
||||
const generateUniqueFilename = (extension = '') =>
|
||||
`${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}${extension}`
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
@@ -829,82 +829,6 @@ test.describe('Load workflow', { tag: '@screenshot' }, () => {
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Restore workflow tabs after browser restart', () => {
|
||||
let workflowA: string
|
||||
let workflowB: string
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
workflowA = generateUniqueFilename()
|
||||
await comfyPage.menu.topbar.saveWorkflow(workflowA)
|
||||
workflowB = generateUniqueFilename()
|
||||
await comfyPage.menu.topbar.triggerTopbarCommand(['New'])
|
||||
await comfyPage.menu.topbar.saveWorkflow(workflowB)
|
||||
|
||||
// Wait for localStorage fallback pointers to be written
|
||||
await comfyPage.page.waitForFunction(() => {
|
||||
for (let i = 0; i < window.localStorage.length; i++) {
|
||||
const key = window.localStorage.key(i)
|
||||
if (key?.startsWith('Comfy.Workflow.LastOpenPaths:')) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
})
|
||||
|
||||
// Simulate browser restart: clear sessionStorage (lost on close)
|
||||
// but keep localStorage (survives browser restart)
|
||||
await comfyPage.page.evaluate(() => {
|
||||
sessionStorage.clear()
|
||||
})
|
||||
await comfyPage.setup({ clearStorage: false })
|
||||
})
|
||||
|
||||
test('Restores topbar workflow tabs after browser restart', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.Workflow.WorkflowTabsPosition',
|
||||
'Topbar'
|
||||
)
|
||||
// Wait for both restored tabs to render (localStorage fallback is async)
|
||||
await expect(
|
||||
comfyPage.page.locator('.workflow-tabs .workflow-label', {
|
||||
hasText: workflowA
|
||||
})
|
||||
).toBeVisible()
|
||||
|
||||
const tabs = await comfyPage.menu.topbar.getTabNames()
|
||||
const activeWorkflowName = await comfyPage.menu.topbar.getActiveTabName()
|
||||
|
||||
expect(tabs).toEqual(expect.arrayContaining([workflowA, workflowB]))
|
||||
expect(tabs.indexOf(workflowA)).toBeLessThan(tabs.indexOf(workflowB))
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
|
||||
test('Restores sidebar workflows after browser restart', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.Workflow.WorkflowTabsPosition',
|
||||
'Sidebar'
|
||||
)
|
||||
await comfyPage.menu.workflowsTab.open()
|
||||
const openWorkflows =
|
||||
await comfyPage.menu.workflowsTab.getOpenedWorkflowNames()
|
||||
const activeWorkflowName =
|
||||
await comfyPage.menu.workflowsTab.getActiveWorkflowName()
|
||||
expect(openWorkflows).toEqual(
|
||||
expect.arrayContaining([workflowA, workflowB])
|
||||
)
|
||||
expect(openWorkflows.indexOf(workflowA)).toBeLessThan(
|
||||
openWorkflows.indexOf(workflowB)
|
||||
)
|
||||
expect(activeWorkflowName).toEqual(workflowB)
|
||||
})
|
||||
})
|
||||
|
||||
test('Auto fit view after loading workflow', async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.EnableWorkflowViewRestore',
|
||||
|
||||
|
Before Width: | Height: | Size: 60 KiB After Width: | Height: | Size: 64 KiB |
|
Before Width: | Height: | Size: 101 KiB After Width: | Height: | Size: 101 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 80 KiB |
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 77 KiB |
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 77 KiB |
@@ -1,257 +0,0 @@
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
|
||||
import type { Page } from '@playwright/test'
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
|
||||
const TEST_PRESET = {
|
||||
name: 'test-preset',
|
||||
newBindings: [
|
||||
{
|
||||
commandId: 'Comfy.Canvas.SelectAll',
|
||||
combo: { key: 'a', ctrl: true, shift: true },
|
||||
targetElementId: 'graph-canvas-container'
|
||||
}
|
||||
],
|
||||
unsetBindings: [
|
||||
{
|
||||
commandId: 'Comfy.Canvas.SelectAll',
|
||||
combo: { key: 'a', ctrl: true },
|
||||
targetElementId: 'graph-canvas-container'
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
async function importPreset(page: Page, preset: typeof TEST_PRESET) {
|
||||
const menuButton = page.getByTestId('keybinding-preset-menu')
|
||||
await menuButton.click()
|
||||
|
||||
const fileChooserPromise = page.waitForEvent('filechooser')
|
||||
await page.getByRole('menuitem', { name: /Import preset/i }).click()
|
||||
const fileChooser = await fileChooserPromise
|
||||
|
||||
const presetPath = path.join(os.tmpdir(), 'test-preset.json')
|
||||
fs.writeFileSync(presetPath, JSON.stringify(preset))
|
||||
await fileChooser.setFiles(presetPath)
|
||||
}
|
||||
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Disabled')
|
||||
})
|
||||
|
||||
test.afterEach(async ({ comfyPage }) => {
|
||||
await comfyPage.request.fetch(
|
||||
`${comfyPage.url}/api/userdata/keybindings%2Ftest-preset.json`,
|
||||
{ method: 'DELETE' }
|
||||
)
|
||||
await comfyPage.settings.setSetting(
|
||||
'Comfy.Keybinding.CurrentPreset',
|
||||
'default'
|
||||
)
|
||||
})
|
||||
|
||||
test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
|
||||
test('Can import a preset, use remapped keybinding, and switch back to default', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
test.setTimeout(30000)
|
||||
const { page } = comfyPage
|
||||
|
||||
// Verify default Ctrl+A select-all works
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
await comfyPage.canvas.press('Control+a')
|
||||
await comfyPage.canvas.press('Delete')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
|
||||
// Open keybinding settings panel
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.category('Keybinding').click()
|
||||
|
||||
await importPreset(page, TEST_PRESET)
|
||||
|
||||
// Verify active preset switched to test-preset
|
||||
const presetTrigger = page
|
||||
.locator('#keybinding-panel-actions')
|
||||
.locator('button[role="combobox"]')
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss, then close settings via Escape
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
|
||||
// Load workflow again, use new keybind Ctrl+Shift+A
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
await comfyPage.canvas.press('Control+Shift+a')
|
||||
await expect
|
||||
.poll(() => comfyPage.nodeOps.getSelectedGraphNodesCount())
|
||||
.toBeGreaterThan(0)
|
||||
await comfyPage.canvas.press('Delete')
|
||||
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(0)
|
||||
|
||||
// Switch back to default preset
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.category('Keybinding').click()
|
||||
|
||||
await presetTrigger.click()
|
||||
await page.getByRole('option', { name: /Default Preset/i }).click()
|
||||
|
||||
// Handle unsaved changes dialog if the preset was marked as modified
|
||||
const discardButton = page.getByRole('button', {
|
||||
name: /Discard and Switch/i
|
||||
})
|
||||
if (await discardButton.isVisible({ timeout: 2000 }).catch(() => false)) {
|
||||
await discardButton.click()
|
||||
}
|
||||
|
||||
await expect(presetTrigger).toContainText('Default Preset')
|
||||
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
})
|
||||
|
||||
test('Can export a preset and re-import it', async ({ comfyPage }) => {
|
||||
test.setTimeout(30000)
|
||||
const { page } = comfyPage
|
||||
const menuButton = page.getByTestId('keybinding-preset-menu')
|
||||
|
||||
// Open keybinding settings panel
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.category('Keybinding').click()
|
||||
|
||||
await importPreset(page, TEST_PRESET)
|
||||
|
||||
// Verify active preset switched to test-preset
|
||||
const presetTrigger = page
|
||||
.locator('#keybinding-panel-actions')
|
||||
.locator('button[role="combobox"]')
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Export via ellipsis menu
|
||||
await menuButton.click()
|
||||
const downloadPromise = page.waitForEvent('download')
|
||||
await page.getByRole('menuitem', { name: /Export preset/i }).click()
|
||||
const download = await downloadPromise
|
||||
|
||||
// Verify filename contains test-preset
|
||||
expect(download.suggestedFilename()).toContain('test-preset')
|
||||
|
||||
// Close settings
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
|
||||
// Verify the downloaded file is valid JSON with correct structure
|
||||
const downloadPath = await download.path()
|
||||
expect(downloadPath).toBeTruthy()
|
||||
const content = fs.readFileSync(downloadPath!, 'utf-8')
|
||||
const parsed = JSON.parse(content) as {
|
||||
name: string
|
||||
newBindings: unknown[]
|
||||
unsetBindings: unknown[]
|
||||
}
|
||||
expect(parsed).toHaveProperty('name')
|
||||
expect(parsed).toHaveProperty('newBindings')
|
||||
expect(parsed).toHaveProperty('unsetBindings')
|
||||
expect(parsed.name).toBe('test-preset')
|
||||
})
|
||||
|
||||
test('Can delete an imported preset', async ({ comfyPage }) => {
|
||||
test.setTimeout(30000)
|
||||
const { page } = comfyPage
|
||||
const menuButton = page.getByTestId('keybinding-preset-menu')
|
||||
|
||||
// Open keybinding settings panel
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.category('Keybinding').click()
|
||||
|
||||
await importPreset(page, TEST_PRESET)
|
||||
|
||||
// Verify active preset switched to test-preset
|
||||
const presetTrigger = page
|
||||
.locator('#keybinding-panel-actions')
|
||||
.locator('button[role="combobox"]')
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Delete via ellipsis menu
|
||||
await menuButton.click()
|
||||
await page.getByRole('menuitem', { name: /Delete preset/i }).click()
|
||||
|
||||
// Confirm deletion in the dialog
|
||||
const confirmDialog = page.getByRole('dialog', {
|
||||
name: /Delete the current preset/i
|
||||
})
|
||||
await confirmDialog.getByRole('button', { name: /Delete/i }).click()
|
||||
|
||||
// Verify preset trigger now shows Default Preset
|
||||
await expect(presetTrigger).toContainText('Default Preset')
|
||||
|
||||
// Close settings
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
})
|
||||
|
||||
test('Can save modifications as a new preset', async ({ comfyPage }) => {
|
||||
test.setTimeout(30000)
|
||||
const { page } = comfyPage
|
||||
const menuButton = page.getByTestId('keybinding-preset-menu')
|
||||
|
||||
// Open keybinding settings panel
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.settingDialog.category('Keybinding').click()
|
||||
|
||||
await importPreset(page, TEST_PRESET)
|
||||
|
||||
// Verify active preset switched to test-preset
|
||||
const presetTrigger = page
|
||||
.locator('#keybinding-panel-actions')
|
||||
.locator('button[role="combobox"]')
|
||||
await expect(presetTrigger).toContainText('test-preset')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Save as new preset via ellipsis menu
|
||||
await menuButton.click()
|
||||
await page.getByRole('menuitem', { name: /Save as new preset/i }).click()
|
||||
|
||||
// Fill in the preset name in the prompt dialog
|
||||
const promptInput = page.locator('.prompt-dialog-content input')
|
||||
await promptInput.fill('my-custom-preset')
|
||||
await promptInput.press('Enter')
|
||||
|
||||
// Wait for toast to auto-dismiss
|
||||
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
|
||||
timeout: 5000
|
||||
})
|
||||
|
||||
// Verify preset trigger shows my-custom-preset
|
||||
await expect(presetTrigger).toContainText('my-custom-preset')
|
||||
|
||||
// Close settings
|
||||
await page.keyboard.press('Escape')
|
||||
await comfyPage.settingDialog.waitForHidden()
|
||||
|
||||
// Cleanup: delete the my-custom-preset file
|
||||
await comfyPage.request.fetch(
|
||||
`${comfyPage.url}/api/userdata/keybindings%2Fmy-custom-preset.json`,
|
||||
{ method: 'DELETE' }
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { Page } from '@playwright/test'
|
||||
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
@@ -9,10 +11,58 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
|
||||
await comfyPage.setup()
|
||||
})
|
||||
|
||||
async function enterAppMode(comfyPage: {
|
||||
page: Page
|
||||
nextFrame: () => Promise<void>
|
||||
}) {
|
||||
// LinearControls requires hasOutputs to be true. Serialize the current
|
||||
// graph, inject linearData with output node IDs, then reload so the
|
||||
// appModeStore picks up the outputs via its activeWorkflow watcher.
|
||||
await comfyPage.page.evaluate(async () => {
|
||||
const graph = window.app!.graph
|
||||
if (!graph) return
|
||||
|
||||
const outputNodeIds = graph.nodes
|
||||
.filter(
|
||||
(n: { type?: string }) =>
|
||||
n.type === 'SaveImage' || n.type === 'PreviewImage'
|
||||
)
|
||||
.map((n: { id: number | string }) => String(n.id))
|
||||
|
||||
// Serialize, inject linearData, and reload to sync stores
|
||||
const workflow = graph.serialize() as unknown as Record<string, unknown>
|
||||
const extra = (workflow.extra ?? {}) as Record<string, unknown>
|
||||
extra.linearData = { inputs: [], outputs: outputNodeIds }
|
||||
workflow.extra = extra
|
||||
await window.app!.loadGraphData(
|
||||
workflow as unknown as Parameters<
|
||||
NonNullable<typeof window.app>['loadGraphData']
|
||||
>[0]
|
||||
)
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Toggle to app mode via the command which sets canvasStore.linearMode
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.extensionManager.command.execute('Comfy.ToggleLinear')
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
async function enterGraphMode(comfyPage: {
|
||||
page: Page
|
||||
nextFrame: () => Promise<void>
|
||||
}) {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
window.app!.extensionManager.command.execute('Comfy.ToggleLinear')
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
test('Displays linear controls when app mode active', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await enterAppMode(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="linear-widgets"]')
|
||||
@@ -20,7 +70,7 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
test('Run button visible in linear mode', async ({ comfyPage }) => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await enterAppMode(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="linear-run-button"]')
|
||||
@@ -28,7 +78,7 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
test('Workflow info section visible', async ({ comfyPage }) => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await enterAppMode(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="linear-workflow-info"]')
|
||||
@@ -36,13 +86,13 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
test('Returns to graph mode', async ({ comfyPage }) => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await enterAppMode(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="linear-widgets"]')
|
||||
).toBeVisible({ timeout: 5000 })
|
||||
|
||||
await comfyPage.appMode.toggleAppMode()
|
||||
await enterGraphMode(comfyPage)
|
||||
|
||||
await expect(comfyPage.canvas).toBeVisible({ timeout: 5000 })
|
||||
await expect(
|
||||
@@ -51,7 +101,7 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
|
||||
})
|
||||
|
||||
test('Canvas not visible in app mode', async ({ comfyPage }) => {
|
||||
await comfyPage.appMode.enterAppModeWithInputs([])
|
||||
await enterAppMode(comfyPage)
|
||||
|
||||
await expect(
|
||||
comfyPage.page.locator('[data-testid="linear-widgets"]')
|
||||
|
||||
@@ -24,20 +24,6 @@ test.describe(
|
||||
)
|
||||
})
|
||||
|
||||
test('@mobile graph canvas toolbar visible', async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.CanvasMenu', true)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const minimapButton = comfyPage.page.getByTestId(
|
||||
TestIds.canvas.toggleMinimapButton
|
||||
)
|
||||
await expect(minimapButton).toBeVisible()
|
||||
|
||||
await expect(comfyPage.canvas).toHaveScreenshot(
|
||||
'mobile-graph-canvas-toolbar.png'
|
||||
)
|
||||
})
|
||||
|
||||
test('@mobile settings dialog', async ({ comfyPage }) => {
|
||||
await comfyPage.settingDialog.open()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
|
Before Width: | Height: | Size: 32 KiB |
@@ -79,7 +79,6 @@ test.describe('Node search box', { tag: '@node' }, () => {
|
||||
'Can auto link batch moved node',
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.AutoPanSpeed', 0)
|
||||
await comfyPage.workflow.loadWorkflow('links/batch_move_links')
|
||||
|
||||
// Get the CLIP output slot (index 1) from the first CheckpointLoaderSimple node (id: 4)
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
|
||||
test.describe('Select Component Escape Key Propagation', { tag: '@ui' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
// Ensure clean state
|
||||
await comfyPage.workflow.setupWorkflowsDirectory({})
|
||||
})
|
||||
|
||||
test('pressing Escape in a Select dropdown closes only the menu and not the parent dialog', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { page } = comfyPage
|
||||
|
||||
// 1. Open the Settings dialog
|
||||
await comfyPage.menu.topbar.openTopbarMenu()
|
||||
await page.getByRole('menuitem', { name: 'Settings' }).click()
|
||||
|
||||
const settingsDialog = page.getByRole('dialog')
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
|
||||
// 2. Open a Select dropdown (e.g., Language)
|
||||
// Select components in ComfyUI use aria-haspopup="listbox"
|
||||
const selectTrigger = settingsDialog
|
||||
.locator('button[aria-haspopup="listbox"]')
|
||||
.first()
|
||||
await selectTrigger.click()
|
||||
|
||||
// Verify the dropdown menu is open
|
||||
const dropdownContent = page.locator('[data-dismissable-layer]')
|
||||
await expect(dropdownContent).toBeVisible()
|
||||
|
||||
// 3. Press Escape
|
||||
// This should close the dropdown but NOT the Settings dialog
|
||||
await page.keyboard.press('Escape')
|
||||
|
||||
// 4. Assertions
|
||||
await expect(dropdownContent).not.toBeVisible()
|
||||
await expect(settingsDialog).toBeVisible()
|
||||
|
||||
// Visual verification that the dialog is still present and focused
|
||||
await expect(settingsDialog).toHaveScreenshot(
|
||||
'settings-dialog-remains-open.png'
|
||||
)
|
||||
|
||||
// 5. Press Escape again
|
||||
// Now that the dropdown is closed, Escape should close the dialog
|
||||
await page.keyboard.press('Escape')
|
||||
await expect(settingsDialog).not.toBeVisible()
|
||||
})
|
||||
})
|
||||
@@ -123,7 +123,6 @@ test.describe('Node Right Click Menu', { tag: ['@screenshot', '@ui'] }, () => {
|
||||
})
|
||||
|
||||
test('Can pin and unpin', async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.Graph.AutoPanSpeed', 0)
|
||||
await comfyPage.canvas.click({
|
||||
position: DefaultGraphPositions.emptyLatentWidgetClick,
|
||||
button: 'right'
|
||||
|
||||
|
Before Width: | Height: | Size: 93 KiB After Width: | Height: | Size: 93 KiB |
@@ -43,31 +43,6 @@ test.describe('Subgraph duplicate ID remapping', { tag: ['@subgraph'] }, () => {
|
||||
expect(rootIds).toEqual([1, 2, 5])
|
||||
})
|
||||
|
||||
test('Promoted widget tuples are stable after full page reload boot path', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const beforeSnapshot =
|
||||
await comfyPage.subgraph.getHostPromotedTupleSnapshot()
|
||||
expect(beforeSnapshot.length).toBeGreaterThan(0)
|
||||
expect(
|
||||
beforeSnapshot.some(({ promotedWidgets }) => promotedWidgets.length > 0)
|
||||
).toBe(true)
|
||||
|
||||
await comfyPage.page.reload()
|
||||
await comfyPage.page.waitForFunction(() => !!window.app)
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(async () => {
|
||||
const afterSnapshot =
|
||||
await comfyPage.subgraph.getHostPromotedTupleSnapshot()
|
||||
expect(afterSnapshot).toEqual(beforeSnapshot)
|
||||
}).toPass({ timeout: 5_000 })
|
||||
})
|
||||
|
||||
test('All links reference valid nodes in their graph', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { getTextSlotPosition } from '../helpers/subgraphTestUtils'
|
||||
|
||||
test.describe(
|
||||
'Subgraph promoted widget-input slot position',
|
||||
{ tag: '@subgraph' },
|
||||
() => {
|
||||
test('Promoted text widget slot is positioned at widget row, not header', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
|
||||
// Render a few frames so arrange() runs
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const result = await getTextSlotPosition(comfyPage.page, '11')
|
||||
expect(result).not.toBeNull()
|
||||
expect(result!.hasPos).toBe(true)
|
||||
|
||||
// The slot Y position should be well below the title area.
|
||||
// If it's near 0 or negative, the slot is stuck at the header (the bug).
|
||||
expect(result!.posY).toBeGreaterThan(result!.titleHeight)
|
||||
})
|
||||
|
||||
test('Slot position remains correct after renaming subgraph input label', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Verify initial position is correct
|
||||
const before = await getTextSlotPosition(comfyPage.page, '11')
|
||||
expect(before).not.toBeNull()
|
||||
expect(before!.hasPos).toBe(true)
|
||||
expect(before!.posY).toBeGreaterThan(before!.titleHeight)
|
||||
|
||||
// Navigate into subgraph and rename the text input
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('11')
|
||||
await subgraphNode.navigateIntoSubgraph()
|
||||
|
||||
const initialLabel = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
if (!graph || !('inputNode' in graph)) return null
|
||||
const textInput = graph.inputs?.find(
|
||||
(i: { type: string }) => i.type === 'STRING'
|
||||
)
|
||||
return textInput?.label || textInput?.name || null
|
||||
})
|
||||
|
||||
if (!initialLabel)
|
||||
throw new Error('Could not find STRING input in subgraph')
|
||||
|
||||
await comfyPage.subgraph.rightClickInputSlot(initialLabel)
|
||||
await comfyPage.contextMenu.clickLitegraphMenuItem('Rename Slot')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const dialog = '.graphdialog input'
|
||||
await comfyPage.page.waitForSelector(dialog, { state: 'visible' })
|
||||
await comfyPage.page.fill(dialog, '')
|
||||
await comfyPage.page.fill(dialog, 'my_custom_prompt')
|
||||
await comfyPage.page.keyboard.press('Enter')
|
||||
await comfyPage.page.waitForSelector(dialog, { state: 'hidden' })
|
||||
|
||||
// Navigate back to parent graph
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
|
||||
// Verify slot position is still at the widget row after rename
|
||||
const after = await getTextSlotPosition(comfyPage.page, '11')
|
||||
expect(after).not.toBeNull()
|
||||
expect(after!.hasPos).toBe(true)
|
||||
expect(after!.posY).toBeGreaterThan(after!.titleHeight)
|
||||
|
||||
// widget.name is the stable identity key — it does NOT change on rename.
|
||||
// The display label is on input.label, read via PromotedWidgetView.label.
|
||||
expect(after!.widgetName).not.toBe('my_custom_prompt')
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -1,57 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { getPromotedWidgetNames } from '../helpers/promotedWidgets'
|
||||
|
||||
test.describe(
|
||||
'Subgraph promoted widget DOM position',
|
||||
{ tag: '@subgraph' },
|
||||
() => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Disabled')
|
||||
})
|
||||
|
||||
test('Promoted seed widget renders in node body, not header', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
|
||||
// Convert KSampler (id 3) to subgraph — seed is auto-promoted.
|
||||
const ksampler = await comfyPage.nodeOps.getNodeRefById('3')
|
||||
await ksampler.click('title')
|
||||
const subgraphNode = await ksampler.convertToSubgraph()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Enable Vue nodes now that the subgraph has been created
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
|
||||
const subgraphNodeId = String(subgraphNode.id)
|
||||
const promotedNames = await getPromotedWidgetNames(
|
||||
comfyPage,
|
||||
subgraphNodeId
|
||||
)
|
||||
expect(promotedNames).toContain('seed')
|
||||
|
||||
// Wait for Vue nodes to render
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
const nodeLocator = comfyPage.vueNodes.getNodeLocator(subgraphNodeId)
|
||||
await expect(nodeLocator).toBeVisible()
|
||||
|
||||
// The seed widget should be visible inside the node body
|
||||
const seedWidget = nodeLocator.getByLabel('seed', { exact: true }).first()
|
||||
await expect(seedWidget).toBeVisible()
|
||||
|
||||
// Verify widget is inside the node body, not the header
|
||||
const headerBox = await nodeLocator
|
||||
.locator('[data-testid^="node-header-"]')
|
||||
.boundingBox()
|
||||
const widgetBox = await seedWidget.boundingBox()
|
||||
expect(headerBox).not.toBeNull()
|
||||
expect(widgetBox).not.toBeNull()
|
||||
|
||||
// Widget top should be below the header bottom
|
||||
expect(widgetBox!.y).toBeGreaterThan(headerBox!.y + headerBox!.height)
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -1,7 +1,6 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { TestIds } from '../fixtures/selectors'
|
||||
|
||||
// Constants
|
||||
const RENAMED_INPUT_NAME = 'renamed_input'
|
||||
@@ -655,28 +654,6 @@ test.describe('Subgraph Operations', { tag: ['@slow', '@subgraph'] }, () => {
|
||||
await comfyPage.nextFrame()
|
||||
expect(await isInSubgraph(comfyPage)).toBe(false)
|
||||
})
|
||||
|
||||
test('Breadcrumb disappears after switching workflows while inside subgraph', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('subgraphs/basic-subgraph')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const breadcrumb = comfyPage.page
|
||||
.getByTestId(TestIds.breadcrumb.subgraph)
|
||||
.locator('.p-breadcrumb')
|
||||
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('2')
|
||||
await subgraphNode.navigateIntoSubgraph()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(breadcrumb).toBeVisible()
|
||||
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(breadcrumb).toBeHidden()
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('DOM Widget Promotion', () => {
|
||||
@@ -790,9 +767,11 @@ test.describe('Subgraph Operations', { tag: ['@slow', '@subgraph'] }, () => {
|
||||
})
|
||||
|
||||
// Click breadcrumb to navigate back to parent graph
|
||||
const homeBreadcrumb = comfyPage.page.locator(
|
||||
'.p-breadcrumb-list > :first-child'
|
||||
)
|
||||
const homeBreadcrumb = comfyPage.page.getByRole('link', {
|
||||
// In the subgraph navigation breadcrumbs, the home/top level
|
||||
// breadcrumb is just the workflow name without the folder path
|
||||
name: 'subgraph-with-promoted-text-widget'
|
||||
})
|
||||
await homeBreadcrumb.waitFor({ state: 'visible' })
|
||||
await homeBreadcrumb.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
@@ -1,117 +0,0 @@
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
const WORKFLOW = 'subgraphs/test-values-input-subgraph'
|
||||
const RENAMED_LABEL = 'my_seed'
|
||||
|
||||
/**
|
||||
* Regression test for subgraph input slot rename propagation.
|
||||
*
|
||||
* Renaming a SubgraphInput slot (e.g. "seed") inside the subgraph must
|
||||
* update the promoted widget label shown on the parent SubgraphNode and
|
||||
* keep the widget positioned in the node body (not the header).
|
||||
*
|
||||
* See: https://github.com/Comfy-Org/ComfyUI_frontend/pull/10195
|
||||
*/
|
||||
test.describe(
|
||||
'Subgraph input slot rename propagation',
|
||||
{ tag: ['@subgraph', '@widget'] },
|
||||
() => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
})
|
||||
|
||||
test('Renaming a subgraph input slot updates the widget label on the parent node', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const { page } = comfyPage
|
||||
|
||||
// 1. Load workflow with subgraph containing a promoted seed widget input
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
const sgNode = comfyPage.vueNodes.getNodeLocator('19')
|
||||
await expect(sgNode).toBeVisible()
|
||||
|
||||
// 2. Verify the seed widget is visible on the parent node
|
||||
const seedWidget = sgNode.getByLabel('seed', { exact: true })
|
||||
await expect(seedWidget).toBeVisible()
|
||||
|
||||
// Verify widget is in the node body, not the header
|
||||
const headerBox = await sgNode
|
||||
.locator('[data-testid^="node-header-"]')
|
||||
.boundingBox()
|
||||
const widgetBox = await seedWidget.boundingBox()
|
||||
expect(headerBox).not.toBeNull()
|
||||
expect(widgetBox).not.toBeNull()
|
||||
expect(widgetBox!.y).toBeGreaterThan(headerBox!.y + headerBox!.height)
|
||||
|
||||
// 3. Enter the subgraph and rename the seed slot.
|
||||
// The subgraph IO rename uses canvas.prompt() which requires the
|
||||
// litegraph context menu, so temporarily disable Vue nodes.
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const sgNodeRef = await comfyPage.nodeOps.getNodeRefById('19')
|
||||
await sgNodeRef.navigateIntoSubgraph()
|
||||
|
||||
// Find the seed SubgraphInput slot
|
||||
const seedSlotName = await page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
if (!graph) return null
|
||||
const inputs = (
|
||||
graph as { inputs?: Array<{ name: string; type: string }> }
|
||||
).inputs
|
||||
return inputs?.find((i) => i.name.includes('seed'))?.name ?? null
|
||||
})
|
||||
expect(seedSlotName).not.toBeNull()
|
||||
|
||||
// 4. Right-click the seed input slot and rename it
|
||||
await comfyPage.subgraph.rightClickInputSlot(seedSlotName!)
|
||||
await comfyPage.contextMenu.clickLitegraphMenuItem('Rename Slot')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const dialog = '.graphdialog input'
|
||||
await page.waitForSelector(dialog, { state: 'visible' })
|
||||
await page.fill(dialog, '')
|
||||
await page.fill(dialog, RENAMED_LABEL)
|
||||
await page.keyboard.press('Enter')
|
||||
await page.waitForSelector(dialog, { state: 'hidden' })
|
||||
|
||||
// 5. Navigate back to parent graph and re-enable Vue nodes
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
// 6. Verify the widget label updated to the renamed value
|
||||
const sgNodeAfter = comfyPage.vueNodes.getNodeLocator('19')
|
||||
await expect(sgNodeAfter).toBeVisible()
|
||||
|
||||
const updatedLabel = await page.evaluate(() => {
|
||||
const node = window.app!.canvas.graph!.getNodeById('19')
|
||||
if (!node) return null
|
||||
const w = node.widgets?.find((w: { name: string }) =>
|
||||
w.name.includes('seed')
|
||||
)
|
||||
return w?.label || w?.name || null
|
||||
})
|
||||
expect(updatedLabel).toBe(RENAMED_LABEL)
|
||||
|
||||
// 7. Verify the widget is still in the body, not the header
|
||||
const seedWidgetAfter = sgNodeAfter.getByLabel('seed', { exact: true })
|
||||
await expect(seedWidgetAfter).toBeVisible()
|
||||
|
||||
const headerAfter = await sgNodeAfter
|
||||
.locator('[data-testid^="node-header-"]')
|
||||
.boundingBox()
|
||||
const widgetAfter = await seedWidgetAfter.boundingBox()
|
||||
expect(headerAfter).not.toBeNull()
|
||||
expect(widgetAfter).not.toBeNull()
|
||||
expect(widgetAfter!.y).toBeGreaterThan(
|
||||
headerAfter!.y + headerAfter!.height
|
||||
)
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -1,347 +1,160 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
|
||||
import type { ComfyPage } from '../fixtures/ComfyPage'
|
||||
import type { PromotedWidgetEntry } from '../helpers/promotedWidgets'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { TestIds } from '../fixtures/selectors'
|
||||
import {
|
||||
getPromotedWidgets,
|
||||
getPseudoPreviewWidgets,
|
||||
getNonPreviewPromotedWidgets
|
||||
getPromotedWidgetSnapshot,
|
||||
getPromotedWidgets
|
||||
} from '../helpers/promotedWidgets'
|
||||
|
||||
const domPreviewSelector = '.image-preview'
|
||||
test.describe('Subgraph Lifecycle', { tag: ['@subgraph', '@widget'] }, () => {
|
||||
test('hydrates legacy proxyWidgets deterministically across reload', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-nested-duplicate-ids'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const expectPromotedWidgetsToResolveToInteriorNodes = async (
|
||||
comfyPage: ComfyPage,
|
||||
hostSubgraphNodeId: string,
|
||||
widgets: PromotedWidgetEntry[]
|
||||
) => {
|
||||
const interiorNodeIds = widgets.map(([id]) => id)
|
||||
const results = await comfyPage.page.evaluate(
|
||||
([hostId, ids]) => {
|
||||
const firstSnapshot = await getPromotedWidgetSnapshot(comfyPage, '5')
|
||||
expect(firstSnapshot.proxyWidgets.length).toBeGreaterThan(0)
|
||||
expect(
|
||||
firstSnapshot.proxyWidgets.every(([nodeId]) => nodeId !== '-1')
|
||||
).toBe(true)
|
||||
|
||||
await comfyPage.page.reload()
|
||||
await comfyPage.setup()
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-nested-duplicate-ids'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const secondSnapshot = await getPromotedWidgetSnapshot(comfyPage, '5')
|
||||
expect(secondSnapshot.proxyWidgets).toEqual(firstSnapshot.proxyWidgets)
|
||||
expect(secondSnapshot.widgetNames).toEqual(firstSnapshot.widgetNames)
|
||||
})
|
||||
|
||||
test('promoted view falls back to disconnected placeholder after source widget removal', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const projection = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
const hostNode = graph.getNodeById(Number(hostId))
|
||||
if (!hostNode?.isSubgraphNode()) return ids.map(() => false)
|
||||
const hostNode = graph.getNodeById('11')
|
||||
if (
|
||||
!hostNode ||
|
||||
typeof hostNode.isSubgraphNode !== 'function' ||
|
||||
!hostNode.isSubgraphNode()
|
||||
)
|
||||
throw new Error('Expected host subgraph node 11')
|
||||
|
||||
return ids.map((id) => {
|
||||
const interiorNode = hostNode.subgraph.getNodeById(Number(id))
|
||||
return interiorNode !== null && interiorNode !== undefined
|
||||
})
|
||||
},
|
||||
[hostSubgraphNodeId, interiorNodeIds] as const
|
||||
)
|
||||
const beforeType = hostNode.widgets?.[0]?.type
|
||||
const proxyWidgets = Array.isArray(hostNode.properties?.proxyWidgets)
|
||||
? hostNode.properties.proxyWidgets.filter(
|
||||
(entry): entry is [string, string] =>
|
||||
Array.isArray(entry) &&
|
||||
entry.length === 2 &&
|
||||
typeof entry[0] === 'string' &&
|
||||
typeof entry[1] === 'string'
|
||||
)
|
||||
: []
|
||||
const firstPromotion = proxyWidgets[0]
|
||||
if (!firstPromotion)
|
||||
throw new Error('Expected at least one promoted widget entry')
|
||||
|
||||
for (const exists of results) {
|
||||
expect(exists).toBe(true)
|
||||
}
|
||||
}
|
||||
const [sourceNodeId, sourceWidgetName] = firstPromotion
|
||||
const subgraph = graph.subgraphs.get(hostNode.type)
|
||||
const sourceNode = subgraph?.getNodeById(Number(sourceNodeId))
|
||||
if (!sourceNode?.widgets)
|
||||
throw new Error('Expected promoted source node widget list')
|
||||
|
||||
test.describe(
|
||||
'Subgraph Lifecycle Edge Behaviors',
|
||||
{ tag: ['@subgraph'] },
|
||||
() => {
|
||||
test.describe('Deterministic Hydrate from Serialized proxyWidgets', () => {
|
||||
test('proxyWidgets entries map to real interior node IDs after load', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
sourceNode.widgets = sourceNode.widgets.filter(
|
||||
(widget) => widget.name !== sourceWidgetName
|
||||
)
|
||||
|
||||
const widgets = await getPromotedWidgets(comfyPage, '11')
|
||||
expect(widgets.length).toBeGreaterThan(0)
|
||||
return {
|
||||
beforeType,
|
||||
afterType: hostNode.widgets?.[0]?.type
|
||||
}
|
||||
})
|
||||
|
||||
for (const [interiorNodeId] of widgets) {
|
||||
expect(Number(interiorNodeId)).toBeGreaterThan(0)
|
||||
expect(projection.beforeType).toBe('customtext')
|
||||
expect(projection.afterType).toBe('button')
|
||||
})
|
||||
|
||||
test('unpacking one preview host keeps remaining pseudo-preview promotions resolvable', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-multiple-promoted-previews'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const beforeNode8 = await getPromotedWidgets(comfyPage, '8')
|
||||
expect(beforeNode8).toEqual([['6', '$$canvas-image-preview']])
|
||||
|
||||
const cleanupResult = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
const invalidPseudoEntries = () => {
|
||||
const invalid: string[] = []
|
||||
for (const node of graph.nodes) {
|
||||
if (
|
||||
typeof node.isSubgraphNode !== 'function' ||
|
||||
!node.isSubgraphNode()
|
||||
)
|
||||
continue
|
||||
|
||||
const subgraph = graph.subgraphs.get(node.type)
|
||||
const proxyWidgets = Array.isArray(node.properties?.proxyWidgets)
|
||||
? node.properties.proxyWidgets.filter(
|
||||
(entry): entry is [string, string] =>
|
||||
Array.isArray(entry) &&
|
||||
entry.length === 2 &&
|
||||
typeof entry[0] === 'string' &&
|
||||
typeof entry[1] === 'string'
|
||||
)
|
||||
: []
|
||||
for (const entry of proxyWidgets) {
|
||||
if (entry[1] !== '$$canvas-image-preview') continue
|
||||
|
||||
const sourceNodeId = Number(entry[0])
|
||||
const sourceNode = subgraph?.getNodeById(sourceNodeId)
|
||||
if (!sourceNode) invalid.push(`${node.id}:${entry[0]}`)
|
||||
}
|
||||
}
|
||||
return invalid
|
||||
}
|
||||
|
||||
await expectPromotedWidgetsToResolveToInteriorNodes(
|
||||
comfyPage,
|
||||
'11',
|
||||
widgets
|
||||
)
|
||||
})
|
||||
const before = invalidPseudoEntries()
|
||||
const hostNode = graph.getNodeById('7')
|
||||
if (
|
||||
!hostNode ||
|
||||
typeof hostNode.isSubgraphNode !== 'function' ||
|
||||
!hostNode.isSubgraphNode()
|
||||
)
|
||||
throw new Error('Expected preview host subgraph node 7')
|
||||
|
||||
test('proxyWidgets entries survive double round-trip without drift', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-multiple-promoted-widgets'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
;(
|
||||
graph as unknown as { unpackSubgraph: (node: unknown) => void }
|
||||
).unpackSubgraph(hostNode)
|
||||
|
||||
const initialWidgets = await getPromotedWidgets(comfyPage, '11')
|
||||
expect(initialWidgets.length).toBeGreaterThan(0)
|
||||
await expectPromotedWidgetsToResolveToInteriorNodes(
|
||||
comfyPage,
|
||||
'11',
|
||||
initialWidgets
|
||||
)
|
||||
|
||||
const serialized1 = await comfyPage.page.evaluate(() =>
|
||||
window.app!.graph!.serialize()
|
||||
)
|
||||
await comfyPage.page.evaluate(
|
||||
(workflow: ComfyWorkflowJSON) => window.app!.loadGraphData(workflow),
|
||||
serialized1 as ComfyWorkflowJSON
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const afterFirst = await getPromotedWidgets(comfyPage, '11')
|
||||
await expectPromotedWidgetsToResolveToInteriorNodes(
|
||||
comfyPage,
|
||||
'11',
|
||||
afterFirst
|
||||
)
|
||||
|
||||
const serialized2 = await comfyPage.page.evaluate(() =>
|
||||
window.app!.graph!.serialize()
|
||||
)
|
||||
await comfyPage.page.evaluate(
|
||||
(workflow: ComfyWorkflowJSON) => window.app!.loadGraphData(workflow),
|
||||
serialized2 as ComfyWorkflowJSON
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const afterSecond = await getPromotedWidgets(comfyPage, '11')
|
||||
await expectPromotedWidgetsToResolveToInteriorNodes(
|
||||
comfyPage,
|
||||
'11',
|
||||
afterSecond
|
||||
)
|
||||
|
||||
expect(afterFirst).toEqual(initialWidgets)
|
||||
expect(afterSecond).toEqual(initialWidgets)
|
||||
})
|
||||
|
||||
test('Compressed target_slot (-1) entries are hydrated to real IDs', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-compressed-target-slot'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const widgets = await getPromotedWidgets(comfyPage, '2')
|
||||
expect(widgets.length).toBeGreaterThan(0)
|
||||
|
||||
for (const [interiorNodeId] of widgets) {
|
||||
expect(interiorNodeId).not.toBe('-1')
|
||||
expect(Number(interiorNodeId)).toBeGreaterThan(0)
|
||||
}
|
||||
|
||||
await expectPromotedWidgetsToResolveToInteriorNodes(
|
||||
comfyPage,
|
||||
'2',
|
||||
widgets
|
||||
)
|
||||
})
|
||||
return {
|
||||
before,
|
||||
after: invalidPseudoEntries(),
|
||||
hasNode7: Boolean(graph.getNodeById('7')),
|
||||
hasNode8: Boolean(graph.getNodeById('8'))
|
||||
}
|
||||
})
|
||||
|
||||
test.describe('Placeholder Behavior After Promoted Source Removal', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
})
|
||||
expect(cleanupResult.before).toEqual([])
|
||||
expect(cleanupResult.after).toEqual([])
|
||||
expect(cleanupResult.hasNode7).toBe(false)
|
||||
expect(cleanupResult.hasNode8).toBe(true)
|
||||
|
||||
test('Removing promoted source node inside subgraph falls back to disconnected placeholder on exterior', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const initialWidgets = await getPromotedWidgets(comfyPage, '11')
|
||||
expect(initialWidgets.length).toBeGreaterThan(0)
|
||||
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('11')
|
||||
await subgraphNode.navigateIntoSubgraph()
|
||||
|
||||
const clipNode = await comfyPage.nodeOps.getNodeRefById('10')
|
||||
await clipNode.click('title')
|
||||
await comfyPage.page.keyboard.press('Delete')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
return await comfyPage.page.evaluate(() => {
|
||||
const hostNode = window.app!.canvas.graph!.getNodeById('11')
|
||||
const proxyWidgets = hostNode?.properties?.proxyWidgets
|
||||
return {
|
||||
proxyWidgetCount: Array.isArray(proxyWidgets)
|
||||
? proxyWidgets.length
|
||||
: 0,
|
||||
firstWidgetType: hostNode?.widgets?.[0]?.type
|
||||
}
|
||||
})
|
||||
})
|
||||
.toEqual({
|
||||
proxyWidgetCount: initialWidgets.length,
|
||||
firstWidgetType: 'button'
|
||||
})
|
||||
})
|
||||
|
||||
test('Promoted widget disappears from DOM after interior node deletion', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-promoted-text-widget'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const textarea = comfyPage.page.getByTestId(
|
||||
TestIds.widgets.domWidgetTextarea
|
||||
)
|
||||
await expect(textarea).toBeVisible()
|
||||
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('11')
|
||||
await subgraphNode.navigateIntoSubgraph()
|
||||
|
||||
const clipNode = await comfyPage.nodeOps.getNodeRefById('10')
|
||||
await clipNode.click('title')
|
||||
await comfyPage.page.keyboard.press('Delete')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
|
||||
await expect(
|
||||
comfyPage.page.getByTestId(TestIds.widgets.domWidgetTextarea)
|
||||
).toHaveCount(0)
|
||||
})
|
||||
})
|
||||
|
||||
test.describe('Unpack/Remove Cleanup for Pseudo-Preview Targets', () => {
|
||||
test('Pseudo-preview entries exist in proxyWidgets for preview subgraph', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-preview-node'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const pseudoWidgets = await getPseudoPreviewWidgets(comfyPage, '5')
|
||||
expect(pseudoWidgets.length).toBeGreaterThan(0)
|
||||
expect(
|
||||
pseudoWidgets.some(([, name]) => name === '$$canvas-image-preview')
|
||||
).toBe(true)
|
||||
})
|
||||
|
||||
test('Non-preview widgets coexist with pseudo-preview entries', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-preview-node'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const pseudoWidgets = await getPseudoPreviewWidgets(comfyPage, '5')
|
||||
const nonPreviewWidgets = await getNonPreviewPromotedWidgets(
|
||||
comfyPage,
|
||||
'5'
|
||||
)
|
||||
|
||||
expect(pseudoWidgets.length).toBeGreaterThan(0)
|
||||
expect(nonPreviewWidgets.length).toBeGreaterThan(0)
|
||||
expect(
|
||||
nonPreviewWidgets.some(([, name]) => name === 'filename_prefix')
|
||||
).toBe(true)
|
||||
})
|
||||
|
||||
test('Unpacking subgraph clears pseudo-preview entries from graph', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-preview-node'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const beforePseudo = await getPseudoPreviewWidgets(comfyPage, '5')
|
||||
expect(beforePseudo.length).toBeGreaterThan(0)
|
||||
|
||||
await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
const subgraphNode = graph.nodes.find((n) => n.isSubgraphNode())
|
||||
if (!subgraphNode || !subgraphNode.isSubgraphNode()) return
|
||||
graph.unpackSubgraph(subgraphNode)
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const subgraphNodeCount = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
return graph.nodes.filter((n) => n.isSubgraphNode()).length
|
||||
})
|
||||
expect(subgraphNodeCount).toBe(0)
|
||||
|
||||
await expect
|
||||
.poll(async () => comfyPage.subgraph.countGraphPseudoPreviewEntries())
|
||||
.toBe(0)
|
||||
})
|
||||
|
||||
test('Removing subgraph node clears pseudo-preview DOM elements', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-preview-node'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const beforePseudo = await getPseudoPreviewWidgets(comfyPage, '5')
|
||||
expect(beforePseudo.length).toBeGreaterThan(0)
|
||||
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById('5')
|
||||
expect(await subgraphNode.exists()).toBe(true)
|
||||
|
||||
await subgraphNode.click('title')
|
||||
await comfyPage.page.keyboard.press('Delete')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const nodeExists = await comfyPage.page.evaluate(() => {
|
||||
return !!window.app!.canvas.graph!.getNodeById('5')
|
||||
})
|
||||
expect(nodeExists).toBe(false)
|
||||
|
||||
await expect
|
||||
.poll(async () => comfyPage.subgraph.countGraphPseudoPreviewEntries())
|
||||
.toBe(0)
|
||||
await expect(comfyPage.page.locator(domPreviewSelector)).toHaveCount(0)
|
||||
})
|
||||
|
||||
test('Unpacking one subgraph does not clear sibling pseudo-preview entries', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-multiple-promoted-previews'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const firstNodeBefore = await getPseudoPreviewWidgets(comfyPage, '7')
|
||||
const secondNodeBefore = await getPseudoPreviewWidgets(comfyPage, '8')
|
||||
|
||||
expect(firstNodeBefore.length).toBeGreaterThan(0)
|
||||
expect(secondNodeBefore.length).toBeGreaterThan(0)
|
||||
|
||||
await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.graph!
|
||||
const subgraphNode = graph.getNodeById('7')
|
||||
if (!subgraphNode || !subgraphNode.isSubgraphNode()) return
|
||||
graph.unpackSubgraph(subgraphNode)
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const firstNodeExists = await comfyPage.page.evaluate(() => {
|
||||
return !!window.app!.graph!.getNodeById('7')
|
||||
})
|
||||
expect(firstNodeExists).toBe(false)
|
||||
|
||||
const secondNodeAfter = await getPseudoPreviewWidgets(comfyPage, '8')
|
||||
expect(secondNodeAfter).toEqual(secondNodeBefore)
|
||||
})
|
||||
})
|
||||
}
|
||||
)
|
||||
const afterNode8 = await getPromotedWidgets(comfyPage, '8')
|
||||
expect(afterNode8).toEqual([['6', '$$canvas-image-preview']])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
|
||||
test.describe('Nested subgraph configure order', { tag: ['@subgraph'] }, () => {
|
||||
const WORKFLOW = 'subgraphs/subgraph-nested-duplicate-ids'
|
||||
|
||||
test('Loads without "No link found" or "Failed to resolve legacy -1" console warnings', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const warnings: string[] = []
|
||||
comfyPage.page.on('console', (msg) => {
|
||||
const text = msg.text()
|
||||
if (
|
||||
text.includes('No link found') ||
|
||||
text.includes('Failed to resolve legacy -1') ||
|
||||
text.includes('No inner link found')
|
||||
) {
|
||||
warnings.push(text)
|
||||
}
|
||||
})
|
||||
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
|
||||
expect(warnings).toEqual([])
|
||||
})
|
||||
|
||||
test('All three subgraph levels resolve promoted widgets', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const results = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
const allGraphs = [graph, ...graph.subgraphs.values()]
|
||||
|
||||
return allGraphs.flatMap((g) =>
|
||||
g._nodes
|
||||
.filter(
|
||||
(n) => typeof n.isSubgraphNode === 'function' && n.isSubgraphNode()
|
||||
)
|
||||
.map((hostNode) => {
|
||||
const proxyWidgets = Array.isArray(
|
||||
hostNode.properties?.proxyWidgets
|
||||
)
|
||||
? hostNode.properties.proxyWidgets
|
||||
: []
|
||||
|
||||
const widgetEntries = proxyWidgets
|
||||
.filter(
|
||||
(e: unknown): e is [string, string] =>
|
||||
Array.isArray(e) &&
|
||||
e.length >= 2 &&
|
||||
typeof e[0] === 'string' &&
|
||||
typeof e[1] === 'string'
|
||||
)
|
||||
.map(([interiorNodeId, widgetName]: [string, string]) => {
|
||||
const sg = hostNode.isSubgraphNode() ? hostNode.subgraph : null
|
||||
const interiorNode = sg?.getNodeById(Number(interiorNodeId))
|
||||
return {
|
||||
interiorNodeId,
|
||||
widgetName,
|
||||
resolved: interiorNode !== null && interiorNode !== undefined
|
||||
}
|
||||
})
|
||||
|
||||
return {
|
||||
hostNodeId: String(hostNode.id),
|
||||
widgetEntries
|
||||
}
|
||||
})
|
||||
)
|
||||
})
|
||||
|
||||
expect(
|
||||
results.length,
|
||||
'Should have subgraph host nodes at multiple nesting levels'
|
||||
).toBeGreaterThanOrEqual(2)
|
||||
|
||||
for (const { hostNodeId, widgetEntries } of results) {
|
||||
expect(
|
||||
widgetEntries.length,
|
||||
`Host node ${hostNodeId} should have promoted widgets`
|
||||
).toBeGreaterThan(0)
|
||||
|
||||
for (const { interiorNodeId, widgetName, resolved } of widgetEntries) {
|
||||
expect(interiorNodeId).not.toBe('-1')
|
||||
expect(Number(interiorNodeId)).toBeGreaterThan(0)
|
||||
expect(widgetName).toBeTruthy()
|
||||
expect(
|
||||
resolved,
|
||||
`Widget "${widgetName}" (interior node ${interiorNodeId}) on host ${hostNodeId} should resolve`
|
||||
).toBe(true)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
test('Prompt execution succeeds without 400 error', async ({ comfyPage }) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const responsePromise = comfyPage.page.waitForResponse('**/api/prompt')
|
||||
|
||||
await comfyPage.command.executeCommand('Comfy.QueuePrompt')
|
||||
|
||||
const response = await responsePromise
|
||||
expect(response.status()).not.toBe(400)
|
||||
})
|
||||
})
|
||||
@@ -1,141 +0,0 @@
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
const WORKFLOW = 'subgraphs/nested-duplicate-widget-names'
|
||||
const PROMOTED_BORDER_CLASS = 'ring-component-node-widget-promoted'
|
||||
|
||||
/**
|
||||
* Regression tests for nested subgraph promotion where multiple interior
|
||||
* nodes share the same widget name (e.g. two CLIPTextEncode nodes both
|
||||
* with a "text" widget).
|
||||
*
|
||||
* The inner subgraph (node 3) promotes both ["1","text"] and ["2","text"].
|
||||
* The outer subgraph (node 4) promotes through node 3 using identity
|
||||
* disambiguation (optional sourceNodeId in the promotion entry).
|
||||
*
|
||||
* See: https://github.com/Comfy-Org/ComfyUI_frontend/pull/10123#discussion_r2956230977
|
||||
*/
|
||||
test.describe(
|
||||
'Nested subgraph duplicate widget names',
|
||||
{ tag: ['@subgraph', '@widget'] },
|
||||
() => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Disabled')
|
||||
})
|
||||
|
||||
test('Inner subgraph node has both text widgets promoted', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const nonPreview = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
const outerNode = graph.getNodeById('4')
|
||||
if (
|
||||
!outerNode ||
|
||||
typeof outerNode.isSubgraphNode !== 'function' ||
|
||||
!outerNode.isSubgraphNode()
|
||||
) {
|
||||
return []
|
||||
}
|
||||
|
||||
const innerSubgraphNode = outerNode.subgraph.getNodeById(3)
|
||||
if (!innerSubgraphNode) return []
|
||||
|
||||
return ((innerSubgraphNode.properties?.proxyWidgets ?? []) as unknown[])
|
||||
.filter(
|
||||
(entry): entry is [string, string] =>
|
||||
Array.isArray(entry) &&
|
||||
entry.length >= 2 &&
|
||||
typeof entry[0] === 'string' &&
|
||||
typeof entry[1] === 'string' &&
|
||||
!entry[1].startsWith('$$')
|
||||
)
|
||||
.map(
|
||||
([nodeId, widgetName]) => [nodeId, widgetName] as [string, string]
|
||||
)
|
||||
})
|
||||
|
||||
expect(nonPreview).toEqual([
|
||||
['1', 'text'],
|
||||
['2', 'text']
|
||||
])
|
||||
})
|
||||
|
||||
test('Promoted widget values from both inner CLIPTextEncode nodes are distinguishable', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const widgetValues = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
const outerNode = graph.getNodeById('4')
|
||||
if (
|
||||
!outerNode ||
|
||||
typeof outerNode.isSubgraphNode !== 'function' ||
|
||||
!outerNode.isSubgraphNode()
|
||||
) {
|
||||
return []
|
||||
}
|
||||
|
||||
const innerSubgraphNode = outerNode.subgraph.getNodeById(3)
|
||||
if (!innerSubgraphNode) return []
|
||||
|
||||
return (innerSubgraphNode.widgets ?? []).map((w) => ({
|
||||
name: w.name,
|
||||
value: w.value
|
||||
}))
|
||||
})
|
||||
|
||||
const textWidgets = widgetValues.filter((w) => w.name.startsWith('text'))
|
||||
expect(textWidgets).toHaveLength(2)
|
||||
|
||||
const values = textWidgets.map((w) => w.value)
|
||||
expect(values).toContain('11111111111')
|
||||
expect(values).toContain('22222222222')
|
||||
})
|
||||
|
||||
test.describe('Promoted border styling in Vue mode', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
})
|
||||
|
||||
test('Intermediate subgraph widgets get promoted border, outermost does not', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(WORKFLOW)
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
// Node 4 is the outer SubgraphNode at root level.
|
||||
// Its widgets are not promoted further (no parent subgraph),
|
||||
// so none of its widget wrappers should carry the promoted ring.
|
||||
const outerNode = comfyPage.vueNodes.getNodeLocator('4')
|
||||
await expect(outerNode).toBeVisible()
|
||||
|
||||
const outerPromotedRings = outerNode.locator(
|
||||
`.${PROMOTED_BORDER_CLASS}`
|
||||
)
|
||||
await expect(outerPromotedRings).toHaveCount(0)
|
||||
|
||||
// Navigate into the outer subgraph (node 4) to reach node 3
|
||||
await comfyPage.vueNodes.enterSubgraph('4')
|
||||
await comfyPage.nextFrame()
|
||||
await comfyPage.vueNodes.waitForNodes()
|
||||
|
||||
// Node 3 is the intermediate SubgraphNode whose "text" widgets
|
||||
// are promoted up to the outer subgraph (node 4).
|
||||
// Its widget wrappers should carry the promoted border ring.
|
||||
const intermediateNode = comfyPage.vueNodes.getNodeLocator('3')
|
||||
await expect(intermediateNode).toBeVisible()
|
||||
|
||||
const intermediatePromotedRings = intermediateNode.locator(
|
||||
`.${PROMOTED_BORDER_CLASS}`
|
||||
)
|
||||
await expect(intermediatePromotedRings).toHaveCount(1)
|
||||
})
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -73,59 +73,5 @@ test.describe(
|
||||
expect(progressAfter).toBeUndefined()
|
||||
}).toPass({ timeout: 2_000 })
|
||||
})
|
||||
|
||||
test('Stale progress is cleared when switching workflows while inside subgraph', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('subgraphs/basic-subgraph')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const subgraphNodeId = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
const subgraphNode = graph.nodes.find(
|
||||
(n) => typeof n.isSubgraphNode === 'function' && n.isSubgraphNode()
|
||||
)
|
||||
return subgraphNode ? String(subgraphNode.id) : null
|
||||
})
|
||||
expect(subgraphNodeId).not.toBeNull()
|
||||
|
||||
await comfyPage.page.evaluate((nodeId) => {
|
||||
const node = window.app!.canvas.graph!.getNodeById(nodeId)!
|
||||
node.progress = 0.7
|
||||
}, subgraphNodeId!)
|
||||
|
||||
const subgraphNode = await comfyPage.nodeOps.getNodeRefById(
|
||||
subgraphNodeId!
|
||||
)
|
||||
await subgraphNode.navigateIntoSubgraph()
|
||||
|
||||
const inSubgraph = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
return !!graph && 'inputNode' in graph
|
||||
})
|
||||
expect(inSubgraph).toBe(true)
|
||||
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.workflow.loadWorkflow('subgraphs/basic-subgraph')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await expect(async () => {
|
||||
const subgraphProgressState = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph!
|
||||
const subgraphNode = graph.nodes.find(
|
||||
(n) => typeof n.isSubgraphNode === 'function' && n.isSubgraphNode()
|
||||
)
|
||||
if (!subgraphNode) {
|
||||
return { exists: false, progress: null }
|
||||
}
|
||||
|
||||
return { exists: true, progress: subgraphNode.progress }
|
||||
})
|
||||
expect(subgraphProgressState.exists).toBe(true)
|
||||
expect(subgraphProgressState.progress).toBeUndefined()
|
||||
}).toPass({ timeout: 5_000 })
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
@@ -2,6 +2,7 @@ import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
|
||||
|
||||
import type { ComfyPage } from '../fixtures/ComfyPage'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
import { TestIds } from '../fixtures/selectors'
|
||||
import { fitToViewInstant } from '../helpers/fitToView'
|
||||
@@ -11,6 +12,25 @@ import {
|
||||
getPromotedWidgets
|
||||
} from '../helpers/promotedWidgets'
|
||||
|
||||
/**
|
||||
* Check whether we're currently in a subgraph.
|
||||
*/
|
||||
async function isInSubgraph(comfyPage: ComfyPage): Promise<boolean> {
|
||||
return comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
return !!graph && 'inputNode' in graph
|
||||
})
|
||||
}
|
||||
|
||||
async function exitSubgraphToParent(comfyPage: ComfyPage): Promise<void> {
|
||||
await comfyPage.page.evaluate(() => {
|
||||
const canvas = window.app!.canvas
|
||||
if (!canvas.graph) return
|
||||
canvas.setGraph(canvas.graph.rootGraph)
|
||||
})
|
||||
await comfyPage.nextFrame()
|
||||
}
|
||||
|
||||
test.describe(
|
||||
'Subgraph Widget Promotion',
|
||||
{ tag: ['@subgraph', '@widget'] },
|
||||
@@ -170,7 +190,7 @@ test.describe(
|
||||
await comfyPage.vueNodes.enterSubgraph('11')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
expect(await comfyPage.subgraph.isInSubgraph()).toBe(true)
|
||||
expect(await isInSubgraph(comfyPage)).toBe(true)
|
||||
})
|
||||
|
||||
test('Multiple promoted widgets render on SubgraphNode in Vue mode', async ({
|
||||
@@ -242,7 +262,7 @@ test.describe(
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Navigate back to parent graph
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await exitSubgraphToParent(comfyPage)
|
||||
|
||||
// Promoted textarea on SubgraphNode should have the same value
|
||||
const promotedTextarea = comfyPage.page.getByTestId(
|
||||
@@ -276,7 +296,7 @@ test.describe(
|
||||
)
|
||||
await expect(interiorTextarea).toHaveValue(testContent)
|
||||
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await exitSubgraphToParent(comfyPage)
|
||||
|
||||
const promotedTextarea = comfyPage.page.getByTestId(
|
||||
TestIds.widgets.domWidgetTextarea
|
||||
@@ -322,7 +342,7 @@ test.describe(
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Navigate back to parent
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await exitSubgraphToParent(comfyPage)
|
||||
|
||||
// SubgraphNode should now have the promoted widget
|
||||
const widgetCount = await getPromotedWidgetCount(comfyPage, '2')
|
||||
@@ -357,7 +377,7 @@ test.describe(
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Navigate back and verify promotion took effect
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await exitSubgraphToParent(comfyPage)
|
||||
await fitToViewInstant(comfyPage)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
@@ -388,7 +408,7 @@ test.describe(
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Navigate back to parent
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await exitSubgraphToParent(comfyPage)
|
||||
|
||||
// SubgraphNode should have fewer widgets
|
||||
const finalWidgetCount = await getPromotedWidgetCount(comfyPage, '2')
|
||||
@@ -544,30 +564,6 @@ test.describe(
|
||||
expect(widgetCount).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
test('Multi-link input representative stays stable through save/reload', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-with-multiple-promoted-widgets'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const beforeSnapshot = await getPromotedWidgets(comfyPage, '11')
|
||||
expect(beforeSnapshot.length).toBeGreaterThan(0)
|
||||
|
||||
const serialized = await comfyPage.page.evaluate(() => {
|
||||
return window.app!.graph!.serialize()
|
||||
})
|
||||
|
||||
await comfyPage.page.evaluate((workflow: ComfyWorkflowJSON) => {
|
||||
return window.app!.loadGraphData(workflow)
|
||||
}, serialized as ComfyWorkflowJSON)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const afterSnapshot = await getPromotedWidgets(comfyPage, '11')
|
||||
expect(afterSnapshot).toEqual(beforeSnapshot)
|
||||
})
|
||||
|
||||
test('Cloning a subgraph node keeps promoted widget entries on original and clone', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
@@ -725,44 +721,6 @@ test.describe(
|
||||
expect(nodeExists).toBe(false)
|
||||
})
|
||||
|
||||
test('Nested promoted widget entries reflect interior changes after slot removal', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
|
||||
|
||||
await comfyPage.workflow.loadWorkflow(
|
||||
'subgraphs/subgraph-nested-promotion'
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const initialNames = await getPromotedWidgetNames(comfyPage, '5')
|
||||
expect(initialNames.length).toBeGreaterThan(0)
|
||||
|
||||
const outerSubgraph = await comfyPage.nodeOps.getNodeRefById('5')
|
||||
await outerSubgraph.navigateIntoSubgraph()
|
||||
|
||||
const removedSlotName = await comfyPage.page.evaluate(() => {
|
||||
const graph = window.app!.canvas.graph
|
||||
if (!graph || !('inputNode' in graph)) return null
|
||||
return graph.inputs?.[0]?.name ?? null
|
||||
})
|
||||
expect(removedSlotName).not.toBeNull()
|
||||
|
||||
await comfyPage.subgraph.rightClickInputSlot()
|
||||
await comfyPage.contextMenu.clickLitegraphMenuItem('Remove Slot')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
|
||||
const finalNames = await getPromotedWidgetNames(comfyPage, '5')
|
||||
const expectedNames = [...initialNames]
|
||||
const removedIndex = expectedNames.indexOf(removedSlotName!)
|
||||
expect(removedIndex).toBeGreaterThanOrEqual(0)
|
||||
expectedNames.splice(removedIndex, 1)
|
||||
|
||||
expect(finalNames).toEqual(expectedNames)
|
||||
})
|
||||
|
||||
test('Removing I/O slot removes associated promoted widget', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
@@ -785,7 +743,15 @@ test.describe(
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Navigate back via breadcrumb
|
||||
await comfyPage.subgraph.exitViaBreadcrumb()
|
||||
await comfyPage.page
|
||||
.getByTestId(TestIds.breadcrumb.subgraph)
|
||||
.waitFor({ state: 'visible', timeout: 5000 })
|
||||
const homeBreadcrumb = comfyPage.page.getByRole('link', {
|
||||
name: 'subgraph-with-promoted-text-widget'
|
||||
})
|
||||
await homeBreadcrumb.waitFor({ state: 'visible' })
|
||||
await homeBreadcrumb.click()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Widget count should be reduced
|
||||
const finalWidgetCount = await getPromotedWidgetCount(comfyPage, '11')
|
||||
|
||||
@@ -1,132 +0,0 @@
|
||||
import { readFileSync } from 'fs'
|
||||
import { resolve } from 'path'
|
||||
|
||||
import { expect } from '@playwright/test'
|
||||
|
||||
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
|
||||
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
|
||||
interface SlotMeasurement {
|
||||
key: string
|
||||
offsetX: number
|
||||
offsetY: number
|
||||
}
|
||||
|
||||
interface NodeSlotData {
|
||||
nodeId: string
|
||||
isSubgraph: boolean
|
||||
nodeW: number
|
||||
nodeH: number
|
||||
slots: SlotMeasurement[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Regression test for link misalignment on SubgraphNodes when loading
|
||||
* workflows with workflowRendererVersion: "LG".
|
||||
*
|
||||
* Root cause: ensureCorrectLayoutScale scales nodes by 1.2x for LG workflows,
|
||||
* and fitView() updates lgCanvas.ds immediately. The Vue TransformPane's CSS
|
||||
* transform lags by a frame, causing clientPosToCanvasPos to produce wrong
|
||||
* slot offsets. The fix uses DOM-relative measurement instead.
|
||||
*/
|
||||
test.describe(
|
||||
'Subgraph slot alignment after LG layout scale',
|
||||
{ tag: ['@subgraph', '@canvas'] },
|
||||
() => {
|
||||
test('slot positions stay within node bounds after loading LG workflow', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const SLOT_BOUNDS_MARGIN = 20
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
|
||||
const workflowPath = resolve(
|
||||
import.meta.dirname,
|
||||
'../assets/subgraphs/basic-subgraph.json'
|
||||
)
|
||||
const workflow = JSON.parse(
|
||||
readFileSync(workflowPath, 'utf-8')
|
||||
) as ComfyWorkflowJSON
|
||||
workflow.extra = {
|
||||
...workflow.extra,
|
||||
workflowRendererVersion: 'LG'
|
||||
}
|
||||
|
||||
await comfyPage.page.evaluate(
|
||||
(wf) =>
|
||||
window.app!.loadGraphData(wf as ComfyWorkflowJSON, true, true, null, {
|
||||
openSource: 'template'
|
||||
}),
|
||||
workflow
|
||||
)
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Wait for slot elements to appear in DOM
|
||||
await comfyPage.page.locator('[data-slot-key]').first().waitFor()
|
||||
|
||||
const result: NodeSlotData[] = await comfyPage.page.evaluate(() => {
|
||||
const nodes = window.app!.graph._nodes
|
||||
const slotData: NodeSlotData[] = []
|
||||
|
||||
for (const node of nodes) {
|
||||
const nodeId = String(node.id)
|
||||
const nodeEl = document.querySelector(
|
||||
`[data-node-id="${nodeId}"]`
|
||||
) as HTMLElement | null
|
||||
if (!nodeEl) continue
|
||||
|
||||
const slotEls = nodeEl.querySelectorAll('[data-slot-key]')
|
||||
if (slotEls.length === 0) continue
|
||||
|
||||
const slots: SlotMeasurement[] = []
|
||||
|
||||
const nodeRect = nodeEl.getBoundingClientRect()
|
||||
for (const slotEl of slotEls) {
|
||||
const slotRect = slotEl.getBoundingClientRect()
|
||||
const slotKey = (slotEl as HTMLElement).dataset.slotKey ?? 'unknown'
|
||||
slots.push({
|
||||
key: slotKey,
|
||||
offsetX: slotRect.left + slotRect.width / 2 - nodeRect.left,
|
||||
offsetY: slotRect.top + slotRect.height / 2 - nodeRect.top
|
||||
})
|
||||
}
|
||||
|
||||
slotData.push({
|
||||
nodeId,
|
||||
isSubgraph: !!node.isSubgraphNode?.(),
|
||||
nodeW: nodeRect.width,
|
||||
nodeH: nodeRect.height,
|
||||
slots
|
||||
})
|
||||
}
|
||||
|
||||
return slotData
|
||||
})
|
||||
|
||||
const subgraphNodes = result.filter((n) => n.isSubgraph)
|
||||
expect(subgraphNodes.length).toBeGreaterThan(0)
|
||||
|
||||
for (const node of subgraphNodes) {
|
||||
for (const slot of node.slots) {
|
||||
expect(
|
||||
slot.offsetX,
|
||||
`Slot ${slot.key} on node ${node.nodeId}: X offset ${slot.offsetX} outside node width ${node.nodeW}`
|
||||
).toBeGreaterThanOrEqual(-SLOT_BOUNDS_MARGIN)
|
||||
expect(
|
||||
slot.offsetX,
|
||||
`Slot ${slot.key} on node ${node.nodeId}: X offset ${slot.offsetX} outside node width ${node.nodeW}`
|
||||
).toBeLessThanOrEqual(node.nodeW + SLOT_BOUNDS_MARGIN)
|
||||
|
||||
expect(
|
||||
slot.offsetY,
|
||||
`Slot ${slot.key} on node ${node.nodeId}: Y offset ${slot.offsetY} outside node height ${node.nodeH}`
|
||||
).toBeGreaterThanOrEqual(-SLOT_BOUNDS_MARGIN)
|
||||
expect(
|
||||
slot.offsetY,
|
||||
`Slot ${slot.key} on node ${node.nodeId}: Y offset ${slot.offsetY} outside node height ${node.nodeH}`
|
||||
).toBeLessThanOrEqual(node.nodeH + SLOT_BOUNDS_MARGIN)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -206,31 +206,6 @@ test.describe('Templates', { tag: ['@slow', '@workflow'] }, () => {
|
||||
await expect(nav).toBeVisible() // Nav should be visible at tablet size
|
||||
})
|
||||
|
||||
test(
|
||||
'select components in filter bar render correctly',
|
||||
{ tag: '@screenshot' },
|
||||
async ({ comfyPage }) => {
|
||||
await comfyPage.command.executeCommand('Comfy.BrowseTemplates')
|
||||
await expect(comfyPage.templates.content).toBeVisible()
|
||||
|
||||
// Wait for filter bar select components to render
|
||||
const dialog = comfyPage.page.getByRole('dialog')
|
||||
const sortBySelect = dialog.getByRole('combobox', { name: /Sort/ })
|
||||
await expect(sortBySelect).toBeVisible()
|
||||
|
||||
// Screenshot the filter bar containing MultiSelect and SingleSelect
|
||||
const filterBar = sortBySelect.locator(
|
||||
'xpath=ancestor::div[contains(@class, "justify-between")]'
|
||||
)
|
||||
await expect(filterBar).toHaveScreenshot(
|
||||
'template-filter-bar-select-components.png',
|
||||
{
|
||||
mask: [comfyPage.page.locator('.p-toast')]
|
||||
}
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
test(
|
||||
'template cards descriptions adjust height dynamically',
|
||||
{ tag: '@screenshot' },
|
||||
|
||||
|
Before Width: | Height: | Size: 4.4 KiB |
|
Before Width: | Height: | Size: 101 KiB After Width: | Height: | Size: 103 KiB |
@@ -2,116 +2,9 @@ import {
|
||||
comfyExpect as expect,
|
||||
comfyPageFixture as test
|
||||
} from '../../../fixtures/ComfyPage'
|
||||
import type { ComfyPage } from '../../../fixtures/ComfyPage'
|
||||
|
||||
const CREATE_GROUP_HOTKEY = 'Control+g'
|
||||
|
||||
type NodeGroupCenteringError = {
|
||||
horizontal: number
|
||||
vertical: number
|
||||
}
|
||||
|
||||
type NodeGroupCenteringErrors = {
|
||||
innerGroup: NodeGroupCenteringError
|
||||
outerGroup: NodeGroupCenteringError
|
||||
}
|
||||
|
||||
const LEGACY_VUE_CENTERING_BASELINE: NodeGroupCenteringErrors = {
|
||||
innerGroup: {
|
||||
horizontal: 16.308832840862777,
|
||||
vertical: 17.390899314547084
|
||||
},
|
||||
outerGroup: {
|
||||
horizontal: 20.30164329441476,
|
||||
vertical: 42.196324096481476
|
||||
}
|
||||
} as const
|
||||
|
||||
const CENTERING_TOLERANCE = {
|
||||
innerGroup: 6,
|
||||
outerGroup: 12
|
||||
} as const
|
||||
|
||||
function expectWithinBaseline(
|
||||
actual: number,
|
||||
baseline: number,
|
||||
tolerance: number
|
||||
) {
|
||||
expect(Math.abs(actual - baseline)).toBeLessThan(tolerance)
|
||||
}
|
||||
|
||||
async function getNodeGroupCenteringErrors(
|
||||
comfyPage: ComfyPage
|
||||
): Promise<NodeGroupCenteringErrors> {
|
||||
return comfyPage.page.evaluate(() => {
|
||||
type GraphNode = {
|
||||
id: number | string
|
||||
pos: ReadonlyArray<number>
|
||||
}
|
||||
type GraphGroup = {
|
||||
title: string
|
||||
pos: ReadonlyArray<number>
|
||||
size: ReadonlyArray<number>
|
||||
}
|
||||
|
||||
const app = window.app!
|
||||
const node = app.graph.nodes[0] as GraphNode | undefined
|
||||
|
||||
if (!node) {
|
||||
throw new Error('Expected a node in the loaded workflow')
|
||||
}
|
||||
|
||||
const nodeElement = document.querySelector<HTMLElement>(
|
||||
`[data-node-id="${node.id}"]`
|
||||
)
|
||||
|
||||
if (!nodeElement) {
|
||||
throw new Error(`Vue node element not found for node ${node.id}`)
|
||||
}
|
||||
|
||||
const groups = app.graph.groups as GraphGroup[]
|
||||
const innerGroup = groups.find((group) => group.title === 'Inner Group')
|
||||
const outerGroup = groups.find((group) => group.title === 'Outer Group')
|
||||
|
||||
if (!innerGroup || !outerGroup) {
|
||||
throw new Error('Expected both Inner Group and Outer Group in graph')
|
||||
}
|
||||
|
||||
const nodeRect = nodeElement.getBoundingClientRect()
|
||||
|
||||
const getCenteringError = (group: GraphGroup): NodeGroupCenteringError => {
|
||||
const [groupStartX, groupStartY] = app.canvasPosToClientPos([
|
||||
group.pos[0],
|
||||
group.pos[1]
|
||||
])
|
||||
const [groupEndX, groupEndY] = app.canvasPosToClientPos([
|
||||
group.pos[0] + group.size[0],
|
||||
group.pos[1] + group.size[1]
|
||||
])
|
||||
|
||||
const groupLeft = Math.min(groupStartX, groupEndX)
|
||||
const groupRight = Math.max(groupStartX, groupEndX)
|
||||
const groupTop = Math.min(groupStartY, groupEndY)
|
||||
const groupBottom = Math.max(groupStartY, groupEndY)
|
||||
|
||||
const leftGap = nodeRect.left - groupLeft
|
||||
const rightGap = groupRight - nodeRect.right
|
||||
const topGap = nodeRect.top - groupTop
|
||||
const bottomGap = groupBottom - nodeRect.bottom
|
||||
|
||||
return {
|
||||
horizontal: Math.abs(leftGap - rightGap),
|
||||
vertical: Math.abs(topGap - bottomGap)
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
innerGroup: getCenteringError(innerGroup),
|
||||
outerGroup: getCenteringError(outerGroup)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
test.describe('Vue Node Groups', { tag: '@screenshot' }, () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
|
||||
@@ -181,45 +74,4 @@ test.describe('Vue Node Groups', { tag: '@screenshot' }, () => {
|
||||
expect(finalOffsetY).toBeCloseTo(initialOffsetY, 0)
|
||||
}).toPass({ timeout: 5000 })
|
||||
})
|
||||
|
||||
test('should keep groups aligned after loading legacy Vue workflows', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
await comfyPage.workflow.loadWorkflow('groups/nested-groups-1-inner-node')
|
||||
await comfyPage.vueNodes.waitForNodes(1)
|
||||
|
||||
const workflowRendererVersion = await comfyPage.page.evaluate(() => {
|
||||
const extra = window.app!.graph.extra as
|
||||
| { workflowRendererVersion?: string }
|
||||
| undefined
|
||||
return extra?.workflowRendererVersion
|
||||
})
|
||||
|
||||
expect(workflowRendererVersion).toMatch(/^Vue/)
|
||||
|
||||
await expect(async () => {
|
||||
const centeringErrors = await getNodeGroupCenteringErrors(comfyPage)
|
||||
|
||||
expectWithinBaseline(
|
||||
centeringErrors.innerGroup.horizontal,
|
||||
LEGACY_VUE_CENTERING_BASELINE.innerGroup.horizontal,
|
||||
CENTERING_TOLERANCE.innerGroup
|
||||
)
|
||||
expectWithinBaseline(
|
||||
centeringErrors.innerGroup.vertical,
|
||||
LEGACY_VUE_CENTERING_BASELINE.innerGroup.vertical,
|
||||
CENTERING_TOLERANCE.innerGroup
|
||||
)
|
||||
expectWithinBaseline(
|
||||
centeringErrors.outerGroup.horizontal,
|
||||
LEGACY_VUE_CENTERING_BASELINE.outerGroup.horizontal,
|
||||
CENTERING_TOLERANCE.outerGroup
|
||||
)
|
||||
expectWithinBaseline(
|
||||
centeringErrors.outerGroup.vertical,
|
||||
LEGACY_VUE_CENTERING_BASELINE.outerGroup.vertical,
|
||||
CENTERING_TOLERANCE.outerGroup
|
||||
)
|
||||
}).toPass({ timeout: 5000 })
|
||||
})
|
||||
})
|
||||
|
||||
|
Before Width: | Height: | Size: 112 KiB After Width: | Height: | Size: 113 KiB |
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 57 KiB |
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 88 KiB |
@@ -47,46 +47,6 @@ test.describe('Vue Node Moving', () => {
|
||||
}
|
||||
)
|
||||
|
||||
test('should not move node when pointer moves less than drag threshold', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const headerPos = await getLoadCheckpointHeaderPos(comfyPage)
|
||||
|
||||
// Move only 2px — below the 3px drag threshold in useNodePointerInteractions
|
||||
await comfyPage.page.mouse.move(headerPos.x, headerPos.y)
|
||||
await comfyPage.page.mouse.down()
|
||||
await comfyPage.page.mouse.move(headerPos.x + 2, headerPos.y + 1, {
|
||||
steps: 5
|
||||
})
|
||||
await comfyPage.page.mouse.up()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const afterPos = await getLoadCheckpointHeaderPos(comfyPage)
|
||||
expect(afterPos.x).toBeCloseTo(headerPos.x, 0)
|
||||
expect(afterPos.y).toBeCloseTo(headerPos.y, 0)
|
||||
|
||||
// The small movement should have selected the node, not dragged it
|
||||
expect(await comfyPage.vueNodes.getSelectedNodeCount()).toBe(1)
|
||||
})
|
||||
|
||||
test('should move node when pointer moves beyond drag threshold', async ({
|
||||
comfyPage
|
||||
}) => {
|
||||
const headerPos = await getLoadCheckpointHeaderPos(comfyPage)
|
||||
|
||||
// Move 50px — well beyond the 3px drag threshold
|
||||
await comfyPage.page.mouse.move(headerPos.x, headerPos.y)
|
||||
await comfyPage.page.mouse.down()
|
||||
await comfyPage.page.mouse.move(headerPos.x + 50, headerPos.y + 50, {
|
||||
steps: 20
|
||||
})
|
||||
await comfyPage.page.mouse.up()
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
const afterPos = await getLoadCheckpointHeaderPos(comfyPage)
|
||||
await expectPosChanged(headerPos, afterPos)
|
||||
})
|
||||
|
||||
test(
|
||||
'@mobile should allow moving nodes by dragging on touch devices',
|
||||
{ tag: '@screenshot' },
|
||||
|
||||
@@ -2,7 +2,6 @@ import {
|
||||
comfyExpect as expect,
|
||||
comfyPageFixture as test
|
||||
} from '../../../../fixtures/ComfyPage'
|
||||
import { TestIds } from '../../../../fixtures/selectors'
|
||||
|
||||
test.describe('Vue Upload Widgets', () => {
|
||||
test.beforeEach(async ({ comfyPage }) => {
|
||||
@@ -20,14 +19,10 @@ test.describe('Vue Upload Widgets', () => {
|
||||
).not.toBeVisible()
|
||||
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.getByTestId(TestIds.errors.imageLoadError).count()
|
||||
)
|
||||
.poll(() => comfyPage.page.getByText('Error loading image').count())
|
||||
.toBeGreaterThan(0)
|
||||
await expect
|
||||
.poll(() =>
|
||||
comfyPage.page.getByTestId(TestIds.errors.videoLoadError).count()
|
||||
)
|
||||
.poll(() => comfyPage.page.getByText('Error loading video').count())
|
||||
.toBeGreaterThan(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -46,16 +46,4 @@ test.describe('Vue Multiline String Widget', () => {
|
||||
|
||||
await expect(textarea).toHaveValue('Keep me around')
|
||||
})
|
||||
test('should use native context menu when focused', async ({ comfyPage }) => {
|
||||
const textarea = getFirstMultilineStringWidget(comfyPage)
|
||||
const vueContextMenu = comfyPage.page.locator('.p-contextmenu')
|
||||
|
||||
await textarea.focus()
|
||||
await textarea.click({ button: 'right' })
|
||||
await expect(vueContextMenu).not.toBeVisible()
|
||||
await textarea.blur()
|
||||
|
||||
await textarea.click({ button: 'right' })
|
||||
await expect(vueContextMenu).toBeVisible()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
# QA Pipeline Model Selection
|
||||
|
||||
## Current Configuration
|
||||
|
||||
| Script | Role | Model | Why |
|
||||
| --------------------- | -------------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------- |
|
||||
| `qa-analyze-pr.ts` | PR/issue analysis, QA guide generation | `gemini-3.1-pro-preview` | Needs deep reasoning over PR diffs, screenshots, and issue threads |
|
||||
| `qa-record.ts` | Playwright step generation | `gemini-3.1-pro-preview` | Step quality is critical — must understand ComfyUI's canvas UI and produce precise action sequences |
|
||||
| `qa-video-review.ts` | Video comparison review | `gemini-3-flash-preview` | Video analysis with structured output; flash is sufficient and faster |
|
||||
| `qa-generate-test.ts` | Regression test generation | `gemini-3-flash-preview` | Code generation from QA reports; flash handles this well |
|
||||
|
||||
## Model Comparison
|
||||
|
||||
### Gemini 3.1 Pro vs GPT-5.4
|
||||
|
||||
| | Gemini 3.1 Pro Preview | GPT-5.4 |
|
||||
| ----------------- | ---------------------- | ----------------- |
|
||||
| Context window | 1M tokens | 1M tokens |
|
||||
| Max output | 65K tokens | 128K tokens |
|
||||
| Video input | Yes | No |
|
||||
| Image input | Yes | Yes |
|
||||
| Audio input | Yes | No |
|
||||
| Pricing (input) | $2/1M tokens | $2.50/1M tokens |
|
||||
| Pricing (output) | $12/1M tokens | $15/1M tokens |
|
||||
| Function calling | Yes | Yes |
|
||||
| Code execution | Yes | Yes (interpreter) |
|
||||
| Structured output | Yes | Yes |
|
||||
|
||||
**Why Gemini over GPT for QA:**
|
||||
|
||||
- Native video understanding (can review recordings directly)
|
||||
- Lower cost at comparable quality
|
||||
- Native multimodal input (screenshots, videos, audio from issue threads)
|
||||
- Better price/performance for high-volume CI usage
|
||||
|
||||
### Gemini 3 Flash vs GPT-5.4 Mini
|
||||
|
||||
| | Gemini 3 Flash Preview | GPT-5.4 Mini |
|
||||
| ---------------- | ---------------------- | --------------- |
|
||||
| Context window | 1M tokens | 1M tokens |
|
||||
| Pricing (input) | $0.50/1M tokens | $0.40/1M tokens |
|
||||
| Pricing (output) | $3/1M tokens | $1.60/1M tokens |
|
||||
| Video input | Yes | No |
|
||||
|
||||
**Why Gemini Flash for video review:**
|
||||
|
||||
- Video input support is required — GPT models cannot process video files
|
||||
- Good enough quality for structured comparison reports
|
||||
|
||||
## Upgrade History
|
||||
|
||||
| Date | Change | Reason |
|
||||
| ---------- | ---------------------------------------------------------------- | -------------------------------------------------------------------------- |
|
||||
| 2026-03-24 | `gemini-2.5-flash` → `gemini-3.1-pro-preview` (record) | Shallow step generation; pro model needed for complex ComfyUI interactions |
|
||||
| 2026-03-24 | `gemini-2.5-pro` → `gemini-3.1-pro-preview` (analyze) | Keep analysis on latest pro |
|
||||
| 2026-03-24 | `gemini-2.5-flash` → `gemini-3-flash-preview` (review, test-gen) | Latest flash for cost-efficient tasks |
|
||||
|
||||
## Override
|
||||
|
||||
All scripts accept `--model <name>` to override the default. Pass any Gemini model ID.
|
||||
@@ -5,7 +5,6 @@ import betterTailwindcss from 'eslint-plugin-better-tailwindcss'
|
||||
import { createTypeScriptImportResolver } from 'eslint-import-resolver-typescript'
|
||||
import { importX } from 'eslint-plugin-import-x'
|
||||
import oxlint from 'eslint-plugin-oxlint'
|
||||
import testingLibrary from 'eslint-plugin-testing-library'
|
||||
// eslint-config-prettier disables ESLint rules that conflict with formatters (oxfmt)
|
||||
import eslintConfigPrettier from 'eslint-config-prettier'
|
||||
import { configs as storybookConfigs } from 'eslint-plugin-storybook'
|
||||
@@ -272,20 +271,6 @@ export default defineConfig([
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
files: ['**/*.test.ts'],
|
||||
plugins: { 'testing-library': testingLibrary },
|
||||
rules: {
|
||||
'testing-library/prefer-screen-queries': 'error',
|
||||
'testing-library/no-container': 'error',
|
||||
'testing-library/no-node-access': 'error',
|
||||
'testing-library/no-wait-for-multiple-assertions': 'error',
|
||||
'testing-library/prefer-find-by': 'error',
|
||||
'testing-library/prefer-presence-queries': 'error',
|
||||
'testing-library/prefer-user-event': 'error',
|
||||
'testing-library/no-debugging-utils': 'error'
|
||||
}
|
||||
},
|
||||
{
|
||||
files: ['scripts/**/*.js'],
|
||||
languageOptions: {
|
||||
|
||||
@@ -6,6 +6,7 @@ const config: KnipConfig = {
|
||||
entry: [
|
||||
'{build,scripts}/**/*.{js,ts}',
|
||||
'src/assets/css/style.css',
|
||||
'src/main.ts',
|
||||
'src/scripts/ui/menu/index.ts',
|
||||
'src/types/index.ts',
|
||||
'src/storybook/mocks/**/*.ts'
|
||||
@@ -13,34 +14,25 @@ const config: KnipConfig = {
|
||||
project: ['**/*.{js,ts,vue}', '*.{js,ts,mts}', '!.claude/**']
|
||||
},
|
||||
'apps/desktop-ui': {
|
||||
entry: ['src/i18n.ts'],
|
||||
entry: ['src/main.ts', 'src/i18n.ts'],
|
||||
project: ['src/**/*.{js,ts,vue}']
|
||||
},
|
||||
'packages/tailwind-utils': {
|
||||
project: ['src/**/*.{js,ts}']
|
||||
},
|
||||
'packages/shared-frontend-utils': {
|
||||
project: ['src/**/*.{js,ts}']
|
||||
project: ['src/**/*.{js,ts}'],
|
||||
entry: ['src/formatUtil.ts', 'src/networkUtil.ts']
|
||||
},
|
||||
'packages/registry-types': {
|
||||
project: ['src/**/*.{js,ts}']
|
||||
},
|
||||
'packages/ingest-types': {
|
||||
project: ['src/**/*.{js,ts}']
|
||||
},
|
||||
'apps/website': {
|
||||
entry: [
|
||||
'src/pages/**/*.astro',
|
||||
'src/layouts/**/*.astro',
|
||||
'src/components/**/*.vue',
|
||||
'src/styles/global.css',
|
||||
'astro.config.ts'
|
||||
],
|
||||
project: ['src/**/*.{astro,vue,ts}', '*.{js,ts,mjs}'],
|
||||
ignoreDependencies: ['@comfyorg/design-system', '@vercel/analytics']
|
||||
project: ['src/**/*.{js,ts}'],
|
||||
entry: ['src/index.ts']
|
||||
}
|
||||
},
|
||||
ignoreBinaries: ['python3', 'wrangler'],
|
||||
ignoreBinaries: ['python3', 'gh', 'generate'],
|
||||
ignoreDependencies: [
|
||||
// Weird importmap things
|
||||
'@iconify-json/lucide',
|
||||
@@ -48,12 +40,19 @@ const config: KnipConfig = {
|
||||
'@primeuix/forms',
|
||||
'@primeuix/styled',
|
||||
'@primeuix/utils',
|
||||
'@primevue/icons'
|
||||
'@primevue/icons',
|
||||
// Used by lucideStrokePlugin.js (CSS @plugin)
|
||||
'@iconify/utils'
|
||||
],
|
||||
ignore: [
|
||||
// Auto generated API types
|
||||
'src/workbench/extensions/manager/types/generatedManagerTypes.ts',
|
||||
'packages/registry-types/src/comfyRegistryTypes.ts',
|
||||
'packages/ingest-types/src/types.gen.ts',
|
||||
'packages/ingest-types/src/zod.gen.ts',
|
||||
'packages/ingest-types/openapi-ts.config.ts',
|
||||
// Used by a custom node (that should move off of this)
|
||||
'src/scripts/ui/components/splitButton.ts',
|
||||
// Used by stacked PR (feat/glsl-live-preview)
|
||||
'src/renderer/glsl/useGLSLRenderer.ts',
|
||||
// Workflow files contain license names that knip misinterprets as binaries
|
||||
@@ -61,8 +60,17 @@ const config: KnipConfig = {
|
||||
// Pending integration in stacked PR
|
||||
'src/components/sidebar/tabs/nodeLibrary/CustomNodesPanel.vue',
|
||||
// Agent review check config, not part of the build
|
||||
'.agents/checks/eslint.strict.config.js'
|
||||
'.agents/checks/eslint.strict.config.js',
|
||||
// Loaded via @plugin directive in CSS, not detected by knip
|
||||
'packages/design-system/src/css/lucideStrokePlugin.js'
|
||||
],
|
||||
compilers: {
|
||||
// https://github.com/webpro-nl/knip/issues/1008#issuecomment-3207756199
|
||||
css: (text: string) =>
|
||||
[...text.replaceAll('plugin', 'import').matchAll(/(?<=@)import[^;]+/g)]
|
||||
.map((match) => match[0].replace(/url\(['"]?([^'"()]+)['"]?\)/, '$1'))
|
||||
.join('\n')
|
||||
},
|
||||
vite: {
|
||||
config: ['vite?(.*).config.mts']
|
||||
},
|
||||
|
||||
@@ -11,7 +11,7 @@ export default {
|
||||
|
||||
'./**/*.js': (stagedFiles: string[]) => formatAndEslint(stagedFiles),
|
||||
|
||||
'./**/*.{ts,tsx,vue,mts,json,yaml}': (stagedFiles: string[]) => {
|
||||
'./**/*.{ts,tsx,vue,mts}': (stagedFiles: string[]) => {
|
||||
const commands = [...formatAndEslint(stagedFiles), 'pnpm typecheck']
|
||||
|
||||
const hasBrowserTestsChanges = stagedFiles
|
||||
|
||||
11
package.json
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@comfyorg/comfyui-frontend",
|
||||
"version": "1.43.4",
|
||||
"version": "1.43.2",
|
||||
"private": true,
|
||||
"description": "Official front-end implementation of ComfyUI",
|
||||
"homepage": "https://comfy.org",
|
||||
@@ -39,7 +39,6 @@
|
||||
"preinstall": "pnpm dlx only-allow pnpm",
|
||||
"prepare": "husky || true && git config blame.ignoreRevsFile .git-blame-ignore-revs || true",
|
||||
"preview": "nx preview",
|
||||
"qa:video-review": "tsx scripts/qa-video-review.ts",
|
||||
"storybook": "nx storybook",
|
||||
"storybook:desktop": "nx run @comfyorg/desktop-ui:storybook",
|
||||
"stylelint:fix": "stylelint --cache --fix '{apps,packages,src}/**/*.{css,vue}'",
|
||||
@@ -82,7 +81,6 @@
|
||||
"@tiptap/starter-kit": "catalog:",
|
||||
"@vueuse/core": "catalog:",
|
||||
"@vueuse/integrations": "catalog:",
|
||||
"@vueuse/router": "^14.2.0",
|
||||
"@xterm/addon-fit": "^0.10.0",
|
||||
"@xterm/addon-serialize": "^0.13.0",
|
||||
"@xterm/xterm": "^5.5.0",
|
||||
@@ -122,7 +120,6 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "catalog:",
|
||||
"@google/generative-ai": "catalog:",
|
||||
"@intlify/eslint-plugin-vue-i18n": "catalog:",
|
||||
"@lobehub/i18n-cli": "catalog:",
|
||||
"@nx/eslint": "catalog:",
|
||||
@@ -137,9 +134,6 @@
|
||||
"@storybook/vue3": "catalog:",
|
||||
"@storybook/vue3-vite": "catalog:",
|
||||
"@tailwindcss/vite": "catalog:",
|
||||
"@testing-library/jest-dom": "catalog:",
|
||||
"@testing-library/user-event": "catalog:",
|
||||
"@testing-library/vue": "catalog:",
|
||||
"@types/fs-extra": "catalog:",
|
||||
"@types/jsdom": "catalog:",
|
||||
"@types/node": "catalog:",
|
||||
@@ -158,7 +152,6 @@
|
||||
"eslint-plugin-import-x": "catalog:",
|
||||
"eslint-plugin-oxlint": "catalog:",
|
||||
"eslint-plugin-storybook": "catalog:",
|
||||
"eslint-plugin-testing-library": "catalog:",
|
||||
"eslint-plugin-unused-imports": "catalog:",
|
||||
"eslint-plugin-vue": "catalog:",
|
||||
"fast-check": "catalog:",
|
||||
@@ -183,7 +176,9 @@
|
||||
"storybook": "catalog:",
|
||||
"stylelint": "catalog:",
|
||||
"tailwindcss": "catalog:",
|
||||
"tailwindcss-primeui": "catalog:",
|
||||
"tsx": "catalog:",
|
||||
"tw-animate-css": "catalog:",
|
||||
"typescript": "catalog:",
|
||||
"typescript-eslint": "catalog:",
|
||||
"unplugin-icons": "catalog:",
|
||||
|
||||
@@ -12,9 +12,7 @@
|
||||
"dependencies": {
|
||||
"@iconify-json/lucide": "catalog:",
|
||||
"@iconify/tailwind4": "catalog:",
|
||||
"@iconify/utils": "catalog:",
|
||||
"tailwindcss-primeui": "catalog:",
|
||||
"tw-animate-css": "catalog:"
|
||||
"@iconify/utils": "catalog:"
|
||||
},
|
||||
"devDependencies": {
|
||||
"tailwindcss": "catalog:",
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
/*
|
||||
* Design System Base — Brand tokens + fonts only.
|
||||
* For marketing sites that don't use PrimeVue or the node editor.
|
||||
* Import the full style.css instead for the desktop app.
|
||||
*/
|
||||
|
||||
@import './fonts.css';
|
||||
|
||||
@theme {
|
||||
/* Font Families */
|
||||
--font-inter: 'Inter', sans-serif;
|
||||
|
||||
/* Palette Colors */
|
||||
--color-charcoal-100: #55565e;
|
||||
--color-charcoal-200: #494a50;
|
||||
--color-charcoal-300: #3c3d42;
|
||||
--color-charcoal-400: #313235;
|
||||
--color-charcoal-500: #2d2e32;
|
||||
--color-charcoal-600: #262729;
|
||||
--color-charcoal-700: #202121;
|
||||
--color-charcoal-800: #171718;
|
||||
|
||||
--color-neutral-550: #636363;
|
||||
|
||||
--color-ash-300: #bbbbbb;
|
||||
--color-ash-500: #828282;
|
||||
--color-ash-800: #444444;
|
||||
|
||||
--color-smoke-100: #f3f3f3;
|
||||
--color-smoke-200: #e9e9e9;
|
||||
--color-smoke-300: #e1e1e1;
|
||||
--color-smoke-400: #d9d9d9;
|
||||
--color-smoke-500: #c5c5c5;
|
||||
--color-smoke-600: #b4b4b4;
|
||||
--color-smoke-700: #a0a0a0;
|
||||
--color-smoke-800: #8a8a8a;
|
||||
|
||||
--color-white: #ffffff;
|
||||
--color-black: #000000;
|
||||
|
||||
/* Brand Colors */
|
||||
--color-electric-400: #f0ff41;
|
||||
--color-sapphire-700: #172dd7;
|
||||
--color-brand-yellow: var(--color-electric-400);
|
||||
--color-brand-blue: var(--color-sapphire-700);
|
||||
}
|
||||
@@ -15,7 +15,7 @@
|
||||
@plugin "./lucideStrokePlugin.js";
|
||||
|
||||
/* Safelist dynamic comfy icons for node library folders */
|
||||
@source inline("icon-[comfy--{ai-model,bfl,bria,bytedance,credits,elevenlabs,extensions-blocks,file-output,gemini,grok,hitpaw,ideogram,image-ai-edit,kling,ltxv,luma,magnific,mask,meshy,minimax,moonvalley-marey,node,openai,pin,pixverse,play,recraft,reve,rodin,runway,sora,stability-ai,template,tencent,topaz,tripo,veo,vidu,wan,wavespeed,workflow,quiver-ai}]");
|
||||
@source inline("icon-[comfy--{ai-model,bfl,bria,bytedance,credits,elevenlabs,extensions-blocks,file-output,gemini,grok,hitpaw,ideogram,image-ai-edit,kling,ltxv,luma,magnific,mask,meshy,minimax,moonvalley-marey,node,openai,pin,pixverse,play,recraft,reve,rodin,runway,sora,stability-ai,template,tencent,topaz,tripo,veo,vidu,wan,wavespeed,workflow}]");
|
||||
|
||||
/* Safelist dynamic comfy icons for essential nodes (kebab-case of node names) */
|
||||
@source inline("icon-[comfy--{save-image,load-video,save-video,load-3-d,save-glb,image-batch,batch-images-node,image-crop,image-scale,image-rotate,image-blur,image-invert,canny,recraft-remove-background-node,kling-lip-sync-audio-to-video-node,load-audio,save-audio,stability-text-to-audio,lora-loader,lora-loader-model-only,primitive-string-multiline,get-video-components,video-slice,tencent-text-to-model-node,tencent-image-to-model-node,open-ai-chat-node,preview-image,image-and-mask-preview,layer-mask-mask-preview,mask-preview,image-preview-from-latent,i-tools-preview-image,i-tools-compare-image,canny-to-image,image-edit,text-to-image,pose-to-image,depth-to-video,image-to-image,canny-to-video,depth-to-image,image-to-video,pose-to-video,text-to-video,image-inpainting,image-outpainting}]");
|
||||
@@ -1901,37 +1901,3 @@ audio.comfy-audio.empty-audio-widget {
|
||||
background-position: 0 0;
|
||||
}
|
||||
}
|
||||
|
||||
@utility scroll-shadows-* {
|
||||
overflow: auto;
|
||||
|
||||
background:
|
||||
/* Shadow Cover TOP */
|
||||
linear-gradient(--value(--color-*) 30%, transparent) center top,
|
||||
/* Shadow Cover BOTTOM */
|
||||
linear-gradient(transparent, --value(--color-*) 70%) center bottom,
|
||||
/* Shadow TOP */
|
||||
radial-gradient(
|
||||
farthest-side at 50% 0,
|
||||
color-mix(in oklab, --value(--color-*), #777777 35%),
|
||||
60%,
|
||||
transparent
|
||||
)
|
||||
center top,
|
||||
/* Shadow BOTTOM */
|
||||
radial-gradient(
|
||||
farthest-side at 50% 100%,
|
||||
color-mix(in oklab, --value(--color-*), #777777 35%),
|
||||
60%,
|
||||
transparent
|
||||
)
|
||||
center bottom;
|
||||
|
||||
background-repeat: no-repeat;
|
||||
background-size:
|
||||
300% 40px,
|
||||
300% 40px,
|
||||
300% 14px,
|
||||
300% 14px;
|
||||
background-attachment: local, local, scroll, scroll;
|
||||
}
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
<svg width="281" height="281" viewBox="0 0 281 281" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M140.069 0C217.427 0.000220786 280.138 62.7116 280.138 140.069V280.138H140.069C62.7116 280.138 0.000220844 217.427 0 140.069C0 62.7114 62.7114 0 140.069 0ZM74.961 66.6054C69.8263 64.8847 64.9385 69.7815 66.6687 74.913L123.558 243.619C125.929 250.65 136.321 248.945 136.321 241.524V135.823H241.329C248.756 135.823 250.453 125.416 243.41 123.056L74.961 66.6054Z" fill="#F8F8F8"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 534 B |
@@ -631,10 +631,3 @@ export function isPreviewableMediaType(mediaType: MediaType): boolean {
|
||||
mediaType === '3D'
|
||||
)
|
||||
}
|
||||
|
||||
export function formatTime(seconds: number): string {
|
||||
if (isNaN(seconds) || seconds === 0) return '0:00'
|
||||
const mins = Math.floor(seconds / 60)
|
||||
const secs = Math.floor(seconds % 60)
|
||||
return `${mins}:${secs.toString().padStart(2, '0')}`
|
||||
}
|
||||
|
||||
2799
pnpm-lock.yaml
generated
@@ -4,21 +4,19 @@ packages:
|
||||
|
||||
catalog:
|
||||
'@alloc/quick-lru': ^5.2.0
|
||||
'@astrojs/vue': ^5.0.0
|
||||
'@comfyorg/comfyui-electron-types': 0.6.2
|
||||
'@eslint/js': ^9.39.1
|
||||
'@formkit/auto-animate': ^0.9.0
|
||||
'@google/generative-ai': ^0.24.1
|
||||
'@iconify-json/lucide': ^1.1.178
|
||||
'@iconify/json': ^2.2.380
|
||||
'@iconify/tailwind4': ^1.2.0
|
||||
'@iconify/utils': ^3.1.0
|
||||
'@intlify/eslint-plugin-vue-i18n': ^4.1.1
|
||||
'@lobehub/i18n-cli': ^1.26.1
|
||||
'@nx/eslint': 22.6.1
|
||||
'@nx/playwright': 22.6.1
|
||||
'@nx/storybook': 22.6.1
|
||||
'@nx/vite': 22.6.1
|
||||
'@nx/eslint': 22.5.2
|
||||
'@nx/playwright': 22.5.2
|
||||
'@nx/storybook': 22.5.2
|
||||
'@nx/vite': 22.5.2
|
||||
'@pinia/testing': ^1.0.3
|
||||
'@playwright/test': ^1.58.1
|
||||
'@primeuix/forms': 0.0.2
|
||||
@@ -36,9 +34,6 @@ catalog:
|
||||
'@storybook/vue3': ^10.2.10
|
||||
'@storybook/vue3-vite': ^10.2.10
|
||||
'@tailwindcss/vite': ^4.2.0
|
||||
'@testing-library/jest-dom': ^6.9.1
|
||||
'@testing-library/user-event': ^14.6.1
|
||||
'@testing-library/vue': ^8.1.0
|
||||
'@tiptap/core': ^2.27.2
|
||||
'@tiptap/extension-link': ^2.27.2
|
||||
'@tiptap/extension-table': ^2.27.2
|
||||
@@ -52,7 +47,6 @@ catalog:
|
||||
'@types/node': ^24.1.0
|
||||
'@types/semver': ^7.7.0
|
||||
'@types/three': ^0.169.0
|
||||
'@vercel/analytics': ^2.0.1
|
||||
'@vitejs/plugin-vue': ^6.0.0
|
||||
'@vitest/coverage-v8': ^4.0.16
|
||||
'@vitest/ui': ^4.0.16
|
||||
@@ -61,7 +55,6 @@ catalog:
|
||||
'@vueuse/integrations': ^14.2.0
|
||||
'@webgpu/types': ^0.1.66
|
||||
algoliasearch: ^5.21.0
|
||||
astro: ^5.10.0
|
||||
axios: ^1.13.5
|
||||
cross-env: ^10.1.0
|
||||
cva: 1.0.0-beta.4
|
||||
@@ -74,7 +67,6 @@ catalog:
|
||||
eslint-plugin-import-x: ^4.16.1
|
||||
eslint-plugin-oxlint: 1.55.0
|
||||
eslint-plugin-storybook: ^10.2.10
|
||||
eslint-plugin-testing-library: ^7.16.1
|
||||
eslint-plugin-unused-imports: ^4.3.0
|
||||
eslint-plugin-vue: ^10.6.2
|
||||
fast-check: ^4.5.3
|
||||
@@ -87,11 +79,11 @@ catalog:
|
||||
jsdom: ^27.4.0
|
||||
jsonata: ^2.1.0
|
||||
jsondiffpatch: ^0.7.3
|
||||
knip: ^6.0.1
|
||||
knip: ^5.75.1
|
||||
lint-staged: ^16.2.7
|
||||
markdown-table: ^3.0.4
|
||||
mixpanel-browser: ^2.71.0
|
||||
nx: 22.6.1
|
||||
nx: 22.5.2
|
||||
oxfmt: ^0.40.0
|
||||
oxlint: ^1.55.0
|
||||
oxlint-tsgolint: ^0.17.0
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import { extractMediaUrls } from './qa-analyze-pr'
|
||||
|
||||
describe('extractMediaUrls', () => {
|
||||
it('extracts markdown image URLs', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
|
||||
})
|
||||
|
||||
it('extracts multiple markdown images', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Some text',
|
||||
''
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/before.png',
|
||||
'https://example.com/after.jpg'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts raw URLs with media extensions', () => {
|
||||
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
|
||||
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
|
||||
})
|
||||
|
||||
it('extracts GitHub user-attachments URLs', () => {
|
||||
const text =
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts private-user-images URLs', () => {
|
||||
const text =
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
|
||||
])
|
||||
})
|
||||
|
||||
it('extracts URLs with query parameters', () => {
|
||||
const text = 'https://example.com/image.png?w=800&h=600'
|
||||
expect(extractMediaUrls(text)).toEqual([
|
||||
'https://example.com/image.png?w=800&h=600'
|
||||
])
|
||||
})
|
||||
|
||||
it('deduplicates URLs', () => {
|
||||
const text = [
|
||||
'',
|
||||
'',
|
||||
'Also https://example.com/same.png'
|
||||
].join('\n')
|
||||
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
|
||||
})
|
||||
|
||||
it('returns empty array for empty input', () => {
|
||||
expect(extractMediaUrls('')).toEqual([])
|
||||
})
|
||||
|
||||
it('returns empty array for text with no media URLs', () => {
|
||||
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
|
||||
})
|
||||
|
||||
it('handles mixed media types', () => {
|
||||
const text = [
|
||||
'',
|
||||
'Video: https://example.com/demo.webm',
|
||||
''
|
||||
].join('\n')
|
||||
const urls = extractMediaUrls(text)
|
||||
expect(urls).toContain('https://example.com/screenshot.png')
|
||||
expect(urls).toContain('https://example.com/demo.webm')
|
||||
expect(urls).toContain('https://example.com/animation.gif')
|
||||
})
|
||||
|
||||
it('ignores non-http URLs in markdown', () => {
|
||||
const text = ''
|
||||
expect(extractMediaUrls(text)).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -1,799 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* QA PR Analysis Script
|
||||
*
|
||||
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
|
||||
* for before/after recording sessions. Fetches PR thread, extracts media,
|
||||
* and produces structured test plans.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-analyze-pr.ts \
|
||||
* --pr-number 10270 \
|
||||
* --repo owner/repo \
|
||||
* --output-dir qa-guides/ \
|
||||
* [--model gemini-3.1-pro-preview]
|
||||
*
|
||||
* Env: GEMINI_API_KEY (required)
|
||||
*/
|
||||
|
||||
import { execSync } from 'node:child_process'
|
||||
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
|
||||
import { resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
// ── Types ──
|
||||
|
||||
interface QaGuideStep {
|
||||
action: string
|
||||
description: string
|
||||
expected_before?: string
|
||||
expected_after?: string
|
||||
}
|
||||
|
||||
interface QaGuide {
|
||||
summary: string
|
||||
test_focus: string
|
||||
prerequisites: string[]
|
||||
steps: QaGuideStep[]
|
||||
visual_checks: string[]
|
||||
}
|
||||
|
||||
interface PrThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
issueComments: string[]
|
||||
reviewComments: string[]
|
||||
reviews: string[]
|
||||
diff: string
|
||||
}
|
||||
|
||||
type TargetType = 'pr' | 'issue'
|
||||
|
||||
interface Options {
|
||||
prNumber: string
|
||||
repo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
apiKey: string
|
||||
mediaBudgetBytes: number
|
||||
maxVideoBytes: number
|
||||
type: TargetType
|
||||
}
|
||||
|
||||
// ── CLI parsing ──
|
||||
|
||||
function parseArgs(): Options {
|
||||
const args = process.argv.slice(2)
|
||||
const opts: Partial<Options> = {
|
||||
model: 'gemini-3.1-pro-preview',
|
||||
apiKey: process.env.GEMINI_API_KEY || '',
|
||||
mediaBudgetBytes: 20 * 1024 * 1024,
|
||||
maxVideoBytes: 10 * 1024 * 1024,
|
||||
type: 'pr'
|
||||
}
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--pr-number':
|
||||
opts.prNumber = args[++i]
|
||||
break
|
||||
case '--repo':
|
||||
opts.repo = args[++i]
|
||||
break
|
||||
case '--output-dir':
|
||||
opts.outputDir = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--type':
|
||||
opts.type = args[++i] as TargetType
|
||||
break
|
||||
case '--help':
|
||||
console.warn(
|
||||
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
|
||||
)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
|
||||
console.error(
|
||||
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (!opts.apiKey) {
|
||||
console.error('GEMINI_API_KEY environment variable is required')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts as Options
|
||||
}
|
||||
|
||||
// ── PR thread fetching ──
|
||||
|
||||
function ghExec(cmd: string): string {
|
||||
try {
|
||||
return execSync(cmd, {
|
||||
encoding: 'utf-8',
|
||||
timeout: 30_000,
|
||||
stdio: ['pipe', 'pipe', 'pipe']
|
||||
}).trim()
|
||||
} catch (err) {
|
||||
console.warn(`gh command failed: ${cmd}`)
|
||||
console.warn((err as Error).message)
|
||||
return ''
|
||||
}
|
||||
}
|
||||
|
||||
function fetchPrThread(prNumber: string, repo: string): PrThread {
|
||||
console.warn('Fetching PR thread...')
|
||||
|
||||
const prView = ghExec(
|
||||
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const prData = prView
|
||||
? JSON.parse(prView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const issueCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
|
||||
)
|
||||
const issueComments: string[] = issueCommentsRaw
|
||||
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewCommentsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
|
||||
)
|
||||
const reviewComments: string[] = reviewCommentsRaw
|
||||
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
const reviewsRaw = ghExec(
|
||||
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
|
||||
)
|
||||
const reviews: string[] = reviewsRaw
|
||||
? JSON.parse(reviewsRaw)
|
||||
.filter((r: { body: string }) => r.body)
|
||||
.map((r: { body: string }) => r.body)
|
||||
: []
|
||||
|
||||
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
|
||||
|
||||
console.warn(
|
||||
`PR #${prNumber}: "${prData.title}" | ` +
|
||||
`${issueComments.length} issue comments, ` +
|
||||
`${reviewComments.length} review comments, ` +
|
||||
`${reviews.length} reviews, ` +
|
||||
`diff: ${diff.length} chars`
|
||||
)
|
||||
|
||||
return {
|
||||
title: prData.title || '',
|
||||
body: prData.body || '',
|
||||
labels: (prData.labels || []).map((l: { name: string }) => l.name),
|
||||
issueComments,
|
||||
reviewComments,
|
||||
reviews,
|
||||
diff
|
||||
}
|
||||
}
|
||||
|
||||
interface IssueThread {
|
||||
title: string
|
||||
body: string
|
||||
labels: string[]
|
||||
comments: string[]
|
||||
}
|
||||
|
||||
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
|
||||
console.warn('Fetching issue thread...')
|
||||
|
||||
const issueView = ghExec(
|
||||
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
|
||||
)
|
||||
const issueData = issueView
|
||||
? JSON.parse(issueView)
|
||||
: { title: '', body: '', labels: [] }
|
||||
|
||||
const commentsRaw = ghExec(
|
||||
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
|
||||
)
|
||||
const comments: string[] = commentsRaw
|
||||
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
|
||||
: []
|
||||
|
||||
console.warn(
|
||||
`Issue #${issueNumber}: "${issueData.title}" | ` +
|
||||
`${comments.length} comments`
|
||||
)
|
||||
|
||||
return {
|
||||
title: issueData.title || '',
|
||||
body: issueData.body || '',
|
||||
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
|
||||
comments
|
||||
}
|
||||
}
|
||||
|
||||
// ── Media extraction ──
|
||||
|
||||
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
|
||||
|
||||
const MEDIA_URL_PATTERNS = [
|
||||
// Markdown images: 
|
||||
/!\[[^\]]*\]\(([^)]+)\)/g,
|
||||
// GitHub user-attachments
|
||||
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
|
||||
// Private user images
|
||||
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
|
||||
// Raw URLs with media extensions (standalone or in text)
|
||||
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
|
||||
]
|
||||
|
||||
export function extractMediaUrls(text: string): string[] {
|
||||
if (!text) return []
|
||||
|
||||
const urls = new Set<string>()
|
||||
|
||||
for (const pattern of MEDIA_URL_PATTERNS) {
|
||||
// Reset lastIndex for global patterns
|
||||
pattern.lastIndex = 0
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = pattern.exec(text)) !== null) {
|
||||
// For markdown images, the URL is in capture group 1
|
||||
const url = match[1] || match[0]
|
||||
// Clean trailing markdown/html artifacts
|
||||
const cleaned = url.replace(/[)>"'\s]+$/, '')
|
||||
if (cleaned.startsWith('http')) {
|
||||
urls.add(cleaned)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [...urls]
|
||||
}
|
||||
|
||||
// ── Media downloading ──
|
||||
|
||||
const ALLOWED_MEDIA_DOMAINS = [
|
||||
'github.com',
|
||||
'raw.githubusercontent.com',
|
||||
'user-images.githubusercontent.com',
|
||||
'private-user-images.githubusercontent.com',
|
||||
'objects.githubusercontent.com',
|
||||
'github.githubassets.com'
|
||||
]
|
||||
|
||||
function isAllowedMediaDomain(url: string): boolean {
|
||||
try {
|
||||
const hostname = new URL(url).hostname
|
||||
return ALLOWED_MEDIA_DOMAINS.some(
|
||||
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
|
||||
)
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
async function downloadMedia(
|
||||
urls: string[],
|
||||
outputDir: string,
|
||||
budgetBytes: number,
|
||||
maxVideoBytes: number
|
||||
): Promise<Array<{ path: string; mimeType: string }>> {
|
||||
const downloaded: Array<{ path: string; mimeType: string }> = []
|
||||
let totalBytes = 0
|
||||
|
||||
const mediaDir = resolve(outputDir, 'media')
|
||||
mkdirSync(mediaDir, { recursive: true })
|
||||
|
||||
for (const url of urls) {
|
||||
if (totalBytes >= budgetBytes) {
|
||||
console.warn(
|
||||
`Media budget exhausted (${totalBytes} bytes), skipping rest`
|
||||
)
|
||||
break
|
||||
}
|
||||
|
||||
if (!isAllowedMediaDomain(url)) {
|
||||
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
headers: { Accept: 'image/*,video/*' },
|
||||
redirect: 'follow'
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
console.warn(`Failed to download ${url}: ${response.status}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const contentLength = response.headers.get('content-length')
|
||||
if (contentLength) {
|
||||
const declaredSize = Number.parseInt(contentLength, 10)
|
||||
if (declaredSize > budgetBytes - totalBytes) {
|
||||
console.warn(
|
||||
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
|
||||
)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
const contentType = response.headers.get('content-type') || ''
|
||||
const buffer = Buffer.from(await response.arrayBuffer())
|
||||
|
||||
// Skip oversized videos
|
||||
const isVideo =
|
||||
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
|
||||
if (isVideo && buffer.length > maxVideoBytes) {
|
||||
console.warn(
|
||||
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (totalBytes + buffer.length > budgetBytes) {
|
||||
console.warn(`Would exceed budget, skipping ${url}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const ext = guessExtension(url, contentType)
|
||||
const filename = `media-${downloaded.length}${ext}`
|
||||
const filepath = resolve(mediaDir, filename)
|
||||
writeFileSync(filepath, buffer)
|
||||
totalBytes += buffer.length
|
||||
|
||||
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
|
||||
|
||||
downloaded.push({ path: filepath, mimeType })
|
||||
console.warn(
|
||||
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
|
||||
)
|
||||
} catch (err) {
|
||||
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Downloaded ${downloaded.length}/${urls.length} media files ` +
|
||||
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
|
||||
)
|
||||
return downloaded
|
||||
}
|
||||
|
||||
function guessExtension(url: string, contentType: string): string {
|
||||
const urlMatch = url.match(MEDIA_EXTENSIONS)
|
||||
if (urlMatch) return urlMatch[0].toLowerCase()
|
||||
|
||||
const typeMap: Record<string, string> = {
|
||||
'image/png': '.png',
|
||||
'image/jpeg': '.jpg',
|
||||
'image/gif': '.gif',
|
||||
'image/webp': '.webp',
|
||||
'video/mp4': '.mp4',
|
||||
'video/webm': '.webm'
|
||||
}
|
||||
return typeMap[contentType.split(';')[0]] || '.bin'
|
||||
}
|
||||
|
||||
function guessMimeType(ext: string): string {
|
||||
const map: Record<string, string> = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime'
|
||||
}
|
||||
return map[ext] || 'application/octet-stream'
|
||||
}
|
||||
|
||||
// ── Gemini analysis ──
|
||||
|
||||
function buildIssueAnalysisPrompt(issue: IssueThread): string {
|
||||
const allText = [
|
||||
`# Issue: ${issue.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
issue.body,
|
||||
'',
|
||||
issue.comments.length > 0
|
||||
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
|
||||
: ''
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
|
||||
|
||||
The UI has:
|
||||
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
|
||||
- Nodes are boxes with input/output slots connected by wires
|
||||
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
|
||||
- Sidebars (Workflows, Node Library, Models)
|
||||
- A topbar with workflow tabs and Queue button
|
||||
- The default workflow loads with these nodes (approximate center coordinates):
|
||||
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
|
||||
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
|
||||
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
|
||||
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
|
||||
|
||||
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
|
||||
### Menu actions
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
|
||||
### Element actions (by visible text)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "rightClick" — right-clicks an element to open context menu (text required)
|
||||
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
|
||||
|
||||
### Canvas actions (by coordinates — viewport is 1280x720)
|
||||
- "clickCanvas" — click at coordinates (x, y required)
|
||||
- "rightClickCanvas" — right-click at coordinates (x, y required)
|
||||
- "doubleClick" — double-click at coordinates to open node search (x, y)
|
||||
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
|
||||
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
|
||||
|
||||
### Utility
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Common ComfyUI interactions
|
||||
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
|
||||
- Double-click empty canvas → opens node search dialog
|
||||
- Ctrl+C / Ctrl+V → copy/paste selected nodes
|
||||
- Delete key → remove selected node
|
||||
- Ctrl+G → group selected nodes
|
||||
- Drag from output slot to input slot → create connection
|
||||
- Click a node to select it, Shift+click for multi-select
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly one key: "reproduce", containing:
|
||||
{
|
||||
"summary": "One sentence: what bug this issue reports",
|
||||
"test_focus": "Specific behavior to reproduce",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "clickCanvas",
|
||||
"description": "Click on first node to select it",
|
||||
"expected_before": "What should happen if the bug is present"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual evidence of the bug to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
|
||||
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
|
||||
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
|
||||
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
|
||||
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
|
||||
- Do NOT include login steps.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
function buildAnalysisPrompt(thread: PrThread): string {
|
||||
const allText = [
|
||||
`# PR: ${thread.title}`,
|
||||
'',
|
||||
'## Description',
|
||||
thread.body,
|
||||
'',
|
||||
thread.issueComments.length > 0
|
||||
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviewComments.length > 0
|
||||
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
thread.reviews.length > 0
|
||||
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
|
||||
: '',
|
||||
'',
|
||||
'## Diff (truncated)',
|
||||
'```',
|
||||
thread.diff.slice(0, 8000),
|
||||
'```'
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
|
||||
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
|
||||
|
||||
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
|
||||
|
||||
${allText}
|
||||
|
||||
## Available test actions
|
||||
Each step must use one of these actions:
|
||||
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
|
||||
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
|
||||
- "clickMenuItem" — clicks an item in the visible submenu (label required)
|
||||
- "fillDialog" — fills dialog input and presses Enter (text required)
|
||||
- "pressKey" — presses a keyboard key (key required)
|
||||
- "click" — clicks an element by visible text (text required)
|
||||
- "wait" — waits briefly (ms required, max 3000)
|
||||
- "screenshot" — takes a screenshot (name required)
|
||||
|
||||
## Output format
|
||||
Return a JSON object with exactly two keys: "before" and "after", each containing:
|
||||
{
|
||||
"summary": "One sentence: what this PR changes",
|
||||
"test_focus": "Specific behaviors to verify in this recording",
|
||||
"prerequisites": ["e.g. Load default workflow"],
|
||||
"steps": [
|
||||
{
|
||||
"action": "openMenu",
|
||||
"description": "Open the main menu to check file options",
|
||||
"expected_before": "Old behavior description (before key only)",
|
||||
"expected_after": "New behavior description (after key only)"
|
||||
}
|
||||
],
|
||||
"visual_checks": ["Specific visual elements to look for"]
|
||||
}
|
||||
|
||||
## Rules
|
||||
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
|
||||
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
|
||||
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
|
||||
- Use information from PR description, screenshots, and comments to understand intended behavior.
|
||||
- Include at least one screenshot step in each guide.
|
||||
- Do NOT include login steps.
|
||||
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
|
||||
- Output ONLY valid JSON, no markdown fences or explanation.`
|
||||
}
|
||||
|
||||
async function analyzeWithGemini(
|
||||
thread: PrThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<{ before: QaGuide; after: QaGuide }> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildAnalysisPrompt(thread)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
// Add media as inline data
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
// Strip markdown fences if present
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
// Handle different response shapes from Gemini
|
||||
let before: QaGuide
|
||||
let after: QaGuide
|
||||
|
||||
if (Array.isArray(parsed) && parsed.length >= 2) {
|
||||
// Array format: [before, after]
|
||||
before = parsed[0]
|
||||
after = parsed[1]
|
||||
} else if (parsed.before && parsed.after) {
|
||||
// Object format: { before, after }
|
||||
before = parsed.before
|
||||
after = parsed.after
|
||||
} else {
|
||||
// Try nested wrapper keys
|
||||
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
|
||||
if (inner.before && inner.after) {
|
||||
before = inner.before
|
||||
after = inner.after
|
||||
} else {
|
||||
console.warn(
|
||||
'Full response:',
|
||||
JSON.stringify(parsed, null, 2).slice(0, 2000)
|
||||
)
|
||||
throw new Error(
|
||||
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
return { before, after }
|
||||
}
|
||||
|
||||
async function analyzeIssueWithGemini(
|
||||
issue: IssueThread,
|
||||
media: Array<{ path: string; mimeType: string }>,
|
||||
model: string,
|
||||
apiKey: string
|
||||
): Promise<QaGuide> {
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const geminiModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildIssueAnalysisPrompt(issue)
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
for (const item of media) {
|
||||
try {
|
||||
const buffer = readFileSync(item.path)
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: item.mimeType,
|
||||
data: buffer.toString('base64')
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
`Failed to read media ${item.path}: ${(err as Error).message}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
console.warn(
|
||||
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
|
||||
)
|
||||
|
||||
const result = await geminiModel.generateContent({
|
||||
contents: [{ role: 'user', parts }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192,
|
||||
responseMimeType: 'application/json'
|
||||
}
|
||||
})
|
||||
|
||||
let text = result.response.text()
|
||||
text = text
|
||||
.replace(/^```(?:json)?\n?/gm, '')
|
||||
.replace(/```$/gm, '')
|
||||
.trim()
|
||||
|
||||
console.warn('Gemini response received')
|
||||
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
|
||||
const parsed = JSON.parse(text)
|
||||
|
||||
const guide: QaGuide =
|
||||
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
|
||||
return guide
|
||||
}
|
||||
|
||||
// ── Main ──
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
mkdirSync(opts.outputDir, { recursive: true })
|
||||
|
||||
if (opts.type === 'issue') {
|
||||
await analyzeIssue(opts)
|
||||
} else {
|
||||
await analyzePr(opts)
|
||||
}
|
||||
}
|
||||
|
||||
async function analyzeIssue(opts: Options) {
|
||||
const issue = fetchIssueThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [issue.body, ...issue.comments].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guide = await analyzeIssueWithGemini(
|
||||
issue,
|
||||
media,
|
||||
opts.model,
|
||||
opts.apiKey
|
||||
)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guide:`)
|
||||
console.warn(` Reproduce: ${beforePath}`)
|
||||
}
|
||||
|
||||
async function analyzePr(opts: Options) {
|
||||
const thread = fetchPrThread(opts.prNumber, opts.repo)
|
||||
|
||||
const allText = [
|
||||
thread.body,
|
||||
...thread.issueComments,
|
||||
...thread.reviewComments,
|
||||
...thread.reviews
|
||||
].join('\n')
|
||||
const mediaUrls = extractMediaUrls(allText)
|
||||
console.warn(`Found ${mediaUrls.length} media URLs`)
|
||||
|
||||
const media = await downloadMedia(
|
||||
mediaUrls,
|
||||
opts.outputDir,
|
||||
opts.mediaBudgetBytes,
|
||||
opts.maxVideoBytes
|
||||
)
|
||||
|
||||
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
|
||||
|
||||
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
|
||||
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
|
||||
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
|
||||
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
|
||||
|
||||
console.warn(`Wrote QA guides:`)
|
||||
console.warn(` Before: ${beforePath}`)
|
||||
console.warn(` After: ${afterPath}`)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
main().catch((err) => {
|
||||
console.error('PR analysis failed:', err)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
@@ -1,176 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Batch-trigger QA runs by creating and pushing sno-qa-* branches.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/qa-batch.sh 10394 10238 9996 # Trigger specific numbers
|
||||
# ./scripts/qa-batch.sh --from tmp/issues.md --top 5 # From triage file
|
||||
# ./scripts/qa-batch.sh --dry-run 10394 10238 # Preview only
|
||||
# ./scripts/qa-batch.sh --cleanup # Delete old sno-qa-* branches
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
DELAY=5
|
||||
DRY_RUN=false
|
||||
CLEANUP=false
|
||||
FROM_FILE=""
|
||||
TOP_N=0
|
||||
NUMBERS=()
|
||||
|
||||
die() { echo "error: $*" >&2; exit 1; }
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: qa-batch.sh [options] [numbers...]
|
||||
|
||||
Options:
|
||||
--from <file> Extract numbers from a triage markdown file
|
||||
--top <N> Take first N entries from Tier 1 (requires --from)
|
||||
--dry-run Print what would happen without pushing
|
||||
--cleanup Delete all sno-qa-* remote branches
|
||||
--delay <secs> Seconds between pushes (default: 5)
|
||||
-h, --help Show this help
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# --- Parse args ---
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--from) FROM_FILE="$2"; shift 2 ;;
|
||||
--top) TOP_N="$2"; shift 2 ;;
|
||||
--dry-run) DRY_RUN=true; shift ;;
|
||||
--cleanup) CLEANUP=true; shift ;;
|
||||
--delay) DELAY="$2"; shift 2 ;;
|
||||
-h|--help) usage ;;
|
||||
-*) die "unknown option: $1" ;;
|
||||
*) NUMBERS+=("$1"); shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# --- Cleanup mode ---
|
||||
if $CLEANUP; then
|
||||
echo "Fetching remote sno-qa-* branches..."
|
||||
branches=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' | awk '{print $2}' | sed 's|refs/heads/||')
|
||||
|
||||
if [[ -z "$branches" ]]; then
|
||||
echo "No sno-qa-* branches found on remote."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Found branches:"
|
||||
while IFS= read -r b; do echo " $b"; done <<< "$branches"
|
||||
echo
|
||||
|
||||
if $DRY_RUN; then
|
||||
echo "[dry-run] Would delete the above branches."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
read -rp "Delete all of the above? [y/N] " confirm
|
||||
if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then
|
||||
echo "Aborted."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
for branch in $branches; do
|
||||
echo "Deleting origin/$branch..."
|
||||
git push origin --delete "$branch"
|
||||
done
|
||||
echo "Done. Cleaned up $(echo "$branches" | wc -l | tr -d ' ') branches."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- Extract numbers from markdown ---
|
||||
if [[ -n "$FROM_FILE" ]]; then
|
||||
[[ -f "$FROM_FILE" ]] || die "file not found: $FROM_FILE"
|
||||
[[ "$TOP_N" -gt 0 ]] || die "--top N required with --from"
|
||||
|
||||
# Extract Tier 1 table rows: | N | [#NNNNN](...) | ...
|
||||
# Stop at the next ## heading after Tier 1
|
||||
extracted=$(awk '/^## Tier 1/,/^## Tier [^1]/' "$FROM_FILE" \
|
||||
| grep -oP '\[#\K\d+' \
|
||||
| head -n "$TOP_N")
|
||||
|
||||
if [[ -z "$extracted" ]]; then
|
||||
die "no numbers found in $FROM_FILE"
|
||||
fi
|
||||
|
||||
while IFS= read -r num; do
|
||||
NUMBERS+=("$num")
|
||||
done <<< "$extracted"
|
||||
fi
|
||||
|
||||
[[ ${#NUMBERS[@]} -gt 0 ]] || die "no numbers specified. Use positional args or --from/--top."
|
||||
|
||||
# --- Validate ---
|
||||
for num in "${NUMBERS[@]}"; do
|
||||
[[ "$num" =~ ^[0-9]+$ ]] || die "invalid number: $num"
|
||||
done
|
||||
|
||||
# Deduplicate
|
||||
# shellcheck disable=SC2207 # mapfile not available on macOS default bash
|
||||
NUMBERS=($(printf '%s\n' "${NUMBERS[@]}" | sort -un))
|
||||
|
||||
# --- Push branches ---
|
||||
echo "Triggering QA for: ${NUMBERS[*]}"
|
||||
if $DRY_RUN; then
|
||||
echo "[dry-run]"
|
||||
fi
|
||||
echo
|
||||
|
||||
pushed=()
|
||||
skipped=()
|
||||
|
||||
# Fetch remote refs once
|
||||
remote_refs=$(git ls-remote --heads origin 'refs/heads/sno-qa-*' 2>/dev/null | awk '{print $2}' | sed 's|refs/heads/||')
|
||||
|
||||
for num in "${NUMBERS[@]}"; do
|
||||
branch="sno-qa-$num"
|
||||
|
||||
# Check if already exists on remote
|
||||
if echo "$remote_refs" | grep -qx "$branch"; then
|
||||
echo " skip: $branch (already exists on remote)"
|
||||
skipped+=("$num")
|
||||
continue
|
||||
fi
|
||||
|
||||
if $DRY_RUN; then
|
||||
echo " would push: $branch"
|
||||
pushed+=("$num")
|
||||
continue
|
||||
fi
|
||||
|
||||
# Create branch at current HEAD and push
|
||||
git branch -f "$branch" HEAD
|
||||
git push origin "$branch"
|
||||
pushed+=("$num")
|
||||
echo " pushed: $branch"
|
||||
|
||||
# Clean up local branch
|
||||
git branch -D "$branch" 2>/dev/null || true
|
||||
|
||||
# Delay between pushes to avoid CI concurrency storm
|
||||
if [[ "$num" != "${NUMBERS[-1]}" ]]; then
|
||||
echo " waiting ${DELAY}s..."
|
||||
sleep "$DELAY"
|
||||
fi
|
||||
done
|
||||
|
||||
# --- Summary ---
|
||||
echo
|
||||
echo "=== Summary ==="
|
||||
echo "Triggered: ${#pushed[@]}"
|
||||
echo "Skipped: ${#skipped[@]}"
|
||||
|
||||
if [[ ${#pushed[@]} -gt 0 ]]; then
|
||||
echo
|
||||
echo "Triggered numbers: ${pushed[*]}"
|
||||
repo_url=$(git remote get-url origin | sed 's/\.git$//' | sed 's|git@github.com:|https://github.com/|')
|
||||
echo "Actions: ${repo_url}/actions"
|
||||
fi
|
||||
|
||||
if [[ ${#skipped[@]} -gt 0 ]]; then
|
||||
echo
|
||||
echo "Skipped (already exist): ${skipped[*]}"
|
||||
echo "Use --cleanup first to remove old branches."
|
||||
fi
|
||||
@@ -1,259 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deploy QA report to Cloudflare Pages.
|
||||
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
|
||||
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
|
||||
# Writes outputs to GITHUB_OUTPUT: badge_status, url
|
||||
set -euo pipefail
|
||||
|
||||
npm install -g wrangler@4.74.0 >/dev/null 2>&1
|
||||
|
||||
DEPLOY_DIR=$(mktemp -d)
|
||||
mkdir -p "$DEPLOY_DIR"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
|
||||
for prefix in qa qa-before; do
|
||||
VID="${DIR}/${prefix}-session.mp4"
|
||||
if [ -f "$VID" ]; then
|
||||
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
|
||||
cp "$VID" "$DEST"
|
||||
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
|
||||
fi
|
||||
done
|
||||
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
|
||||
for numbered in "$DIR"/qa-session-[0-9].mp4; do
|
||||
[ -f "$numbered" ] || continue
|
||||
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
|
||||
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
|
||||
cp "$numbered" "$DEST"
|
||||
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
|
||||
done
|
||||
# Generate GIF thumbnail from after video (or first pass)
|
||||
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
|
||||
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
|
||||
if [ -f "$THUMB_SRC" ]; then
|
||||
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
|
||||
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
|
||||
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|
||||
|| echo "GIF generation failed for ${os} (non-fatal)"
|
||||
fi
|
||||
done
|
||||
|
||||
# Build video cards and report sections
|
||||
CARDS=""
|
||||
# shellcheck disable=SC2034 # accessed via eval
|
||||
ICONS_Linux="🐧" ICONS_macOS="🍎" ICONS_Windows="🪟"
|
||||
CARD_COUNT=0
|
||||
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
|
||||
|
||||
for os in Linux macOS Windows; do
|
||||
eval "ICON=\$ICONS_${os}"
|
||||
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
|
||||
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
|
||||
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
|
||||
[ "$HAS_AFTER" = "0" ] && continue
|
||||
|
||||
# Collect all reports for this platform (single + multi-pass)
|
||||
REPORT_FILES=""
|
||||
REPORT_LINK=""
|
||||
REPORT_HTML=""
|
||||
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
|
||||
done
|
||||
|
||||
if [ -n "$REPORT_FILES" ]; then
|
||||
# Concatenate all reports into one combined report file
|
||||
COMBINED_MD=""
|
||||
for rpt in $REPORT_FILES; do
|
||||
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
|
||||
RPT_MD=$(sed 's/&/\&/g; s/</\</g; s/>/\>/g' "$rpt")
|
||||
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD} --- "
|
||||
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
|
||||
done
|
||||
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
|
||||
FIRST_BASENAME=$(basename "$FIRST_REPORT")
|
||||
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
|
||||
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
|
||||
fi
|
||||
|
||||
if [ "$HAS_BEFORE" = "1" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
|
||||
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
|
||||
else
|
||||
PASS_VIDEOS=""
|
||||
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
|
||||
[ -f "$pass_vid" ] || continue
|
||||
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
|
||||
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls muted preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
|
||||
done
|
||||
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
|
||||
fi
|
||||
CARD_COUNT=$((CARD_COUNT + 1))
|
||||
done
|
||||
|
||||
# Build commit info and target link for the report header
|
||||
COMMIT_HTML=""
|
||||
REPO_URL="https://github.com/${REPO}"
|
||||
if [ -n "${TARGET_NUM:-}" ]; then
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
|
||||
else
|
||||
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
|
||||
fi
|
||||
fi
|
||||
if [ -n "${BEFORE_SHA:-}" ]; then
|
||||
SHORT_BEFORE="${BEFORE_SHA:0:7}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
|
||||
fi
|
||||
if [ -n "${AFTER_SHA:-}" ]; then
|
||||
SHORT_AFTER="${AFTER_SHA:0:7}"
|
||||
AFTER_LABEL="PR"
|
||||
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
|
||||
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} · }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
|
||||
fi
|
||||
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" · ${COMMIT_HTML}"
|
||||
|
||||
RUN_LINK=""
|
||||
if [ -n "${RUN_URL:-}" ]; then
|
||||
RUN_LINK=" · <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
|
||||
fi
|
||||
|
||||
# Generate index.html from template
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
|
||||
|
||||
# Write dynamic content to temp files for safe substitution
|
||||
# Cloudflare Pages _headers file — enable range requests for video seeking
|
||||
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
|
||||
/*.mp4
|
||||
Accept-Ranges: bytes
|
||||
Cache-Control: public, max-age=86400
|
||||
HEADERSEOF
|
||||
|
||||
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
|
||||
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
|
||||
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
|
||||
# Badge HTML with copy button (placeholder URL filled after deploy)
|
||||
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
|
||||
python3 -c "
|
||||
import sys, pathlib
|
||||
d = pathlib.Path(sys.argv[1])
|
||||
t = pathlib.Path(sys.argv[2]).read_text()
|
||||
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
|
||||
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
|
||||
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
|
||||
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
|
||||
sys.stdout.write(t)
|
||||
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
|
||||
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html"
|
||||
|
||||
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
|
||||
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
|
||||
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
|
||||
ERROREOF
|
||||
|
||||
# Generate badge SVGs into deploy dir
|
||||
# Verdict detection: check each report file's ## Summary section individually,
|
||||
# then aggregate across passes for a "X/Y REPRODUCED" badge.
|
||||
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
|
||||
if [ -d video-reviews ]; then
|
||||
for rpt in video-reviews/*-qa-video-report.md; do
|
||||
[ -f "$rpt" ] || continue
|
||||
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
|
||||
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
|
||||
# Check negatives FIRST — "fails to reproduce" contains "reproduce" but is negative
|
||||
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
|
||||
INCONC_COUNT=$((INCONC_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|unable to reproduce\|fails\? to reproduce\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed'; then
|
||||
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
|
||||
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
|
||||
REPRO_COUNT=$((REPRO_COUNT + 1))
|
||||
fi
|
||||
done
|
||||
fi
|
||||
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
|
||||
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
|
||||
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
|
||||
echo "Verdict: ${REPRO_COUNT}✓ ${NOT_REPRO_COUNT}✗ ${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
|
||||
|
||||
# Badge text:
|
||||
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
|
||||
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
|
||||
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
|
||||
if [ "$TOTAL_REPORTS" -le 1 ]; then
|
||||
# Single report — simple label
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
|
||||
elif [ "$FAIL_COUNT" -gt 0 ]; then
|
||||
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
else
|
||||
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
|
||||
PARTS=""
|
||||
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}✓"
|
||||
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}✗"
|
||||
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}⚠"
|
||||
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
|
||||
# Color based on best outcome
|
||||
if [ "$REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#2196f3"
|
||||
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
|
||||
REPRO_COLOR="#9f9f9f"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Badge label includes the target number for identification
|
||||
BADGE_LABEL="QA"
|
||||
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA"
|
||||
|
||||
if [ "$TARGET_TYPE" = "issue" ]; then
|
||||
if [ "$TOTAL_REPORTS" -gt 1 ]; then
|
||||
# Multi-pass: vertical box badge with breakdown
|
||||
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
|
||||
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS"
|
||||
else
|
||||
BADGE_STATUS="${REPRO_RESULT:-FINISHED}"
|
||||
/tmp/gen-badge.sh "$BADGE_STATUS" "${REPRO_COLOR}" "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL"
|
||||
fi
|
||||
else
|
||||
# Extract the Overall Risk section for fix quality verdict.
|
||||
# Only look at the "## Overall Risk" section to avoid false matches from
|
||||
# severity labels (e.g. `MAJOR`) or negated phrases ("no regressions").
|
||||
RISK_TEXT=""
|
||||
if [ -d video-reviews ]; then
|
||||
# Strip markdown bold/italic so **High** matches as "High"
|
||||
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20)
|
||||
fi
|
||||
# Check LOW first — "high confidence" contains "high" but means low risk
|
||||
SOLN_RESULT="" SOLN_COLOR="#4c1"
|
||||
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]')
|
||||
if [ -n "$RISK_FIRST" ]; then
|
||||
case "$RISK_FIRST" in
|
||||
*low*|*minimal*) SOLN_RESULT="APPROVED" SOLN_COLOR="#4c1" ;;
|
||||
*medium*|*moderate*) SOLN_RESULT="MINOR ISSUES" SOLN_COLOR="#dfb317" ;;
|
||||
*high*|*critical*) SOLN_RESULT="MAJOR ISSUES" SOLN_COLOR="#e05d44" ;;
|
||||
esac
|
||||
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
|
||||
SOLN_RESULT="APPROVED" SOLN_COLOR="#4c1"
|
||||
fi
|
||||
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN} | Fix: ${SOLN_RESULT:-UNKNOWN}"
|
||||
/tmp/gen-badge-dual.sh \
|
||||
"${REPRO_RESULT:-UNKNOWN}" "${REPRO_COLOR}" \
|
||||
"${SOLN_RESULT:-UNKNOWN}" "${SOLN_COLOR}" \
|
||||
"$DEPLOY_DIR/badge.svg" "$BADGE_LABEL"
|
||||
fi
|
||||
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
|
||||
URL=$(wrangler pages deploy "$DEPLOY_DIR" \
|
||||
--project-name="comfy-qa" \
|
||||
--branch="$BRANCH" 2>&1 \
|
||||
| grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1)
|
||||
|
||||
echo "url=${URL:-https://${BRANCH}.comfy-qa.pages.dev}" >> "$GITHUB_OUTPUT"
|
||||
echo "Deployed to: ${URL}"
|
||||
@@ -1,208 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
/**
|
||||
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
|
||||
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
|
||||
*
|
||||
* Usage:
|
||||
* pnpm exec tsx scripts/qa-generate-test.ts \
|
||||
* --qa-report <path> QA video review report (markdown)
|
||||
* --pr-diff <path> PR diff file
|
||||
* --output <path> Output .spec.ts file path
|
||||
* --model <name> Gemini model (default: gemini-3-flash-preview)
|
||||
*/
|
||||
import { readFile, writeFile } from 'node:fs/promises'
|
||||
import { basename, resolve } from 'node:path'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
|
||||
interface CliOptions {
|
||||
qaReport: string
|
||||
prDiff: string
|
||||
output: string
|
||||
model: string
|
||||
}
|
||||
|
||||
const DEFAULTS: CliOptions = {
|
||||
qaReport: '',
|
||||
prDiff: '',
|
||||
output: '',
|
||||
model: 'gemini-3-flash-preview'
|
||||
}
|
||||
|
||||
// ── Fixture API reference for the prompt ────────────────────────────
|
||||
const FIXTURE_API = `
|
||||
## ComfyUI Playwright Test Fixture API
|
||||
|
||||
Import pattern:
|
||||
\`\`\`typescript
|
||||
import { expect } from '@playwright/test'
|
||||
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
|
||||
\`\`\`
|
||||
|
||||
### Available helpers on \`comfyPage\`:
|
||||
- \`comfyPage.page\` — raw Playwright Page
|
||||
- \`comfyPage.menu.topbar\` — Topbar helper:
|
||||
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
|
||||
- \`.getActiveTabName(): Promise<string>\` — get active tab name
|
||||
- \`.saveWorkflow(name)\` — Save via File > Save dialog
|
||||
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
|
||||
- \`.exportWorkflow(name)\` — Export via File > Export dialog
|
||||
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
|
||||
- \`.getWorkflowTab(name)\` — get a tab locator by name
|
||||
- \`.closeWorkflowTab(name)\` — close a tab
|
||||
- \`.openTopbarMenu()\` — open the hamburger menu
|
||||
- \`.openSubmenu(label)\` — hover to open a submenu
|
||||
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
|
||||
- \`.open()\` / \`.close()\` — toggle sidebar
|
||||
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
|
||||
- \`.getPersistedItem(name)\` — get a workflow item locator
|
||||
- \`comfyPage.workflow\` — WorkflowHelper:
|
||||
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
|
||||
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
|
||||
- \`.deleteWorkflow(name)\` — delete a workflow
|
||||
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
|
||||
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
|
||||
- \`comfyPage.settings.setSetting(key, value)\` — change settings
|
||||
- \`comfyPage.keyboard\` — KeyboardHelper:
|
||||
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
|
||||
- \`comfyPage.nodeOps\` — NodeOperationsHelper
|
||||
- \`comfyPage.canvas\` — CanvasHelper
|
||||
- \`comfyPage.contextMenu\` — ContextMenu
|
||||
- \`comfyPage.toast\` — ToastHelper
|
||||
- \`comfyPage.confirmDialog\` — confirmation dialog
|
||||
- \`comfyPage.nextFrame()\` — wait for Vue re-render
|
||||
|
||||
### Test patterns:
|
||||
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
|
||||
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
|
||||
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
|
||||
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
|
||||
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
|
||||
`
|
||||
|
||||
// ── Prompt builder ──────────────────────────────────────────────────
|
||||
function buildPrompt(qaReport: string, prDiff: string): string {
|
||||
return `You are a Playwright test generator for the ComfyUI frontend.
|
||||
|
||||
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
|
||||
described in the QA report below. The test must:
|
||||
|
||||
1. Use the ComfyUI Playwright fixture API (documented below)
|
||||
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
|
||||
3. NOT test code implementation details
|
||||
4. Be concise — only test the behavior that the PR changed
|
||||
5. Follow existing test conventions (see API reference)
|
||||
|
||||
${FIXTURE_API}
|
||||
|
||||
## QA Video Review Report
|
||||
${qaReport}
|
||||
|
||||
## PR Diff (for context on what changed)
|
||||
${prDiff.slice(0, 8000)}
|
||||
|
||||
## Output Requirements
|
||||
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
|
||||
- Start with imports, end with closing brace
|
||||
- Use descriptive test names that explain the expected behavior
|
||||
- Add screenshot assertions where visual verification matters
|
||||
- Keep it focused: 2-5 test cases covering the core behavioral change
|
||||
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
|
||||
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
|
||||
`
|
||||
}
|
||||
|
||||
// ── Gemini call ─────────────────────────────────────────────────────
|
||||
async function generateTest(
|
||||
qaReport: string,
|
||||
prDiff: string,
|
||||
model: string
|
||||
): Promise<string> {
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
|
||||
|
||||
const genAI = new GoogleGenerativeAI(apiKey)
|
||||
const genModel = genAI.getGenerativeModel({ model })
|
||||
|
||||
const prompt = buildPrompt(qaReport, prDiff)
|
||||
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
|
||||
|
||||
const result = await genModel.generateContent({
|
||||
contents: [{ role: 'user', parts: [{ text: prompt }] }],
|
||||
generationConfig: {
|
||||
temperature: 0.2,
|
||||
maxOutputTokens: 8192
|
||||
}
|
||||
})
|
||||
|
||||
const text = result.response.text()
|
||||
|
||||
// Strip markdown fences if model wraps output
|
||||
return text
|
||||
.replace(/^```(?:typescript|ts)?\n?/, '')
|
||||
.replace(/\n?```$/, '')
|
||||
.trim()
|
||||
}
|
||||
|
||||
// ── CLI ─────────────────────────────────────────────────────────────
|
||||
function parseArgs(): CliOptions {
|
||||
const args = process.argv.slice(2)
|
||||
const opts = { ...DEFAULTS }
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--qa-report':
|
||||
opts.qaReport = args[++i]
|
||||
break
|
||||
case '--pr-diff':
|
||||
opts.prDiff = args[++i]
|
||||
break
|
||||
case '--output':
|
||||
opts.output = args[++i]
|
||||
break
|
||||
case '--model':
|
||||
opts.model = args[++i]
|
||||
break
|
||||
case '--help':
|
||||
console.warn(`Usage:
|
||||
pnpm exec tsx scripts/qa-generate-test.ts [options]
|
||||
|
||||
Options:
|
||||
--qa-report <path> QA video review report (markdown) [required]
|
||||
--pr-diff <path> PR diff file [required]
|
||||
--output <path> Output .spec.ts path [required]
|
||||
--model <name> Gemini model (default: gemini-3-flash-preview)`)
|
||||
process.exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
if (!opts.qaReport || !opts.prDiff || !opts.output) {
|
||||
console.error('Missing required args. Run with --help for usage.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
return opts
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs()
|
||||
|
||||
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
|
||||
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
|
||||
|
||||
console.warn(
|
||||
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
|
||||
)
|
||||
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
|
||||
|
||||
const testCode = await generateTest(qaReport, prDiff, opts.model)
|
||||
|
||||
const outputPath = resolve(opts.output)
|
||||
await writeFile(outputPath, testCode + '\n')
|
||||
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err)
|
||||
process.exit(1)
|
||||
})
|
||||
1711
scripts/qa-record.ts
@@ -1,130 +0,0 @@
|
||||
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
|
||||
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<style>
|
||||
:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
|
||||
*{margin:0;padding:0;box-sizing:border-box}
|
||||
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
|
||||
body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}
|
||||
.container{max-width:1200px;margin:0 auto}
|
||||
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
|
||||
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
|
||||
.header-icon svg{color:var(--primary)}
|
||||
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
|
||||
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
|
||||
.card{background:linear-gradient(135deg,oklch(100% 0 0/.05),oklch(100% 0 0/.015));backdrop-filter:blur(16px) saturate(150%);border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
|
||||
.card:hover{border-color:oklch(100% 0 0/.16);box-shadow:0 8px 32px oklch(0% 0 0/.3),inset 0 1px 0 oklch(100% 0 0/.1);transform:translateY(-2px)}
|
||||
.video-wrap{position:relative;background:oklch(4% 0.01 265);border-bottom:1px solid var(--border-faint)}
|
||||
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
|
||||
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
|
||||
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
|
||||
.icon{font-size:1.125rem}
|
||||
.links{display:flex;gap:.5rem}
|
||||
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
|
||||
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
|
||||
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
|
||||
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
|
||||
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
|
||||
.comp-panel{border-right:1px solid var(--border-faint)}
|
||||
.comp-panel:last-child{border-right:none}
|
||||
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:oklch(10% 0.01 265);display:flex;align-items:center;gap:.4rem}
|
||||
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
|
||||
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
|
||||
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
|
||||
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
|
||||
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
|
||||
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
|
||||
.report summary:hover{color:var(--fg)}
|
||||
.report summary svg{flex-shrink:0;opacity:.5}
|
||||
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
|
||||
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
|
||||
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
|
||||
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
|
||||
.report-body p{margin:.4rem 0}
|
||||
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
|
||||
.report-body li{margin:.25rem 0}
|
||||
.report-body code{background:oklch(16% 0.02 265);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
|
||||
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
|
||||
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
|
||||
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
|
||||
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
|
||||
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
|
||||
.report-body th{background:oklch(14% 0.02 265);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
|
||||
.report-body tr:nth-child(even){background:oklch(10% 0.01 265/.5)}
|
||||
.report-body tr:hover{background:oklch(16% 0.02 265/.5)}
|
||||
.report-body strong{color:var(--fg)}
|
||||
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
|
||||
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
|
||||
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
|
||||
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
|
||||
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
|
||||
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
|
||||
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
|
||||
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
|
||||
.badge-img{height:20px;display:block}
|
||||
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
|
||||
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
|
||||
.vseek{width:100%;padding:0 .75rem;background:oklch(6% 0.01 265);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
|
||||
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
|
||||
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
|
||||
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
|
||||
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
|
||||
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
|
||||
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:oklch(6% 0.01 265);border-top:1px solid var(--border-faint);flex-wrap:wrap}
|
||||
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
|
||||
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
|
||||
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
|
||||
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
|
||||
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
|
||||
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
|
||||
</style></head><body><div class=container>
|
||||
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend · Automated QA{{COMMIT_HTML}}{{RUN_LINK}}</div>{{BADGE_HTML}}</div></header>
|
||||
<div class=grid>{{CARDS}}</div>
|
||||
</div><script>
|
||||
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
|
||||
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
|
||||
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
|
||||
document.querySelectorAll('.video-wrap video').forEach(v=>{
|
||||
v.playbackRate=0.5;v.removeAttribute('autoplay');v.pause();
|
||||
const c=document.createElement('div');c.className='vctrl';
|
||||
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
|
||||
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
|
||||
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
|
||||
btn('\u23EE',()=>{v.currentTime=0});
|
||||
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
|
||||
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
|
||||
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
|
||||
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
|
||||
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
|
||||
sep();
|
||||
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===0.5)b.classList.add('active');return b});
|
||||
sep();c.appendChild(time);
|
||||
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
|
||||
// Custom seekbar — works even without server range request support
|
||||
const seekWrap=document.createElement('div');seekWrap.className='vseek';
|
||||
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
|
||||
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
|
||||
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
|
||||
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
|
||||
let seeking=false;
|
||||
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
|
||||
seekBar.onchange=()=>{seeking=false};
|
||||
v.closest('.video-wrap').after(seekWrap);
|
||||
seekWrap.after(c);
|
||||
v.ontimeupdate=()=>{
|
||||
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
|
||||
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
|
||||
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
|
||||
};
|
||||
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
|
||||
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
|
||||
v.parentElement.addEventListener('keydown',e=>{
|
||||
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
|
||||
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
|
||||
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
|
||||
});
|
||||
v.parentElement.setAttribute('tabindex','0');
|
||||
});
|
||||
</script></body></html>
|
||||
@@ -1,150 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import {
|
||||
extractPlatformFromArtifactDirName,
|
||||
pickLatestVideosByPlatform,
|
||||
selectVideoCandidateByFile
|
||||
} from './qa-video-review'
|
||||
|
||||
describe('extractPlatformFromArtifactDirName', () => {
|
||||
it('extracts and normalizes known qa artifact directory names', () => {
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
|
||||
).toBe('windows')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
|
||||
).toBe('macos')
|
||||
expect(
|
||||
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
|
||||
).toBe('linux')
|
||||
})
|
||||
|
||||
it('falls back to slugifying unknown directory names', () => {
|
||||
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
|
||||
'custom-platform-run'
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('pickLatestVideosByPlatform', () => {
|
||||
it('keeps only the latest candidate per platform', () => {
|
||||
const selected = pickLatestVideosByPlatform([
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-old.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
}
|
||||
])
|
||||
|
||||
expect(selected).toEqual([
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/linux.mp4',
|
||||
mtimeMs: 150
|
||||
},
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/windows-new.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('selectVideoCandidateByFile', () => {
|
||||
it('selects a single candidate by artifacts-relative path', () => {
|
||||
const selected = selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-Linux-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
|
||||
expect(selected).toEqual({
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
})
|
||||
})
|
||||
|
||||
it('throws when basename matches multiple videos', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
},
|
||||
{
|
||||
platformName: 'linux',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
|
||||
mtimeMs: 200
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('matched 2 videos')
|
||||
})
|
||||
|
||||
it('throws when there is no matching video', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: 'qa-report-macOS-1/qa-session.mp4'
|
||||
}
|
||||
)
|
||||
).toThrow('No video matched')
|
||||
})
|
||||
|
||||
it('throws when video file is missing', () => {
|
||||
expect(() =>
|
||||
selectVideoCandidateByFile(
|
||||
[
|
||||
{
|
||||
platformName: 'windows',
|
||||
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
|
||||
mtimeMs: 100
|
||||
}
|
||||
],
|
||||
{
|
||||
artifactsDir: '/tmp/qa-artifacts',
|
||||
videoFile: ' '
|
||||
}
|
||||
)
|
||||
).toThrow('--video-file is required')
|
||||
})
|
||||
})
|
||||
@@ -1,743 +0,0 @@
|
||||
#!/usr/bin/env tsx
|
||||
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
|
||||
import { basename, dirname, extname, relative, resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { GoogleGenerativeAI } from '@google/generative-ai'
|
||||
import { globSync } from 'glob'
|
||||
|
||||
interface CliOptions {
|
||||
artifactsDir: string
|
||||
videoFile: string
|
||||
beforeVideo: string
|
||||
outputDir: string
|
||||
model: string
|
||||
requestTimeoutMs: number
|
||||
dryRun: boolean
|
||||
prContext: string
|
||||
targetUrl: string
|
||||
passLabel: string
|
||||
}
|
||||
|
||||
interface VideoCandidate {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
mtimeMs: number
|
||||
}
|
||||
|
||||
const DEFAULT_OPTIONS: CliOptions = {
|
||||
artifactsDir: './tmp/qa-artifacts',
|
||||
videoFile: '',
|
||||
beforeVideo: '',
|
||||
outputDir: './tmp',
|
||||
model: 'gemini-3-flash-preview',
|
||||
requestTimeoutMs: 300_000,
|
||||
dryRun: false,
|
||||
prContext: '',
|
||||
targetUrl: '',
|
||||
passLabel: ''
|
||||
}
|
||||
|
||||
const USAGE = `Usage:
|
||||
pnpm exec tsx scripts/qa-video-review.ts [options]
|
||||
|
||||
Options:
|
||||
--artifacts-dir <path> Artifacts root directory
|
||||
(default: ./tmp/qa-artifacts)
|
||||
--video-file <name-or-path> Video file to analyze (required)
|
||||
(supports basename or relative/absolute path)
|
||||
--before-video <path> Before video (main branch) for comparison
|
||||
When provided, sends both videos to Gemini
|
||||
for comparative before/after analysis
|
||||
--output-dir <path> Output directory for markdown reports
|
||||
(default: ./tmp)
|
||||
--model <name> Gemini model
|
||||
(default: gemini-3-flash-preview)
|
||||
--request-timeout-ms <n> Request timeout in milliseconds
|
||||
(default: 300000)
|
||||
--pr-context <file> File with PR context (title, body, diff)
|
||||
for PR-aware review
|
||||
--target-url <url> Issue or PR URL to include in the report
|
||||
--pass-label <label> Label for multi-pass reports (e.g. pass1)
|
||||
Output becomes {platform}-{label}-qa-video-report.md
|
||||
--dry-run Discover videos and output targets only
|
||||
--help Show this help text
|
||||
|
||||
Environment:
|
||||
GEMINI_API_KEY Required unless --dry-run
|
||||
`
|
||||
|
||||
function parsePositiveInteger(rawValue: string, flagName: string): number {
|
||||
const parsedValue = Number.parseInt(rawValue, 10)
|
||||
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
|
||||
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
|
||||
}
|
||||
return parsedValue
|
||||
}
|
||||
|
||||
function parseCliOptions(args: string[]): CliOptions {
|
||||
const options: CliOptions = { ...DEFAULT_OPTIONS }
|
||||
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const argument = args[index]
|
||||
const nextValue = args[index + 1]
|
||||
const requireValue = (flagName: string): string => {
|
||||
if (!nextValue || nextValue.startsWith('--')) {
|
||||
throw new Error(`Missing value for ${flagName}`)
|
||||
}
|
||||
index += 1
|
||||
return nextValue
|
||||
}
|
||||
|
||||
if (argument === '--help') {
|
||||
process.stdout.write(USAGE)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
if (argument === '--artifacts-dir') {
|
||||
options.artifactsDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--video-file') {
|
||||
options.videoFile = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--output-dir') {
|
||||
options.outputDir = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--model') {
|
||||
options.model = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--request-timeout-ms') {
|
||||
options.requestTimeoutMs = parsePositiveInteger(
|
||||
requireValue(argument),
|
||||
argument
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--before-video') {
|
||||
options.beforeVideo = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pr-context') {
|
||||
options.prContext = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--target-url') {
|
||||
options.targetUrl = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--pass-label') {
|
||||
options.passLabel = requireValue(argument)
|
||||
continue
|
||||
}
|
||||
|
||||
if (argument === '--dry-run') {
|
||||
options.dryRun = true
|
||||
continue
|
||||
}
|
||||
|
||||
throw new Error(`Unknown argument: ${argument}`)
|
||||
}
|
||||
|
||||
return options
|
||||
}
|
||||
|
||||
function normalizePlatformName(value: string): string {
|
||||
const slug = value
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
|
||||
return slug.length > 0 ? slug : 'unknown-platform'
|
||||
}
|
||||
|
||||
export function extractPlatformFromArtifactDirName(dirName: string): string {
|
||||
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
|
||||
return normalizePlatformName(matchedValue ?? dirName)
|
||||
}
|
||||
|
||||
function extractPlatformFromVideoPath(videoPath: string): string {
|
||||
const artifactDirName = basename(dirname(videoPath))
|
||||
return extractPlatformFromArtifactDirName(artifactDirName)
|
||||
}
|
||||
|
||||
export function pickLatestVideosByPlatform(
|
||||
candidates: VideoCandidate[]
|
||||
): VideoCandidate[] {
|
||||
const latestByPlatform = new Map<string, VideoCandidate>()
|
||||
|
||||
for (const candidate of candidates) {
|
||||
const current = latestByPlatform.get(candidate.platformName)
|
||||
if (!current || candidate.mtimeMs > current.mtimeMs) {
|
||||
latestByPlatform.set(candidate.platformName, candidate)
|
||||
}
|
||||
}
|
||||
|
||||
return [...latestByPlatform.values()].sort((a, b) =>
|
||||
a.platformName.localeCompare(b.platformName)
|
||||
)
|
||||
}
|
||||
|
||||
function toProjectRelativePath(targetPath: string): string {
|
||||
const relativePath = relative(process.cwd(), targetPath)
|
||||
if (relativePath.startsWith('.')) {
|
||||
return relativePath
|
||||
}
|
||||
return `./${relativePath}`
|
||||
}
|
||||
|
||||
function errorToString(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
|
||||
function normalizePathForMatch(value: string): string {
|
||||
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
|
||||
}
|
||||
|
||||
export function selectVideoCandidateByFile(
|
||||
candidates: VideoCandidate[],
|
||||
options: { artifactsDir: string; videoFile: string }
|
||||
): VideoCandidate {
|
||||
const requestedValue = options.videoFile.trim()
|
||||
if (requestedValue.length === 0) {
|
||||
throw new Error('--video-file is required')
|
||||
}
|
||||
|
||||
const artifactsRoot = resolve(options.artifactsDir)
|
||||
const requestedAbsolutePath = resolve(requestedValue)
|
||||
const requestedPathKey = normalizePathForMatch(requestedValue)
|
||||
|
||||
const matches = candidates.filter((candidate) => {
|
||||
const candidateAbsolutePath = resolve(candidate.videoPath)
|
||||
if (candidateAbsolutePath === requestedAbsolutePath) {
|
||||
return true
|
||||
}
|
||||
|
||||
const candidateBaseName = basename(candidate.videoPath)
|
||||
if (candidateBaseName === requestedValue) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToCwd = normalizePathForMatch(
|
||||
relative(process.cwd(), candidateAbsolutePath)
|
||||
)
|
||||
if (relativeToCwd === requestedPathKey) {
|
||||
return true
|
||||
}
|
||||
|
||||
const relativeToArtifacts = normalizePathForMatch(
|
||||
relative(artifactsRoot, candidateAbsolutePath)
|
||||
)
|
||||
return relativeToArtifacts === requestedPathKey
|
||||
})
|
||||
|
||||
if (matches.length === 1) {
|
||||
return matches[0]
|
||||
}
|
||||
|
||||
if (matches.length === 0) {
|
||||
const availableVideos = candidates.map((candidate) =>
|
||||
toProjectRelativePath(candidate.videoPath)
|
||||
)
|
||||
throw new Error(
|
||||
[
|
||||
`No video matched --video-file "${options.videoFile}".`,
|
||||
'Available videos:',
|
||||
...availableVideos.map((videoPath) => `- ${videoPath}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
[
|
||||
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
|
||||
'Please pass a more specific path.',
|
||||
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
async function collectVideoCandidates(
|
||||
artifactsDir: string
|
||||
): Promise<VideoCandidate[]> {
|
||||
const absoluteArtifactsDir = resolve(artifactsDir)
|
||||
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
|
||||
cwd: absoluteArtifactsDir,
|
||||
absolute: true,
|
||||
nodir: true
|
||||
}).sort()
|
||||
|
||||
const candidates = await Promise.all(
|
||||
videoPaths.map(async (videoPath) => {
|
||||
const videoStat = await stat(videoPath)
|
||||
return {
|
||||
platformName: extractPlatformFromVideoPath(videoPath),
|
||||
videoPath,
|
||||
mtimeMs: videoStat.mtimeMs
|
||||
}
|
||||
})
|
||||
)
|
||||
|
||||
return candidates
|
||||
}
|
||||
|
||||
function getMimeType(filePath: string): string {
|
||||
const ext = extname(filePath).toLowerCase()
|
||||
const mimeMap: Record<string, string> = {
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime',
|
||||
'.avi': 'video/x-msvideo',
|
||||
'.mkv': 'video/x-matroska',
|
||||
'.m4v': 'video/mp4'
|
||||
}
|
||||
return mimeMap[ext] || 'video/mp4'
|
||||
}
|
||||
|
||||
function buildReviewPrompt(options: {
|
||||
platformName: string
|
||||
videoPath: string
|
||||
prContext: string
|
||||
isComparative: boolean
|
||||
}): string {
|
||||
const { platformName, videoPath, prContext, isComparative } = options
|
||||
|
||||
if (isComparative) {
|
||||
return buildComparativePrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
return buildSingleVideoPrompt(platformName, videoPath, prContext)
|
||||
}
|
||||
|
||||
function buildComparativePrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
|
||||
'',
|
||||
'You are given TWO videos:',
|
||||
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
|
||||
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
|
||||
'',
|
||||
'Both videos show the same test steps executed on different code versions.',
|
||||
''
|
||||
]
|
||||
|
||||
if (prContext) {
|
||||
lines.push('## PR Context', prContext, '')
|
||||
}
|
||||
|
||||
lines.push(
|
||||
'## Your Task',
|
||||
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'',
|
||||
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
|
||||
' Describe what you observe — this establishes the baseline.',
|
||||
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
|
||||
'3. **Comparison**: What specifically changed between before and after?',
|
||||
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
|
||||
' that were NOT present in the BEFORE video?',
|
||||
'',
|
||||
'Note: Brief black frames during page transitions are NORMAL.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible differences. Avoid speculation.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
|
||||
'',
|
||||
'## Behavior Changes',
|
||||
'Summarize ALL behavioral differences as a markdown TABLE:',
|
||||
'| Behavior | Before (main) | After (PR) | Verdict |',
|
||||
'',
|
||||
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
|
||||
'- **Before (main)**: how it works/looks in the BEFORE video',
|
||||
'- **After (PR)**: how it works/looks in the AFTER video',
|
||||
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
|
||||
'',
|
||||
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
|
||||
'that were tested, so reviewers can confirm nothing was missed.',
|
||||
'',
|
||||
'## Timeline Comparison',
|
||||
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
|
||||
'| Time | Type | Severity | Before (main) | After (PR) |',
|
||||
'',
|
||||
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
|
||||
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
|
||||
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
|
||||
'- **Before (main)**: what is observed in the BEFORE video at that time',
|
||||
'- **After (PR)**: what is observed in the AFTER video at that time',
|
||||
'',
|
||||
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
|
||||
'omit that row. Focus on meaningful differences, not narrating every frame.',
|
||||
'',
|
||||
'## Confirmed Issues',
|
||||
'For each issue, use this exact format:',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'',
|
||||
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
|
||||
'',
|
||||
'**Evidence:** [What you observed at the given timestamp in which video]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk',
|
||||
'(Assess whether the PR achieves its goal based on the before/after comparison)'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
function buildSingleVideoPrompt(
|
||||
platformName: string,
|
||||
videoPath: string,
|
||||
prContext: string
|
||||
): string {
|
||||
const lines = [
|
||||
'You are a senior QA engineer reviewing a UI test session recording.',
|
||||
''
|
||||
]
|
||||
|
||||
const isIssueContext =
|
||||
prContext &&
|
||||
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
|
||||
prContext
|
||||
)
|
||||
|
||||
if (prContext) {
|
||||
if (isIssueContext) {
|
||||
lines.push(
|
||||
'## Issue Context',
|
||||
'This video attempts to reproduce a reported bug on the main branch.',
|
||||
'Your review MUST evaluate whether the reported bug is visible and reproducible.',
|
||||
'',
|
||||
prContext,
|
||||
'',
|
||||
'## Review Instructions',
|
||||
'1. Does the video demonstrate the reported bug occurring?',
|
||||
'2. Is the bug clearly visible and reproducible from the steps shown?',
|
||||
'3. Are there any other issues visible during the reproduction attempt?',
|
||||
'',
|
||||
'## CRITICAL: Honesty Requirements',
|
||||
'- If the video only shows login, idle canvas, or trivial menu interactions WITHOUT actually performing the reproduction steps, say "INCONCLUSIVE — reproduction steps were not performed".',
|
||||
'- Do NOT claim a bug is "confirmed" unless you can clearly see the bug behavior described in the issue.',
|
||||
'- Do NOT hallucinate findings. If the video does not show meaningful interaction, say so clearly.',
|
||||
'- Rate confidence as "Low" if the video does not actually demonstrate the bug scenario.',
|
||||
''
|
||||
)
|
||||
} else {
|
||||
lines.push(
|
||||
'## PR Context',
|
||||
'The video is a QA session testing a specific pull request.',
|
||||
'Your review MUST evaluate whether the PR achieves its stated purpose.',
|
||||
'',
|
||||
prContext,
|
||||
'',
|
||||
'## Review Instructions',
|
||||
"1. Does the video demonstrate the PR's intended behavior working correctly?",
|
||||
'2. Are there regressions or side effects caused by the PR changes?',
|
||||
'3. Does the observed behavior match what the PR claims to implement/fix?',
|
||||
''
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
lines.push(
|
||||
`Review this QA session video for platform "${platformName}".`,
|
||||
`Source video: ${toProjectRelativePath(videoPath)}.`,
|
||||
'The video shows the full test session — analyze it chronologically.',
|
||||
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
|
||||
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
|
||||
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
|
||||
'Report only concrete, visible problems and avoid speculation.',
|
||||
'If confidence is low, mark it explicitly.',
|
||||
'',
|
||||
'Return markdown with these sections exactly:',
|
||||
'## Summary',
|
||||
isIssueContext
|
||||
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
|
||||
: prContext
|
||||
? '(Explain what the PR intended and whether the video confirms it works)'
|
||||
: '',
|
||||
'## Confirmed Issues',
|
||||
'For each confirmed issue, use this exact format (one block per issue):',
|
||||
'',
|
||||
'### [Short issue title]',
|
||||
'`HIGH` `01:03` `Confidence: High`',
|
||||
'',
|
||||
'[Description of the issue — what went wrong and what was expected]',
|
||||
'',
|
||||
'**Evidence:** [What you observed in the video at the given timestamp]',
|
||||
'',
|
||||
'**Suggested Fix:** [Actionable recommendation]',
|
||||
'',
|
||||
'---',
|
||||
'',
|
||||
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
|
||||
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
|
||||
'Do NOT use a table for issues — use the block format above.',
|
||||
'## Possible Issues (Needs Human Verification)',
|
||||
'## Overall Risk'
|
||||
)
|
||||
|
||||
return lines.filter(Boolean).join('\n')
|
||||
}
|
||||
|
||||
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
|
||||
|
||||
async function readVideoFile(videoPath: string): Promise<Buffer> {
|
||||
const fileStat = await stat(videoPath)
|
||||
if (fileStat.size > MAX_VIDEO_BYTES) {
|
||||
throw new Error(
|
||||
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
|
||||
)
|
||||
}
|
||||
return readFile(videoPath)
|
||||
}
|
||||
|
||||
async function requestGeminiReview(options: {
|
||||
apiKey: string
|
||||
model: string
|
||||
platformName: string
|
||||
videoPath: string
|
||||
beforeVideoPath: string
|
||||
timeoutMs: number
|
||||
prContext: string
|
||||
}): Promise<string> {
|
||||
const genAI = new GoogleGenerativeAI(options.apiKey)
|
||||
const model = genAI.getGenerativeModel({ model: options.model })
|
||||
|
||||
const isComparative = options.beforeVideoPath.length > 0
|
||||
const prompt = buildReviewPrompt({
|
||||
platformName: options.platformName,
|
||||
videoPath: options.videoPath,
|
||||
prContext: options.prContext,
|
||||
isComparative
|
||||
})
|
||||
|
||||
const parts: Array<
|
||||
{ text: string } | { inlineData: { mimeType: string; data: string } }
|
||||
> = [{ text: prompt }]
|
||||
|
||||
if (isComparative) {
|
||||
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
|
||||
parts.push(
|
||||
{ text: 'Video 1 — BEFORE (main branch):' },
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.beforeVideoPath),
|
||||
data: beforeBuffer.toString('base64')
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
const afterBuffer = await readVideoFile(options.videoPath)
|
||||
if (isComparative) {
|
||||
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
|
||||
}
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mimeType: getMimeType(options.videoPath),
|
||||
data: afterBuffer.toString('base64')
|
||||
}
|
||||
})
|
||||
|
||||
const result = await model.generateContent(parts, {
|
||||
timeout: options.timeoutMs
|
||||
})
|
||||
const response = result.response
|
||||
const text = response.text()
|
||||
|
||||
if (!text || text.trim().length === 0) {
|
||||
throw new Error('Gemini API returned no output text')
|
||||
}
|
||||
|
||||
return text.trim()
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
function buildReportMarkdown(input: {
|
||||
platformName: string
|
||||
model: string
|
||||
videoPath: string
|
||||
videoSizeBytes: number
|
||||
beforeVideoPath?: string
|
||||
beforeVideoSizeBytes?: number
|
||||
reviewText: string
|
||||
targetUrl?: string
|
||||
}): string {
|
||||
const headerLines = [
|
||||
`# ${input.platformName} QA Video Report`,
|
||||
'',
|
||||
`- Generated at: ${new Date().toISOString()}`,
|
||||
`- Model: \`${input.model}\``
|
||||
]
|
||||
|
||||
if (input.targetUrl) {
|
||||
headerLines.push(`- Target: ${input.targetUrl}`)
|
||||
}
|
||||
|
||||
if (input.beforeVideoPath) {
|
||||
headerLines.push(
|
||||
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
|
||||
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
|
||||
'- Mode: **Comparative (before/after)**'
|
||||
)
|
||||
} else {
|
||||
headerLines.push(
|
||||
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
|
||||
`- Video size: ${formatBytes(input.videoSizeBytes)}`
|
||||
)
|
||||
}
|
||||
|
||||
headerLines.push('', '## AI Review', '')
|
||||
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
|
||||
}
|
||||
|
||||
async function reviewVideo(
|
||||
video: VideoCandidate,
|
||||
options: CliOptions,
|
||||
apiKey: string
|
||||
): Promise<void> {
|
||||
let prContext = ''
|
||||
if (options.prContext) {
|
||||
try {
|
||||
prContext = await readFile(options.prContext, 'utf-8')
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
|
||||
)
|
||||
} catch {
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const beforeVideoPath = options.beforeVideo
|
||||
? resolve(options.beforeVideo)
|
||||
: ''
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
|
||||
)
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
|
||||
)
|
||||
|
||||
const reviewText = await requestGeminiReview({
|
||||
apiKey,
|
||||
model: options.model,
|
||||
platformName: video.platformName,
|
||||
videoPath: video.videoPath,
|
||||
beforeVideoPath,
|
||||
timeoutMs: options.requestTimeoutMs,
|
||||
prContext
|
||||
})
|
||||
|
||||
const videoStat = await stat(video.videoPath)
|
||||
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
|
||||
const outputPath = resolve(
|
||||
options.outputDir,
|
||||
`${video.platformName}${passSegment}-qa-video-report.md`
|
||||
)
|
||||
|
||||
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
|
||||
platformName: video.platformName,
|
||||
model: options.model,
|
||||
videoPath: video.videoPath,
|
||||
videoSizeBytes: videoStat.size,
|
||||
reviewText,
|
||||
targetUrl: options.targetUrl || undefined
|
||||
}
|
||||
|
||||
if (beforeVideoPath) {
|
||||
const beforeStat = await stat(beforeVideoPath)
|
||||
reportInput.beforeVideoPath = beforeVideoPath
|
||||
reportInput.beforeVideoSizeBytes = beforeStat.size
|
||||
}
|
||||
|
||||
const reportMarkdown = buildReportMarkdown(reportInput)
|
||||
|
||||
await mkdir(dirname(outputPath), { recursive: true })
|
||||
await writeFile(outputPath, reportMarkdown, 'utf-8')
|
||||
|
||||
process.stdout.write(
|
||||
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
|
||||
)
|
||||
}
|
||||
|
||||
function isExecutedAsScript(metaUrl: string): boolean {
|
||||
const modulePath = fileURLToPath(metaUrl)
|
||||
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
|
||||
return modulePath === scriptPath
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const options = parseCliOptions(process.argv.slice(2))
|
||||
const candidates = await collectVideoCandidates(options.artifactsDir)
|
||||
|
||||
if (candidates.length === 0) {
|
||||
process.stdout.write(
|
||||
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
const selectedVideo = selectVideoCandidateByFile(candidates, {
|
||||
artifactsDir: options.artifactsDir,
|
||||
videoFile: options.videoFile
|
||||
})
|
||||
|
||||
process.stdout.write(
|
||||
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
|
||||
)
|
||||
|
||||
if (options.dryRun) {
|
||||
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
|
||||
return
|
||||
}
|
||||
|
||||
const apiKey = process.env.GEMINI_API_KEY
|
||||
if (!apiKey) {
|
||||
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
|
||||
}
|
||||
|
||||
await reviewVideo(selectedVideo, options, apiKey)
|
||||
}
|
||||
|
||||
if (isExecutedAsScript(import.meta.url)) {
|
||||
void main().catch((error: unknown) => {
|
||||
const message = errorToString(error)
|
||||
process.stderr.write(`qa-video-review failed: ${message}\n`)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
21
src/App.vue
@@ -9,10 +9,13 @@ import { captureException } from '@sentry/vue'
|
||||
import BlockUI from 'primevue/blockui'
|
||||
import { computed, onMounted, onUnmounted, watch } from 'vue'
|
||||
|
||||
import { useI18n } from 'vue-i18n'
|
||||
|
||||
import GlobalDialog from '@/components/dialog/GlobalDialog.vue'
|
||||
import config from '@/config'
|
||||
import { isDesktop } from '@/platform/distribution/types'
|
||||
import { useSettingStore } from '@/platform/settings/settingStore'
|
||||
import { useToastStore } from '@/platform/updates/common/toastStore'
|
||||
import { app } from '@/scripts/app'
|
||||
import { useWorkspaceStore } from '@/stores/workspaceStore'
|
||||
import { electronAPI } from '@/utils/envUtil'
|
||||
@@ -20,6 +23,7 @@ import { parsePreloadError } from '@/utils/preloadErrorUtil'
|
||||
import { useDialogService } from '@/services/dialogService'
|
||||
import { useConflictDetection } from '@/workbench/extensions/manager/composables/useConflictDetection'
|
||||
|
||||
const { t } = useI18n()
|
||||
const workspaceStore = useWorkspaceStore()
|
||||
app.extensionManager = useWorkspaceStore()
|
||||
|
||||
@@ -94,17 +98,12 @@ onMounted(() => {
|
||||
}
|
||||
})
|
||||
}
|
||||
// Disabled: Third-party custom node extensions frequently trigger this toast
|
||||
// (e.g., bare "vue" imports, wrong relative paths to scripts/app.js, missing
|
||||
// core dependencies). These are plugin bugs, not ComfyUI core failures, but
|
||||
// the generic error message alarms users and offers no actionable guidance.
|
||||
// The console.error above still logs the details for developers to debug.
|
||||
// useToastStore().add({
|
||||
// severity: 'error',
|
||||
// summary: t('g.preloadErrorTitle'),
|
||||
// detail: t('g.preloadError'),
|
||||
// life: 10000
|
||||
// })
|
||||
useToastStore().add({
|
||||
severity: 'error',
|
||||
summary: t('g.preloadErrorTitle'),
|
||||
detail: t('g.preloadError'),
|
||||
life: 10000
|
||||
})
|
||||
})
|
||||
|
||||
// Capture resource load failures (CSS, scripts) in non-localhost distributions
|
||||
|
||||