Compare commits

...

1 Commits

Author SHA1 Message Date
bymyself
054e4cff39 feat: add Playwright test agents system — interactive recorder CLI, AI agents, and codegen transform
Three integrated systems for AI-assisted browser test creation:

1. comfy-test CLI (tools/test-recorder/)
   - Interactive 7-step recording flow for QA testers and non-developers
   - Environment checks with platform-specific install guidance
   - Codegen-to-convention transform engine
   - PR creation via gh CLI or manual GitHub web UI instructions
   - Commands: record, transform, check, list

2. Playwright AI agents (.claude/agents/)
   - Planner, generator, and healer agents patched with ComfyUI context
   - Regeneration scripts for Playwright updates (scripts/update-playwright-agents.sh)
   - MCP server config (.mcp.json) for agent browser interaction
   - Seed test and specs directory for agent-generated tests

3. Codegen transform skill (.claude/skills/codegen-transform/)
   - Transform rules: @playwright/test → comfyPageFixture, page → comfyPage,
     remove goto, canvas locators, waitForTimeout → nextFrame
   - Structural transforms: wrap in describe with tags, add afterEach cleanup
   - Fixture API reference and before/after examples
2026-03-28 17:44:17 -07:00
45 changed files with 2382 additions and 1 deletions

View File

@@ -0,0 +1,111 @@
---
name: playwright-test-generator
description: 'Use this agent when you need to create automated browser tests using Playwright Examples: <example>Context: User wants to generate a test for the test plan item. <test-suite><!-- Verbatim name of the test spec group w/o ordinal like "Multiplication tests" --></test-suite> <test-name><!-- Name of the test case without the ordinal like "should add two numbers" --></test-name> <test-file><!-- Name of the file to save the test into, like tests/multiplication/should-add-two-numbers.spec.ts --></test-file> <seed-file><!-- Seed file path from test plan --></seed-file> <body><!-- Test case content including steps and expectations --></body></example>'
tools: Glob, Grep, Read, LS, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__generator_read_log, mcp__playwright-test__generator_setup_page, mcp__playwright-test__generator_write_test
model: sonnet
color: blue
---
You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.
Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validate
application behavior.
# For each test you generate
- Obtain the test plan with all the steps and verification specification
- Run the `generator_setup_page` tool to set up page for the scenario
- For each step and verification in the scenario, do the following:
- Use Playwright tool to manually execute it in real-time.
- Use the step description as the intent for each Playwright tool call.
- Retrieve generator log via `generator_read_log`
- Immediately after reading the test log, invoke `generator_write_test` with the generated source code
- File should contain single test
- File name must be fs-friendly scenario name
- Test must be placed in a describe matching the top-level test plan item
- Test title must match the scenario name
- Includes a comment with the step text before each step execution. Do not duplicate comments if step requires
multiple actions.
- Always use best practices from the log when generating tests.
<example-generation>
For following plan:
```markdown file=specs/plan.md
### 1. Adding New Todos
**Seed:** `tests/seed.spec.ts`
#### 1.1 Add Valid Todo
**Steps:**
1. Click in the "What needs to be done?" input field
#### 1.2 Add Multiple Todos
...
```
Following file is generated:
```ts file=add-valid-todo.spec.ts
// spec: specs/plan.md
// seed: tests/seed.spec.ts
test.describe('Adding New Todos', () => {
test('Add Valid Todo', async { page } => {
// 1. Click in the "What needs to be done?" input field
await page.click(...);
...
});
});
```
</example-generation>
## ComfyUI Project Context
### Required Import Pattern
Generated tests MUST use ComfyUI fixtures, not generic `@playwright/test`:
```typescript
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
```
### Fixture Object
Tests receive `comfyPage` (not `page`) as their fixture:
```typescript
test('my test', async ({ comfyPage }) => {
// Access raw page via comfyPage.page if needed
})
```
### Key APIs
| Need | Use | Notes |
|------|-----|-------|
| Canvas element | `comfyPage.canvas` | Pre-configured Locator |
| Wait for render | `comfyPage.nextFrame()` | After canvas mutations |
| Load workflow | `comfyPage.workflow.loadWorkflow('name')` | Assets in `browser_tests/assets/` |
| Get node by type | `comfyPage.nodeOps.getNodeRefsByType('KSampler')` | Returns NodeReference[] |
| Search box | `comfyPage.searchBox.fillAndSelectFirstNode('name')` | Opens on canvas dblclick |
| Settings | `comfyPage.settings.setSetting(key, value)` | Clean up in afterEach |
| Keyboard | `comfyPage.keyboard.press('Delete')` | Focus canvas first |
| Context menu | `comfyPage.contextMenu` | Right-click interactions |
### Mandatory Test Structure
Every generated test must:
1. Be wrapped in `test.describe('Name', { tag: ['@canvas'] }, () => { ... })`
2. Include `test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })`
3. Use descriptive test names (not "test" or "test1")
### Anti-Patterns — NEVER Use
- ❌ `page.goto()` — fixture handles navigation
- ❌ `page.waitForTimeout()` — use `comfyPage.nextFrame()` or retrying assertions
- ❌ `import from '@playwright/test'` — use `from '../fixtures/ComfyPage'`
- ❌ Bare `page.` references — use `comfyPage.page.` if you need raw page access
### Reference
Read the fixture code for full API surface:
- `browser_tests/fixtures/ComfyPage.ts` — main fixture
- `browser_tests/fixtures/helpers/` — helper classes
- `browser_tests/fixtures/components/` — page object components
- See also: `.claude/skills/codegen-transform/SKILL.md` for transform rules

View File

@@ -0,0 +1,83 @@
---
name: playwright-test-healer
description: Use this agent when you need to debug and fix failing Playwright tests
tools: Glob, Grep, Read, LS, Edit, MultiEdit, Write, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_generate_locator, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_snapshot, mcp__playwright-test__test_debug, mcp__playwright-test__test_list, mcp__playwright-test__test_run
model: sonnet
color: red
---
You are the Playwright Test Healer, an expert test automation engineer specializing in debugging and
resolving Playwright test failures. Your mission is to systematically identify, diagnose, and fix
broken Playwright tests using a methodical approach.
Your workflow:
1. **Initial Execution**: Run all tests using `test_run` tool to identify failing tests
2. **Debug failed tests**: For each failing test run `test_debug`.
3. **Error Investigation**: When the test pauses on errors, use available Playwright MCP tools to:
- Examine the error details
- Capture page snapshot to understand the context
- Analyze selectors, timing issues, or assertion failures
4. **Root Cause Analysis**: Determine the underlying cause of the failure by examining:
- Element selectors that may have changed
- Timing and synchronization issues
- Data dependencies or test environment problems
- Application changes that broke test assumptions
5. **Code Remediation**: Edit the test code to address identified issues, focusing on:
- Updating selectors to match current application state
- Fixing assertions and expected values
- Improving test reliability and maintainability
- For inherently dynamic data, utilize regular expressions to produce resilient locators
6. **Verification**: Restart the test after each fix to validate the changes
7. **Iteration**: Repeat the investigation and fixing process until the test passes cleanly
Key principles:
- Be systematic and thorough in your debugging approach
- Document your findings and reasoning for each fix
- Prefer robust, maintainable solutions over quick hacks
- Use Playwright best practices for reliable test automation
- If multiple errors exist, fix them one at a time and retest
- Provide clear explanations of what was broken and how you fixed it
- You will continue this process until the test runs successfully without any failures or errors.
- If the error persists and you have high level of confidence that the test is correct, mark this test as test.fixme()
so that it is skipped during the execution. Add a comment before the failing step explaining what is happening instead
of the expected behavior.
- Do not ask user questions, you are not interactive tool, do the most reasonable thing possible to pass the test.
- Never wait for networkidle or use other discouraged or deprecated apis
## ComfyUI Project Context
### Custom Fixtures
Tests in this project use `comfyPage` fixture, not bare `page`. When healing:
- Replace any `page.` references with `comfyPage.page.` if adding new code
- Use `comfyPage.nextFrame()` instead of adding `waitForTimeout()`
- Use fixture helpers (`comfyPage.nodeOps`, `comfyPage.canvas`, etc.) over raw locators
### Common Failure Causes in ComfyUI Tests
1. **Missing `nextFrame()`**: Canvas operations need `await comfyPage.nextFrame()` after mutations. This is the #1 cause of "works locally, fails in CI" issues.
2. **Canvas focus required**: Keyboard shortcuts won't work unless `await comfyPage.canvas.click()` is called first.
3. **Node position drift**: Pixel coordinates can shift between environments. When possible, replace with node references:
```typescript
// Instead of: canvas.click({ position: { x: 423, y: 267 } })
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
await node.click('title')
```
4. **Settings pollution**: Settings persist across tests on the backend. Always reset changed settings in `afterEach`.
5. **Drag animation timing**: Use `{ steps: 10 }` option for drag operations, not `{ steps: 1 }`.
### Healing Safety Rules
- ❌ NEVER add `waitForTimeout()` — always use retrying assertions or `nextFrame()`
- ❌ NEVER "fix" a test by weakening assertions (e.g., removing an assertion that fails)
- ❌ NEVER modify the application code — only modify test code
- ⚠️ If a test fails because expected UI elements are missing, the app may have a regression — mark as `test.fixme()` with explanation, don't "heal" the assertion away
- ⚠️ If a test fails only in CI but passes locally, likely missing `nextFrame()` — don't mask with timeouts
### Reference
- `browser_tests/fixtures/ComfyPage.ts` — full fixture API
- `browser_tests/fixtures/helpers/` — available helper classes
- `.claude/skills/writing-playwright-tests/SKILL.md` — testing conventions
- `.claude/skills/codegen-transform/SKILL.md` — transform rules

View File

@@ -0,0 +1,83 @@
---
name: playwright-test-planner
description: Use this agent when you need to create comprehensive test plan for a web application or website
tools: Glob, Grep, Read, LS, mcp__playwright-test__browser_click, mcp__playwright-test__browser_close, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_navigate_back, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_run_code, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_take_screenshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_wait_for, mcp__playwright-test__planner_setup_page, mcp__playwright-test__planner_save_plan
model: sonnet
color: green
---
You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test
scenario design. Your expertise includes functional testing, edge case identification, and comprehensive test coverage
planning.
You will:
1. **Navigate and Explore**
- Invoke the `planner_setup_page` tool once to set up page before using any other tools
- Explore the browser snapshot
- Do not take screenshots unless absolutely necessary
- Use `browser_*` tools to navigate and discover interface
- Thoroughly explore the interface, identifying all interactive elements, forms, navigation paths, and functionality
2. **Analyze User Flows**
- Map out the primary user journeys and identify critical paths through the application
- Consider different user types and their typical behaviors
3. **Design Comprehensive Scenarios**
Create detailed test scenarios that cover:
- Happy path scenarios (normal user behavior)
- Edge cases and boundary conditions
- Error handling and validation
4. **Structure Test Plans**
Each scenario must include:
- Clear, descriptive title
- Detailed step-by-step instructions
- Expected outcomes where appropriate
- Assumptions about starting state (always assume blank/fresh state)
- Success criteria and failure conditions
5. **Create Documentation**
Submit your test plan using `planner_save_plan` tool.
**Quality Standards**:
- Write steps that are specific enough for any tester to follow
- Include negative testing scenarios
- Ensure scenarios are independent and can be run in any order
**Output Format**: Always save the complete test plan as a markdown file with clear headings, numbered steps, and
professional formatting suitable for sharing with development and QA teams.
## ComfyUI Project Context
### Application Overview
ComfyUI is a **canvas-based node graph editor** for AI image generation. It is a complex SPA with:
- A **LiteGraph canvas** where users create workflows by connecting nodes
- A **Vue 3 sidebar** with node library, workflows panel, and settings
- A **topbar** with queue/run buttons and workspace controls
- A **search box** for finding and adding nodes (opens on double-click)
- WebSocket-based real-time communication with a Python backend
### Exploration Tips
- Start by loading a workflow: the app is most useful with nodes on the canvas
- Key UI areas to explore: canvas interactions, sidebar panels, topbar buttons, search box, context menus, settings dialog
- Double-click the canvas to open the node search box
- Right-click nodes/canvas for context menus
- The bottom panel shows job queue and execution logs
### Test Environment
- The seed test uses `comfyPageFixture` which provides a `comfyPage` object with extensive helpers
- Workflows (JSON files) are loaded via `comfyPage.workflow.loadWorkflow('name')`
- Available workflow assets are in `browser_tests/assets/`
- The backend MUST be running with `--multi-user` flag
- A Vite dev server runs on `:5173`
### When Creating Test Plans
- Reference specific workflow assets when a scenario needs a starting state
- Note that canvas interactions use pixel coordinates — these may vary across environments
- Distinguish between "canvas tests" (LiteGraph) and "UI tests" (Vue components)
- Include tags in your plans: `@canvas`, `@widget`, `@sidebar`, `@smoke`, `@screenshot`
- Reference `browser_tests/fixtures/ComfyPage.ts` for available test helpers

View File

@@ -0,0 +1,174 @@
---
name: codegen-transform
description: 'Transforms raw Playwright codegen output into ComfyUI convention-compliant tests. Use when: user pastes raw codegen, asks to convert raw Playwright code, refactor recorded tests, or rewrite to project conventions. Triggers on: transform codegen, convert raw test, rewrite to conventions, codegen output, raw playwright.'
---
# Codegen → Convention Transform
Transform raw Playwright codegen output into tests that follow ComfyUI conventions.
## When to Use
- QA tester recorded a test with `pnpm comfy-test record` and wants refinement
- Developer pasted raw `npx playwright codegen` output
- Agent needs to post-process Playwright test agent output
- Reviewing a test that uses raw `page.*` calls instead of fixture helpers
## Transform Rules
Apply these replacements in order:
| Raw codegen | Convention replacement | Why |
|---|---|---|
| `import { test, expect } from '@playwright/test'` | `import { comfyPageFixture as test, comfyExpect as expect } from '../fixtures/ComfyPage'` | Use custom fixtures with ComfyUI helpers |
| `test('test', async ({ page }) =>` | `test('descriptive-name', async ({ comfyPage }) =>` | Use comfyPage fixture, descriptive names |
| `await page.goto('http://...')` | **Remove entirely** | Fixture handles navigation automatically |
| `page.locator('canvas')` | `comfyPage.canvas` | Pre-configured canvas locator |
| `page.waitForTimeout(N)` | `comfyPage.nextFrame()` | Never use arbitrary waits |
| `page.getByPlaceholder('Search Nodes...')` | `comfyPage.searchBox.input` | Use search box page object |
| `page` (bare reference) | `comfyPage.page` | Access raw page through fixture |
| Bare `test(...)` | `test.describe('Feature', { tag: ['@canvas'] }, () => { test(...) })` | All tests need describe + tags |
| No cleanup | Add `test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })` | Canvas tests need cleanup |
## Fixture API Quick Reference
| Need | Use | Notes |
|------|-----|-------|
| Canvas element | `comfyPage.canvas` | Pre-configured Locator |
| Wait for render | `comfyPage.nextFrame()` | After canvas mutations. NOT needed after `loadWorkflow()` |
| Load workflow | `comfyPage.workflow.loadWorkflow('name')` | Assets in `browser_tests/assets/` |
| Get node by type | `comfyPage.nodeOps.getNodeRefsByType('KSampler')` | Returns array of NodeReference |
| Get node by title | `comfyPage.nodeOps.getNodeRefsByTitle('My Node')` | Returns array of NodeReference |
| Search box | `comfyPage.searchBox` | Has `.input`, `.fillAndSelectFirstNode()` |
| Settings | `comfyPage.settings.setSetting(key, value)` | Persistent — clean up in afterEach |
| Keyboard | `comfyPage.keyboard.press('Delete')` | Focus canvas first |
| Drag & drop | `comfyPage.dragDrop.*` | Use `{ steps: 10 }` for reliability |
| Context menu | `comfyPage.contextMenu.*` | Right-click interactions |
| Toast messages | `comfyPage.toast.*` | Notification assertions |
| Subgraph | `comfyPage.subgraph.*` | Subgraph/group node operations |
| Vue Nodes | `comfyPage.vueNodes.*` | Requires opt-in: `setSetting('Comfy.VueNodes.Enabled', true)` |
| Mouse ops | `comfyPage.page` + `ComfyMouse` | For precise canvas mouse interactions |
| Bottom panel | `comfyPage.bottomPanel.*` | Job queue, logs panel |
| Commands | `comfyPage.command.*` | Command palette interactions |
| Clipboard | `comfyPage.clipboard.*` | Copy/paste operations |
## Canvas Coordinates → Node References
Raw codegen records fragile pixel coordinates. Replace with node references when possible:
```typescript
// ❌ Raw codegen — fragile pixel coordinates
await page.locator('canvas').click({ position: { x: 423, y: 267 } })
// ✅ If clicking a specific node
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
await node.click('title')
// ✅ If double-clicking canvas to open search
await comfyPage.canvas.dblclick({ position: { x: 500, y: 400 } })
await comfyPage.searchBox.fillAndSelectFirstNode('KSampler')
```
**When to keep coordinates**: Canvas background clicks (pan, zoom), empty area clicks to deselect. These are inherently position-based.
## Complete Before/After Example
### Raw codegen output
```typescript
import { test, expect } from '@playwright/test'
test('test', async ({ page }) => {
await page.goto('http://localhost:5173/')
await page.locator('canvas').dblclick({ position: { x: 500, y: 400 } })
await page.getByPlaceholder('Search Nodes...').fill('KSampler')
await page.getByPlaceholder('Search Nodes...').press('Enter')
await page.locator('canvas').click({ position: { x: 600, y: 300 } })
await page.waitForTimeout(1000)
await page.getByRole('button', { name: 'Queue' }).click()
})
```
### Convention-compliant output
```typescript
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test.describe('Queue workflow with KSampler', { tag: ['@canvas'] }, () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.canvasOps.resetView()
})
test('should add KSampler node and queue', async ({ comfyPage }) => {
// Open search and add KSampler
await comfyPage.canvas.dblclick({ position: { x: 500, y: 400 } })
await comfyPage.searchBox.fillAndSelectFirstNode('KSampler')
await comfyPage.nextFrame()
// Queue the workflow
await comfyPage.menu.topbar.runButton.click()
})
})
```
### What changed and why
1. **Imports**: `@playwright/test``../fixtures/ComfyPage` (custom fixtures)
2. **Fixture**: `{ page }``{ comfyPage }` (access all helpers)
3. **goto removed**: Fixture navigates automatically
4. **Search box**: Raw locator → `comfyPage.searchBox.fillAndSelectFirstNode()`
5. **waitForTimeout**: Replaced with `comfyPage.nextFrame()`
6. **Queue button**: Used `comfyPage.menu.topbar.runButton` page object
7. **Structure**: Wrapped in `describe` with `@canvas` tag and `afterEach` cleanup
8. **Test name**: Generic "test" → descriptive name
## Decision Guide
| Question | Answer |
|----------|--------|
| Canvas or DOM interaction? | Canvas: `comfyPage.nodeOps.*`. DOM: `comfyPage.vueNodes.*` (needs opt-in) |
| Need `nextFrame()`? | Yes after canvas mutations. No after `loadWorkflow()`, no after DOM clicks |
| Which tag? | `@canvas` for canvas tests, `@widget` for widget tests, `@screenshot` for visual regression |
| Need cleanup? | Yes for canvas tests (`resetView`), yes if changing settings (`setSetting` back) |
| Keep pixel coords? | Only for empty canvas clicks. Replace with node refs for node interactions |
| Use `page` directly? | Only via `comfyPage.page` for Playwright APIs not wrapped by fixtures |
## Tags Reference
| Tag | When to use |
|-----|-------------|
| `@canvas` | Any test interacting with the canvas |
| `@widget` | Testing widget inputs |
| `@smoke` | Quick essential tests |
| `@screenshot` | Visual regression (Linux CI only) |
| `@mobile` | Mobile viewport (runs on Pixel 5) |
| `@2x` | HiDPI tests (2x scale) |
| `@0.5x` | Low-DPI tests (0.5x scale) |
| `@slow` | Tests taking >10 seconds |
| `@perf` | Performance measurement tests |
## Anti-Patterns
1. **Never use `waitForTimeout`** → use `nextFrame()` or retrying assertions
2. **Never use `page.goto`** → fixture handles navigation
3. **Never import from `@playwright/test`** → use `../fixtures/ComfyPage`
4. **Never use bare CSS selectors** → use test IDs or semantic locators
5. **Never share state between tests** → each test is independent
6. **Never commit local screenshots** → Linux CI generates baselines
## For Deeper Reference
Read fixture code directly — it's the source of truth:
| Purpose | Path |
|---------|------|
| Main fixture | `browser_tests/fixtures/ComfyPage.ts` |
| Helper classes | `browser_tests/fixtures/helpers/` |
| Component objects | `browser_tests/fixtures/components/` |
| Test selectors | `browser_tests/fixtures/selectors.ts` |
| Vue Node helpers | `browser_tests/fixtures/VueNodeHelpers.ts` |
| Existing tests | `browser_tests/tests/` |
| Test assets | `browser_tests/assets/` |

12
.mcp.json Normal file
View File

@@ -0,0 +1,12 @@
{
"mcpServers": {
"playwright-test": {
"command": "pnpm",
"args": [
"exec",
"playwright",
"run-test-mcp-server"
]
}
}
}

View File

@@ -10,6 +10,8 @@
"dist/*",
"packages/registry-types/src/comfyRegistryTypes.ts",
"playwright-report/*",
"scripts/patch-playwright-agents.js",
"tools/test-recorder/*",
"src/extensions/core/*",
"src/scripts/*",
"src/types/generatedManagerTypes.ts",

View File

@@ -49,6 +49,10 @@ This project uses **pnpm**. Always prefer scripts defined in `package.json` (e.g
- `pnpm preview`: Preview the production build locally
- `pnpm test:unit`: Run Vitest unit tests
- `pnpm test:browser:local`: Run Playwright E2E tests (`browser_tests/`)
- `pnpm comfy-test record`: Interactive test recorder (guided setup for non-devs)
- `pnpm comfy-test transform <file>`: Transform raw codegen to conventions
- `pnpm comfy-test check`: Check environment prerequisites
- `pnpm comfy-test list`: List available test workflows
- `pnpm lint` / `pnpm lint:fix`: Lint (ESLint)
- `pnpm format` / `pnpm format:check`: oxfmt
- `pnpm typecheck`: Vue TSC type checking

View File

@@ -41,3 +41,49 @@ browser_tests/
A Playwright test-writing skill exists at `.claude/skills/writing-playwright-tests/SKILL.md`.
The skill documents **meta-level guidance only** (gotchas, anti-patterns, decision guides). It does **not** duplicate fixture APIs - agents should read the fixture code directly in `browser_tests/fixtures/`.
## AI-Assisted Test Creation
Three systems work together for test authoring:
### 1. Interactive Recorder CLI (`comfy-test`)
For QA testers and non-developers. Guides through the full flow:
```bash
pnpm comfy-test record # Interactive 7-step recording flow
pnpm comfy-test transform # Transform raw codegen to conventions
pnpm comfy-test check # Verify environment prerequisites
pnpm comfy-test list # List available workflow assets
```
Source: `tools/test-recorder/`
### 2. Codegen Transform Skill
For AI agents transforming raw Playwright codegen output. See `.claude/skills/codegen-transform/SKILL.md`.
Key transforms:
- `@playwright/test``../fixtures/ComfyPage` imports
- `page` destructure → `comfyPage` fixture
- `page.goto()` → removed (fixture handles navigation)
- `page.locator('canvas')``comfyPage.canvas`
- `waitForTimeout()``comfyPage.nextFrame()`
- Wraps in `test.describe` with tags and `afterEach` cleanup
### 3. Playwright AI Agents
Three agents in `.claude/agents/` are patched with ComfyUI context:
- **planner** — explores the app and creates test plans in `browser_tests/specs/`
- **generator** — converts test plans into executable `.spec.ts` files
- **healer** — debugs and fixes failing tests
To regenerate after Playwright updates: `bash scripts/update-playwright-agents.sh`
### MCP Server
The `.mcp.json` configures `playwright-test` MCP server for agent browser interaction:
```bash
pnpm exec playwright run-test-mcp-server
```

View File

@@ -70,6 +70,30 @@ await comfyPage.setup({ mockReleases: false })
For tests that specifically need to test release functionality, see the example in `tests/releaseNotifications.spec.ts`.
## Recording Tests (For Non-Developers)
If you're a QA tester or non-developer, use the interactive recorder:
```bash
pnpm comfy-test record
```
This guides you through a 7-step flow:
1. **Environment check** — verifies all tools are installed (with install instructions if not)
2. **Project setup** — installs dependencies
3. **Backend check** — ensures ComfyUI is running
4. **Configure** — set test name, tags, and starting workflow
5. **Record** — opens browser with Playwright Inspector for recording
6. **Transform** — paste recorded code, auto-transforms to project conventions
7. **PR creation** — creates a PR via `gh` CLI or gives manual instructions
Other commands:
```bash
pnpm comfy-test check # Just run environment checks
pnpm comfy-test transform <file> # Transform a raw codegen file
pnpm comfy-test list # List available workflow assets
```
## Running Tests
**Always use UI mode for development:**

View File

@@ -0,0 +1,3 @@
# Specs
This is a directory for test plans.

View File

@@ -0,0 +1,26 @@
/**
* Seed test for Playwright AI agents.
*
* This test bootstraps the ComfyUI environment for agent exploration.
* When agents (Planner, Generator, Healer) run, they execute this test
* first to set up the browser state, then use it as a template for
* generated tests.
*
* Usage:
* - Planner: Runs this to explore the app, then generates a test plan
* - Generator: Uses this as an import/fixture template
* - Healer: Runs this to establish baseline state
*/
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test('seed', async ({ comfyPage }) => {
// Load the default workflow — gives agents a realistic starting state
await comfyPage.workflow.loadWorkflow('default')
await comfyPage.nextFrame()
// Verify the app is ready
await expect(comfyPage.canvas).toBeVisible()
})

View File

@@ -26,6 +26,9 @@ const config: KnipConfig = {
},
'packages/registry-types': {
project: ['src/**/*.{js,ts}']
},
'tools/test-recorder': {
project: ['src/**/*.ts']
}
},
ignoreBinaries: ['python3', 'gh'],

View File

@@ -43,6 +43,7 @@
"storybook:desktop": "nx run @comfyorg/desktop-ui:storybook",
"stylelint:fix": "stylelint --cache --fix '{apps,packages,src}/**/*.{css,vue}'",
"stylelint": "stylelint --cache '{apps,packages,src}/**/*.{css,vue}'",
"comfy-test": "tsx tools/test-recorder/src/index.ts",
"test:browser": "pnpm exec nx e2e",
"test:browser:local": "cross-env PLAYWRIGHT_LOCAL=1 pnpm test:browser",
"test:unit": "nx run test",

104
pnpm-lock.yaml generated
View File

@@ -910,6 +910,22 @@ importers:
specifier: 'catalog:'
version: 5.9.3
tools/test-recorder:
dependencies:
'@clack/prompts':
specifier: ^0.9.1
version: 0.9.1
picocolors:
specifier: 'catalog:'
version: 1.1.1
devDependencies:
'@types/node':
specifier: 'catalog:'
version: 24.10.4
typescript:
specifier: 'catalog:'
version: 5.9.3
packages:
'@acemir/cssom@0.9.30':
@@ -1549,6 +1565,12 @@ packages:
'@cacheable/utils@2.3.2':
resolution: {integrity: sha512-8kGE2P+HjfY8FglaOiW+y8qxcaQAfAhVML+i66XJR3YX5FtyDqn6Txctr3K2FrbxLKixRRYYBWMbuGciOhYNDg==}
'@clack/core@0.4.1':
resolution: {integrity: sha512-Pxhij4UXg8KSr7rPek6Zowm+5M22rbd2g1nfojHJkxp5YkFqiZ2+YLEM/XGVIzvGOcM0nqjIFxrpDwWRZYWYjA==}
'@clack/prompts@0.9.1':
resolution: {integrity: sha512-JIpyaboYZeWYlyP0H+OoPPxd6nqueG/CmN6ixBiNFsIDHREevjIf0n0Ohh5gr5C8pEDknzgvz+pIJ8dMhzWIeg==}
'@comfyorg/comfyui-electron-types@0.6.2':
resolution: {integrity: sha512-r3By5Wbizq8jagUrhtcym79HYUTinsvoBnYkFFWbUmrURBWIaC0HduFVkRkI1PNdI76piW+JSOJJnw00YCVXeg==}
@@ -2457,21 +2479,25 @@ packages:
resolution: {integrity: sha512-D+tPXB0tkSuDPsuXvyQIsF3f3PBWfAwIe9FkBWtVoDVYqE+jbz+tVGsjQMNWGafLE4sC8ZQdjhsxyT8I53Anbw==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@nx/nx-linux-arm64-musl@22.5.2':
resolution: {integrity: sha512-UbO527qqa8KLBi13uXto5SmxcZv1Smer7sPexJonshDlmrJsyvx5m8nm6tcSv04W5yQEL90vPlTux8dNvEDWrw==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@nx/nx-linux-x64-gnu@22.5.2':
resolution: {integrity: sha512-wR6596Vr/Z+blUAmjLHG2TCQMs4O1oi9JXK1J/PoPeO9UqdHwStCJBAd61zDFSUYJe0x+dkeRQu96fE5BW8Kcg==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@nx/nx-linux-x64-musl@22.5.2':
resolution: {integrity: sha512-MBXOw4AH4FWl4orwVykj/e75awTNDePogrl3pXNX9NcQLdj6JzS4e2jaALQeRBQLxQzeFvFQV/W4PBzoPV6/NA==}
cpu: [x64]
os: [linux]
libc: [musl]
'@nx/nx-win32-arm64-msvc@22.5.2':
resolution: {integrity: sha512-SaWSZkRH5uV8vP2lj6RRv+kw2IzaIDXkutReOXpooshIWZl9KjrQELNTCZTYyhLDsMlcyhSvLFlTiA4NkZ8udw==}
@@ -2637,41 +2663,49 @@ packages:
resolution: {integrity: sha512-SVjjjtMW66Mza76PBGJLqB0KKyFTBnxmtDXLJPbL6ZPGSctcXVmujz7/WAc0rb9m2oV0cHQTtVjnq6orQnI/jg==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@oxc-resolver/binding-linux-arm64-musl@11.15.0':
resolution: {integrity: sha512-JDv2/AycPF2qgzEiDeMJCcSzKNDm3KxNg0KKWipoKEMDFqfM7LxNwwSVyAOGmrYlE4l3dg290hOMsr9xG7jv9g==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@oxc-resolver/binding-linux-ppc64-gnu@11.15.0':
resolution: {integrity: sha512-zbu9FhvBLW4KJxo7ElFvZWbSt4vP685Qc/Gyk/Ns3g2gR9qh2qWXouH8PWySy+Ko/qJ42+HJCLg+ZNcxikERfg==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@oxc-resolver/binding-linux-riscv64-gnu@11.15.0':
resolution: {integrity: sha512-Kfleehe6B09C2qCnyIU01xLFqFXCHI4ylzkicfX/89j+gNHh9xyNdpEvit88Kq6i5tTGdavVnM6DQfOE2qNtlg==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@oxc-resolver/binding-linux-riscv64-musl@11.15.0':
resolution: {integrity: sha512-J7LPiEt27Tpm8P+qURDwNc8q45+n+mWgyys4/V6r5A8v5gDentHRGUx3iVk5NxdKhgoGulrzQocPTZVosq25Eg==}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@oxc-resolver/binding-linux-s390x-gnu@11.15.0':
resolution: {integrity: sha512-+8/d2tAScPjVJNyqa7GPGnqleTB/XW9dZJQ2D/oIM3wpH3TG+DaFEXBbk4QFJ9K9AUGBhvQvWU2mQyhK/yYn3Q==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@oxc-resolver/binding-linux-x64-gnu@11.15.0':
resolution: {integrity: sha512-xtvSzH7Nr5MCZI2FKImmOdTl9kzuQ51RPyLh451tvD2qnkg3BaqI9Ox78bTk57YJhlXPuxWSOL5aZhKAc9J6qg==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@oxc-resolver/binding-linux-x64-musl@11.15.0':
resolution: {integrity: sha512-14YL1zuXj06+/tqsuUZuzL0T425WA/I4nSVN1kBXeC5WHxem6lQ+2HGvG+crjeJEqHgZUT62YIgj88W+8E7eyg==}
cpu: [x64]
os: [linux]
libc: [musl]
'@oxc-resolver/binding-openharmony-arm64@11.15.0':
resolution: {integrity: sha512-/7Qli+1Wk93coxnrQaU8ySlICYN8HsgyIrzqjgIkQEpI//9eUeaeIHZptNl2fMvBGeXa7k2QgLbRNaBRgpnvMw==}
@@ -2745,48 +2779,56 @@ packages:
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-arm64-musl@0.34.0':
resolution: {integrity: sha512-H+F8+71gHQoGTFPPJ6z4dD0Fzfzi0UP8Zx94h5kUmIFThLvMq5K1Y/bUUubiXwwHfwb5C3MPjUpYijiy0rj51Q==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [musl]
'@oxfmt/binding-linux-ppc64-gnu@0.34.0':
resolution: {integrity: sha512-dIGnzTNhCXqQD5pzBwduLg8pClm+t8R53qaE9i5h8iua1iaFAJyLffh4847CNZSlASb7gn1Ofuv7KoG/EpoGZg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-riscv64-gnu@0.34.0':
resolution: {integrity: sha512-FGQ2GTTooilDte/ogwWwkHuuL3lGtcE3uKM2EcC7kOXNWdUfMY6Jx3JCodNVVbFoybv4A+HuCj8WJji2uu1Ceg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-riscv64-musl@0.34.0':
resolution: {integrity: sha512-2dGbGneJ7ptOIVKMwEIHdCkdZEomh74X3ggo4hCzEXL/rl9HwfsZDR15MkqfQqAs6nVXMvtGIOMxjDYa5lwKaA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@oxfmt/binding-linux-s390x-gnu@0.34.0':
resolution: {integrity: sha512-cCtGgmrTrxq3OeSG0UAO+w6yLZTMeOF4XM9SAkNrRUxYhRQELSDQ/iNPCLyHhYNi38uHJQbS5RQweLUDpI4ajA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-x64-gnu@0.34.0':
resolution: {integrity: sha512-7AvMzmeX+k7GdgitXp99GQoIV/QZIpAS7rwxQvC/T541yWC45nwvk4mpnU8N+V6dE5SPEObnqfhCjO80s7qIsg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [glibc]
'@oxfmt/binding-linux-x64-musl@0.34.0':
resolution: {integrity: sha512-uNiglhcmivJo1oDMh3hoN/Z0WsbEXOpRXZdQ3W/IkOpyV8WF308jFjSC1ZxajdcNRXWej0zgge9QXba58Owt+g==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [musl]
'@oxfmt/binding-openharmony-arm64@0.34.0':
resolution: {integrity: sha512-5eFsTjCyji25j6zznzlMc+wQAZJoL9oWy576xhqd2efv+N4g1swIzuSDcb1dz4gpcVC6veWe9pAwD7HnrGjLwg==}
@@ -2889,48 +2931,56 @@ packages:
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-arm64-musl@1.49.0':
resolution: {integrity: sha512-xeqkMOARgGBlEg9BQuPDf6ZW711X6BT5qjDyeM5XNowCJeTSdmMhpePJjTEiVbbr3t21sIlK8RE6X5bc04nWyQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [musl]
'@oxlint/binding-linux-ppc64-gnu@1.49.0':
resolution: {integrity: sha512-uvcqRO6PnlJGbL7TeePhTK5+7/JXbxGbN+C6FVmfICDeeRomgQqrfVjf0lUrVpUU8ii8TSkIbNdft3M+oNlOsQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-riscv64-gnu@1.49.0':
resolution: {integrity: sha512-Dw1HkdXAwHNH+ZDserHP2RzXQmhHtpsYYI0hf8fuGAVCIVwvS6w1+InLxpPMY25P8ASRNiFN3hADtoh6lI+4lg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-riscv64-musl@1.49.0':
resolution: {integrity: sha512-EPlMYaA05tJ9km/0dI9K57iuMq3Tw+nHst7TNIegAJZrBPtsOtYaMFZEaWj02HA8FI5QvSnRHMt+CI+RIhXJBQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@oxlint/binding-linux-s390x-gnu@1.49.0':
resolution: {integrity: sha512-yZiQL9qEwse34aMbnMb5VqiAWfDY+fLFuoJbHOuzB1OaJZbN1MRF9Nk+W89PIpGr5DNPDipwjZb8+Q7wOywoUQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-x64-gnu@1.49.0':
resolution: {integrity: sha512-CcCDwMMXSchNkhdgvhVn3DLZ4EnBXAD8o8+gRzahg+IdSt/72y19xBgShJgadIRF0TsRcV/MhDUMwL5N/W54aQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [glibc]
'@oxlint/binding-linux-x64-musl@1.49.0':
resolution: {integrity: sha512-u3HfKV8BV6t6UCCbN0RRiyqcymhrnpunVmLFI8sEa5S/EBu+p/0bJ3D7LZ2KT6PsBbrB71SWq4DeFrskOVgIZg==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [musl]
'@oxlint/binding-openharmony-arm64@1.49.0':
resolution: {integrity: sha512-dRDpH9fw+oeUMpM4br0taYCFpW6jQtOuEIec89rOgDA1YhqwmeRcx0XYeCv7U48p57qJ1XZHeMGM9LdItIjfzA==}
@@ -3099,24 +3149,28 @@ packages:
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@rolldown/binding-linux-arm64-musl@1.0.0-rc.3':
resolution: {integrity: sha512-Z03/wrqau9Bicfgb3Dbs6SYTHliELk2PM2LpG2nFd+cGupTMF5kanLEcj2vuuJLLhptNyS61rtk7SOZ+lPsTUA==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [arm64]
os: [linux]
libc: [musl]
'@rolldown/binding-linux-x64-gnu@1.0.0-rc.3':
resolution: {integrity: sha512-iSXXZsQp08CSilff/DCTFZHSVEpEwdicV3W8idHyrByrcsRDVh9sGC3sev6d8BygSGj3vt8GvUKBPCoyMA4tgQ==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [glibc]
'@rolldown/binding-linux-x64-musl@1.0.0-rc.3':
resolution: {integrity: sha512-qaj+MFudtdCv9xZo9znFvkgoajLdc+vwf0Kz5N44g+LU5XMe+IsACgn3UG7uTRlCCvhMAGXm1XlpEA5bZBrOcw==}
engines: {node: ^20.19.0 || >=22.12.0}
cpu: [x64]
os: [linux]
libc: [musl]
'@rolldown/binding-openharmony-arm64@1.0.0-rc.3':
resolution: {integrity: sha512-U662UnMETyjT65gFmG9ma+XziENrs7BBnENi/27swZPYagubfHRirXHG2oMl+pEax2WvO7Kb9gHZmMakpYqBHQ==}
@@ -3194,56 +3248,67 @@ packages:
resolution: {integrity: sha512-dV3T9MyAf0w8zPVLVBptVlzaXxka6xg1f16VAQmjg+4KMSTWDvhimI/Y6mp8oHwNrmnmVl9XxJ/w/mO4uIQONA==}
cpu: [arm]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm-musleabihf@4.53.5':
resolution: {integrity: sha512-wIGYC1x/hyjP+KAu9+ewDI+fi5XSNiUi9Bvg6KGAh2TsNMA3tSEs+Sh6jJ/r4BV/bx/CyWu2ue9kDnIdRyafcQ==}
cpu: [arm]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-arm64-gnu@4.53.5':
resolution: {integrity: sha512-Y+qVA0D9d0y2FRNiG9oM3Hut/DgODZbU9I8pLLPwAsU0tUKZ49cyV1tzmB/qRbSzGvY8lpgGkJuMyuhH7Ma+Vg==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm64-musl@4.53.5':
resolution: {integrity: sha512-juaC4bEgJsyFVfqhtGLz8mbopaWD+WeSOYr5E16y+1of6KQjc0BpwZLuxkClqY1i8sco+MdyoXPNiCkQou09+g==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-loong64-gnu@4.53.5':
resolution: {integrity: sha512-rIEC0hZ17A42iXtHX+EPJVL/CakHo+tT7W0pbzdAGuWOt2jxDFh7A/lRhsNHBcqL4T36+UiAgwO8pbmn3dE8wA==}
cpu: [loong64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-ppc64-gnu@4.53.5':
resolution: {integrity: sha512-T7l409NhUE552RcAOcmJHj3xyZ2h7vMWzcwQI0hvn5tqHh3oSoclf9WgTl+0QqffWFG8MEVZZP1/OBglKZx52Q==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-riscv64-gnu@4.53.5':
resolution: {integrity: sha512-7OK5/GhxbnrMcxIFoYfhV/TkknarkYC1hqUw1wU2xUN3TVRLNT5FmBv4KkheSG2xZ6IEbRAhTooTV2+R5Tk0lQ==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-riscv64-musl@4.53.5':
resolution: {integrity: sha512-GwuDBE/PsXaTa76lO5eLJTyr2k8QkPipAyOrs4V/KJufHCZBJ495VCGJol35grx9xryk4V+2zd3Ri+3v7NPh+w==}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-s390x-gnu@4.53.5':
resolution: {integrity: sha512-IAE1Ziyr1qNfnmiQLHBURAD+eh/zH1pIeJjeShleII7Vj8kyEm2PF77o+lf3WTHDpNJcu4IXJxNO0Zluro8bOw==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-gnu@4.53.5':
resolution: {integrity: sha512-Pg6E+oP7GvZ4XwgRJBuSXZjcqpIW3yCBhK4BcsANvb47qMvAbCjR6E+1a/U2WXz1JJxp9/4Dno3/iSJLcm5auw==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-musl@4.53.5':
resolution: {integrity: sha512-txGtluxDKTxaMDzUduGP0wdfng24y1rygUMnmlUJ88fzCCULCLn7oE5kb2+tRB+MWq1QDZT6ObT5RrR8HFRKqg==}
cpu: [x64]
os: [linux]
libc: [musl]
'@rollup/rollup-openharmony-arm64@4.53.5':
resolution: {integrity: sha512-3DFiLPnTxiOQV993fMc+KO8zXHTcIjgaInrqlG8zDp1TlhYl6WgrOHuJkJQ6M8zHEcntSJsUp1XFZSY8C1DYbg==}
@@ -3519,24 +3584,28 @@ packages:
engines: {node: '>= 20'}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@tailwindcss/oxide-linux-arm64-musl@4.2.0':
resolution: {integrity: sha512-XKcSStleEVnbH6W/9DHzZv1YhjE4eSS6zOu2eRtYAIh7aV4o3vIBs+t/B15xlqoxt6ef/0uiqJVB6hkHjWD/0A==}
engines: {node: '>= 20'}
cpu: [arm64]
os: [linux]
libc: [musl]
'@tailwindcss/oxide-linux-x64-gnu@4.2.0':
resolution: {integrity: sha512-/hlXCBqn9K6fi7eAM0RsobHwJYa5V/xzWspVTzxnX+Ft9v6n+30Pz8+RxCn7sQL/vRHHLS30iQPrHQunu6/vJA==}
engines: {node: '>= 20'}
cpu: [x64]
os: [linux]
libc: [glibc]
'@tailwindcss/oxide-linux-x64-musl@4.2.0':
resolution: {integrity: sha512-lKUaygq4G7sWkhQbfdRRBkaq4LY39IriqBQ+Gk6l5nKq6Ay2M2ZZb1tlIyRNgZKS8cbErTwuYSor0IIULC0SHw==}
engines: {node: '>= 20'}
cpu: [x64]
os: [linux]
libc: [musl]
'@tailwindcss/oxide-wasm32-wasi@4.2.0':
resolution: {integrity: sha512-xuDjhAsFdUuFP5W9Ze4k/o4AskUtI8bcAGU4puTYprr89QaYFmhYOPfP+d1pH+k9ets6RoE23BXZM1X1jJqoyw==}
@@ -4012,41 +4081,49 @@ packages:
resolution: {integrity: sha512-34gw7PjDGB9JgePJEmhEqBhWvCiiWCuXsL9hYphDF7crW7UgI05gyBAi6MF58uGcMOiOqSJ2ybEeCvHcq0BCmQ==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@unrs/resolver-binding-linux-arm64-musl@1.11.1':
resolution: {integrity: sha512-RyMIx6Uf53hhOtJDIamSbTskA99sPHS96wxVE/bJtePJJtpdKGXO1wY90oRdXuYOGOTuqjT8ACccMc4K6QmT3w==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@unrs/resolver-binding-linux-ppc64-gnu@1.11.1':
resolution: {integrity: sha512-D8Vae74A4/a+mZH0FbOkFJL9DSK2R6TFPC9M+jCWYia/q2einCubX10pecpDiTmkJVUH+y8K3BZClycD8nCShA==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@unrs/resolver-binding-linux-riscv64-gnu@1.11.1':
resolution: {integrity: sha512-frxL4OrzOWVVsOc96+V3aqTIQl1O2TjgExV4EKgRY09AJ9leZpEg8Ak9phadbuX0BA4k8U5qtvMSQQGGmaJqcQ==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@unrs/resolver-binding-linux-riscv64-musl@1.11.1':
resolution: {integrity: sha512-mJ5vuDaIZ+l/acv01sHoXfpnyrNKOk/3aDoEdLO/Xtn9HuZlDD6jKxHlkN8ZhWyLJsRBxfv9GYM2utQ1SChKew==}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@unrs/resolver-binding-linux-s390x-gnu@1.11.1':
resolution: {integrity: sha512-kELo8ebBVtb9sA7rMe1Cph4QHreByhaZ2QEADd9NzIQsYNQpt9UkM9iqr2lhGr5afh885d/cB5QeTXSbZHTYPg==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@unrs/resolver-binding-linux-x64-gnu@1.11.1':
resolution: {integrity: sha512-C3ZAHugKgovV5YvAMsxhq0gtXuwESUKc5MhEtjBpLoHPLYM+iuwSj3lflFwK3DPm68660rZ7G8BMcwSro7hD5w==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@unrs/resolver-binding-linux-x64-musl@1.11.1':
resolution: {integrity: sha512-rV0YSoyhK2nZ4vEswT/QwqzqQXw5I6CjoaYMOX0TqBlWhojUf8P94mvI7nuJTeaCkkds3QE4+zS8Ko+GdXuZtA==}
cpu: [x64]
os: [linux]
libc: [musl]
'@unrs/resolver-binding-wasm32-wasi@1.11.1':
resolution: {integrity: sha512-5u4RkfxJm+Ng7IWgkzi3qrFOvLvQYnPBmjmZQ8+szTK/b31fQCnleNl1GgEt7nIsZRIf5PLhPwT0WM+q45x/UQ==}
@@ -6426,24 +6503,28 @@ packages:
engines: {node: '>= 12.0.0'}
cpu: [arm64]
os: [linux]
libc: [glibc]
lightningcss-linux-arm64-musl@1.31.1:
resolution: {integrity: sha512-mVZ7Pg2zIbe3XlNbZJdjs86YViQFoJSpc41CbVmKBPiGmC4YrfeOyz65ms2qpAobVd7WQsbW4PdsSJEMymyIMg==}
engines: {node: '>= 12.0.0'}
cpu: [arm64]
os: [linux]
libc: [musl]
lightningcss-linux-x64-gnu@1.31.1:
resolution: {integrity: sha512-xGlFWRMl+0KvUhgySdIaReQdB4FNudfUTARn7q0hh/V67PVGCs3ADFjw+6++kG1RNd0zdGRlEKa+T13/tQjPMA==}
engines: {node: '>= 12.0.0'}
cpu: [x64]
os: [linux]
libc: [glibc]
lightningcss-linux-x64-musl@1.31.1:
resolution: {integrity: sha512-eowF8PrKHw9LpoZii5tdZwnBcYDxRw2rRCyvAXLi34iyeYfqCQNA9rmUM0ce62NlPhCvof1+9ivRaTY6pSKDaA==}
engines: {node: '>= 12.0.0'}
cpu: [x64]
os: [linux]
libc: [musl]
lightningcss-win32-arm64-msvc@1.31.1:
resolution: {integrity: sha512-aJReEbSEQzx1uBlQizAOBSjcmr9dCdL3XuC/6HLXAxmtErsj2ICo5yYggg1qOODQMtnjNQv2UHb9NpOuFtYe4w==}
@@ -7669,6 +7750,9 @@ packages:
resolution: {integrity: sha512-2wcC/oGxHis/BoHkkPwldgiPSYcpZK3JU28WoMVv55yHJgcZ8rlXvuG9iZggz+sU1d4bRgIGASwyWqjxu3FM0g==}
engines: {node: '>=18'}
sisteransi@1.0.5:
resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==}
slash@3.0.0:
resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==}
engines: {node: '>=8'}
@@ -8402,6 +8486,9 @@ packages:
vue-component-type-helpers@3.2.5:
resolution: {integrity: sha512-tkvNr+bU8+xD/onAThIe7CHFvOJ/BO6XCOrxMzeytJq40nTfpGDJuVjyCM8ccGZKfAbGk2YfuZyDMXM56qheZQ==}
vue-component-type-helpers@3.2.6:
resolution: {integrity: sha512-O02tnvIfOQVmnvoWwuSydwRoHjZVt8UEBR+2p4rT35p8GAy5VTlWP8o5qXfJR/GWCN0nVZoYWsVUvx2jwgdBmQ==}
vue-demi@0.14.10:
resolution: {integrity: sha512-nMZBOwuzabUO0nLgIcc6rycZEebF6eeUfaiQx9+WSk8e29IbLvPU9feI6tqW4kTo3hvoYAJkMh8n8D0fuISphg==}
engines: {node: '>=12'}
@@ -9585,6 +9672,17 @@ snapshots:
hashery: 1.3.0
keyv: 5.5.5
'@clack/core@0.4.1':
dependencies:
picocolors: 1.1.1
sisteransi: 1.0.5
'@clack/prompts@0.9.1':
dependencies:
'@clack/core': 0.4.1
picocolors: 1.1.1
sisteransi: 1.0.5
'@comfyorg/comfyui-electron-types@0.6.2': {}
'@csstools/color-helpers@5.1.0': {}
@@ -11485,7 +11583,7 @@ snapshots:
storybook: 10.2.10(@testing-library/dom@10.4.1)(prettier@3.7.4)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
type-fest: 2.19.0
vue: 3.5.13(typescript@5.9.3)
vue-component-type-helpers: 3.2.5
vue-component-type-helpers: 3.2.6
'@swc/helpers@0.5.17':
dependencies:
@@ -16547,6 +16645,8 @@ snapshots:
mrmime: 2.0.1
totalist: 3.0.1
sisteransi@1.0.5: {}
slash@3.0.0: {}
slice-ansi@4.0.0:
@@ -17500,6 +17600,8 @@ snapshots:
vue-component-type-helpers@3.2.5: {}
vue-component-type-helpers@3.2.6: {}
vue-demi@0.14.10(vue@3.5.13(typescript@5.9.3)):
dependencies:
vue: 3.5.13(typescript@5.9.3)

View File

@@ -1,6 +1,7 @@
packages:
- apps/**
- packages/**
- tools/**
catalog:
'@alloc/quick-lru': ^5.2.0

View File

@@ -0,0 +1,154 @@
#!/usr/bin/env node
import { readFileSync, writeFileSync } from 'node:fs'
import { join } from 'node:path'
const agentsDir = join(process.cwd(), '.claude', 'agents')
const patches = {
'playwright-test-planner.md': `
## ComfyUI Project Context
### Application Overview
ComfyUI is a **canvas-based node graph editor** for AI image generation. It is a complex SPA with:
- A **LiteGraph canvas** where users create workflows by connecting nodes
- A **Vue 3 sidebar** with node library, workflows panel, and settings
- A **topbar** with queue/run buttons and workspace controls
- A **search box** for finding and adding nodes (opens on double-click)
- WebSocket-based real-time communication with a Python backend
### Exploration Tips
- Start by loading a workflow: the app is most useful with nodes on the canvas
- Key UI areas to explore: canvas interactions, sidebar panels, topbar buttons, search box, context menus, settings dialog
- Double-click the canvas to open the node search box
- Right-click nodes/canvas for context menus
- The bottom panel shows job queue and execution logs
### Test Environment
- The seed test uses \`comfyPageFixture\` which provides a \`comfyPage\` object with extensive helpers
- Workflows (JSON files) are loaded via \`comfyPage.workflow.loadWorkflow('name')\`
- Available workflow assets are in \`browser_tests/assets/\`
- The backend MUST be running with \`--multi-user\` flag
- A Vite dev server runs on \`:5173\`
### When Creating Test Plans
- Reference specific workflow assets when a scenario needs a starting state
- Note that canvas interactions use pixel coordinates — these may vary across environments
- Distinguish between "canvas tests" (LiteGraph) and "UI tests" (Vue components)
- Include tags in your plans: \`@canvas\`, \`@widget\`, \`@sidebar\`, \`@smoke\`, \`@screenshot\`
- Reference \`browser_tests/fixtures/ComfyPage.ts\` for available test helpers`,
'playwright-test-generator.md': `
## ComfyUI Project Context
### Required Import Pattern
Generated tests MUST use ComfyUI fixtures, not generic \`@playwright/test\`:
\`\`\`typescript
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
\`\`\`
### Fixture Object
Tests receive \`comfyPage\` (not \`page\`) as their fixture:
\`\`\`typescript
test('my test', async ({ comfyPage }) => {
// Access raw page via comfyPage.page if needed
})
\`\`\`
### Key APIs
| Need | Use | Notes |
|------|-----|-------|
| Canvas element | \`comfyPage.canvas\` | Pre-configured Locator |
| Wait for render | \`comfyPage.nextFrame()\` | After canvas mutations |
| Load workflow | \`comfyPage.workflow.loadWorkflow('name')\` | Assets in \`browser_tests/assets/\` |
| Get node by type | \`comfyPage.nodeOps.getNodeRefsByType('KSampler')\` | Returns NodeReference[] |
| Search box | \`comfyPage.searchBox.fillAndSelectFirstNode('name')\` | Opens on canvas dblclick |
| Settings | \`comfyPage.settings.setSetting(key, value)\` | Clean up in afterEach |
| Keyboard | \`comfyPage.keyboard.press('Delete')\` | Focus canvas first |
| Context menu | \`comfyPage.contextMenu\` | Right-click interactions |
### Mandatory Test Structure
Every generated test must:
1. Be wrapped in \`test.describe('Name', { tag: ['@canvas'] }, () => { ... })\`
2. Include \`test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })\`
3. Use descriptive test names (not "test" or "test1")
### Anti-Patterns — NEVER Use
- ❌ \`page.goto()\` — fixture handles navigation
- ❌ \`page.waitForTimeout()\` — use \`comfyPage.nextFrame()\` or retrying assertions
- ❌ \`import from '@playwright/test'\` — use \`from '../fixtures/ComfyPage'\`
- ❌ Bare \`page.\` references — use \`comfyPage.page.\` if you need raw page access
### Reference
Read the fixture code for full API surface:
- \`browser_tests/fixtures/ComfyPage.ts\` — main fixture
- \`browser_tests/fixtures/helpers/\` — helper classes
- \`browser_tests/fixtures/components/\` — page object components
- See also: \`.claude/skills/codegen-transform/SKILL.md\` for transform rules`,
'playwright-test-healer.md': `
## ComfyUI Project Context
### Custom Fixtures
Tests in this project use \`comfyPage\` fixture, not bare \`page\`. When healing:
- Replace any \`page.\` references with \`comfyPage.page.\` if adding new code
- Use \`comfyPage.nextFrame()\` instead of adding \`waitForTimeout()\`
- Use fixture helpers (\`comfyPage.nodeOps\`, \`comfyPage.canvas\`, etc.) over raw locators
### Common Failure Causes in ComfyUI Tests
1. **Missing \`nextFrame()\`**: Canvas operations need \`await comfyPage.nextFrame()\` after mutations. This is the #1 cause of "works locally, fails in CI" issues.
2. **Canvas focus required**: Keyboard shortcuts won't work unless \`await comfyPage.canvas.click()\` is called first.
3. **Node position drift**: Pixel coordinates can shift between environments. When possible, replace with node references:
\`\`\`typescript
// Instead of: canvas.click({ position: { x: 423, y: 267 } })
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
await node.click('title')
\`\`\`
4. **Settings pollution**: Settings persist across tests on the backend. Always reset changed settings in \`afterEach\`.
5. **Drag animation timing**: Use \`{ steps: 10 }\` option for drag operations, not \`{ steps: 1 }\`.
### Healing Safety Rules
- ❌ NEVER add \`waitForTimeout()\` — always use retrying assertions or \`nextFrame()\`
- ❌ NEVER "fix" a test by weakening assertions (e.g., removing an assertion that fails)
- ❌ NEVER modify the application code — only modify test code
- ⚠️ If a test fails because expected UI elements are missing, the app may have a regression — mark as \`test.fixme()\` with explanation, don't "heal" the assertion away
- ⚠️ If a test fails only in CI but passes locally, likely missing \`nextFrame()\` — don't mask with timeouts
### Reference
- \`browser_tests/fixtures/ComfyPage.ts\` — full fixture API
- \`browser_tests/fixtures/helpers/\` — available helper classes
- \`.claude/skills/writing-playwright-tests/SKILL.md\` — testing conventions
- \`.claude/skills/codegen-transform/SKILL.md\` — transform rules`
}
const CONTEXT_HEADING = '## ComfyUI Project Context'
for (const [filename, patch] of Object.entries(patches)) {
const filePath = join(agentsDir, filename)
let content = readFileSync(filePath, 'utf-8')
// Strip existing ComfyUI context section (heading to EOF)
const idx = content.indexOf(CONTEXT_HEADING)
if (idx !== -1) {
// Trim trailing whitespace before the heading too
content = content.substring(0, idx).trimEnd()
console.log(` ♻️ ${filename}: stripped existing ComfyUI context`)
}
content += patch
writeFileSync(filePath, content, 'utf-8')
console.log(`${filename}: patched`)
}

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
# Navigate to repo root (script location relative)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
echo "🎭 Updating Playwright agent definitions..."
echo ""
# Step 1: Regenerate base agent files
echo "Step 1: Running init-agents..."
cd "$REPO_ROOT"
pnpm exec playwright init-agents --loop=claude
echo ""
echo "Step 2: Applying ComfyUI patches..."
node "$SCRIPT_DIR/patch-playwright-agents.js"
echo ""
echo "✅ Agent files updated and patched."
echo ""
echo "Files modified:"
echo " .claude/agents/playwright-test-planner.md"
echo " .claude/agents/playwright-test-generator.md"
echo " .claude/agents/playwright-test-healer.md"
echo ""
echo "Review changes with: git diff .claude/agents/"

View File

@@ -0,0 +1,26 @@
# @comfyorg/test-recorder
Interactive CLI for recording and transforming Playwright browser tests for ComfyUI.
## Usage
From the repo root:
```bash
pnpm comfy-test record # Record a new test
pnpm comfy-test transform # Transform raw codegen to conventions
pnpm comfy-test check # Check environment prerequisites
pnpm comfy-test list # List available workflows
```
## For QA Testers
See the [Browser Tests README](../../browser_tests/README.md) for full setup instructions.
## Development
```bash
cd tools/test-recorder
pnpm build # Compile TypeScript
pnpm dev # Watch mode
```

View File

@@ -0,0 +1,24 @@
{
"name": "@comfyorg/test-recorder",
"version": "0.1.0",
"private": true,
"description": "Interactive CLI for recording and transforming Playwright browser tests",
"type": "module",
"bin": {
"comfy-test": "./dist/index.js"
},
"scripts": {
"build": "tsc",
"dev": "tsc --watch",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@clack/prompts": "^0.9.1",
"picocolors": "catalog:"
},
"devDependencies": {
"@types/node": "catalog:",
"typescript": "catalog:"
},
"packageManager": "pnpm@10.17.1"
}

View File

@@ -0,0 +1,33 @@
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkBackend(port = 8188): Promise<CheckResult> {
const url = `http://localhost:${port}`
try {
const res = await fetch(`${url}/system_stats`, {
signal: AbortSignal.timeout(3000)
})
if (res.ok) {
pass('ComfyUI backend', url)
return { name: 'ComfyUI backend', ok: true, version: url }
}
throw new Error(`Status ${res.status}`)
} catch {
fail('ComfyUI backend', `not running on :${port}`)
const instructions = [
'ComfyUI backend must be running for browser tests.',
'',
'In a separate terminal, navigate to your ComfyUI folder and run:',
'',
' python main.py --multi-user',
'',
'⚠️ The --multi-user flag is REQUIRED for parallel test support.'
]
info(instructions)
return {
name: 'ComfyUI backend',
ok: false,
installInstructions: instructions
}
}
}

View File

@@ -0,0 +1,25 @@
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkDevServer(port = 5173): Promise<CheckResult> {
const url = `http://localhost:${port}`
try {
const res = await fetch(url, { signal: AbortSignal.timeout(3000) })
if (res.ok || res.status === 304) {
pass('Dev server', url)
return { name: 'Dev server', ok: true, version: url }
}
throw new Error(`Status ${res.status}`)
} catch {
fail('Dev server', `not running on :${port}`)
const instructions = [
'Start the Vite dev server in another terminal:',
'',
' pnpm dev',
'',
`Then wait for it to show "Local: http://localhost:${port}"`
]
info(instructions)
return { name: 'Dev server', ok: false, installInstructions: instructions }
}
}

View File

@@ -0,0 +1,55 @@
import { execSync } from 'node:child_process'
import { detectPlatform } from './platform'
import { pass, warn, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkGh(): Promise<CheckResult> {
try {
const version = execSync('gh --version', { encoding: 'utf-8' })
.split('\n')[0]
.trim()
// Check if authenticated
try {
execSync('gh auth status', { encoding: 'utf-8', stdio: 'pipe' })
pass('GitHub CLI (gh)', version.replace('gh version ', ''))
return { name: 'GitHub CLI', ok: true, optional: true, version }
} catch {
warn('GitHub CLI (gh)', 'installed but not authenticated')
info([
'Run `gh auth login` to authenticate.',
"Without auth, we'll help you create PRs manually."
])
return { name: 'GitHub CLI', ok: false, optional: true }
}
} catch {
warn('GitHub CLI (gh)', 'not installed (optional)')
const platform = detectPlatform()
const instructions =
platform === 'macos'
? [
"gh CLI lets us auto-create PRs. Without it, we'll help",
'you create the PR manually on github.com.',
'',
'To install (optional):',
' brew install gh',
' gh auth login'
]
: platform === 'windows'
? [
'gh CLI is optional. Install from: https://cli.github.com/',
"Without it, we'll help you create PRs via github.com."
]
: [
'gh CLI is optional. Install:',
' sudo apt install gh # Debian/Ubuntu',
' gh auth login'
]
info(instructions)
return {
name: 'GitHub CLI',
ok: false,
optional: true,
installInstructions: instructions
}
}
}

View File

@@ -0,0 +1,37 @@
import { execSync } from 'node:child_process'
import { detectPlatform } from './platform'
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkGit(): Promise<CheckResult> {
try {
const version = execSync('git --version', { encoding: 'utf-8' }).trim()
const v = version.replace('git version ', '')
pass('Git', v)
return { name: 'Git', ok: true, version: v }
} catch {
fail('Git', 'not installed')
const platform = detectPlatform()
const instructions =
platform === 'macos'
? [
'Git is included with Xcode CLI Tools. Install them first:',
'',
' xcode-select --install'
]
: platform === 'windows'
? [
'Download Git from: https://git-scm.com/download/win',
'Run the installer with default settings.'
]
: [
'Install git using your package manager:',
'',
' sudo apt install git # Debian/Ubuntu',
' sudo dnf install git # Fedora',
' sudo pacman -S git # Arch'
]
info(instructions)
return { name: 'Git', ok: false, installInstructions: instructions }
}
}

View File

@@ -0,0 +1,59 @@
import { execSync } from 'node:child_process'
import { detectPlatform } from './platform'
import { pass, fail, warn, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkNode(): Promise<CheckResult> {
try {
const version = execSync('node --version', { encoding: 'utf-8' }).trim()
const major = parseInt(version.replace('v', '').split('.')[0])
if (major < 20) {
warn('Node.js', `${version} (need v20+)`)
const instructions = [
`Node.js ${version} is too old. You need v20 or later.`,
'',
'Update via nvm:',
' nvm install 20',
' nvm use 20'
]
info(instructions)
return {
name: 'Node.js',
ok: false,
version,
installInstructions: instructions
}
}
pass('Node.js', version)
return { name: 'Node.js', ok: true, version }
} catch {
fail('Node.js', 'not installed')
const platform = detectPlatform()
const instructions =
platform === 'macos'
? [
'Install Node.js via nvm (recommended):',
'',
' curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash',
'',
'Close and reopen Terminal, then run:',
'',
' nvm install 20'
]
: platform === 'windows'
? [
'Download Node.js from: https://nodejs.org/',
'Choose the LTS version (v20+).',
'Run the installer with default settings.'
]
: [
'Install Node.js via nvm:',
'',
' curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash',
' source ~/.bashrc',
' nvm install 20'
]
info(instructions)
return { name: 'Node.js', ok: false, installInstructions: instructions }
}
}

View File

@@ -0,0 +1,24 @@
import os from 'node:os'
import { pass } from '../ui/logger'
import type { CheckResult } from './types'
type Platform = 'macos' | 'windows' | 'linux'
export function detectPlatform(): Platform {
switch (os.platform()) {
case 'darwin':
return 'macos'
case 'win32':
return 'windows'
default:
return 'linux'
}
}
export function checkPlatform(): CheckResult {
const p = os.platform()
const arch = os.arch()
const release = os.release()
pass('Operating System', `${p} ${release} (${arch})`)
return { name: 'Operating System', ok: true, version: `${p} ${arch}` }
}

View File

@@ -0,0 +1,51 @@
import { execSync } from 'node:child_process'
import { pass, fail, warn, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkPlaywright(): Promise<CheckResult> {
try {
// Check if chromium browser is installed
execSync('pnpm exec playwright install --dry-run chromium', {
encoding: 'utf-8',
stdio: 'pipe'
})
pass('Playwright browsers', 'chromium installed')
return { name: 'Playwright browsers', ok: true, version: 'chromium' }
} catch {
// Browser might not be installed, try to check another way
try {
const result = execSync('pnpm exec playwright --version', {
encoding: 'utf-8',
stdio: 'pipe'
}).trim()
warn('Playwright', `${result} (browsers may need installing)`)
const instructions = [
'Playwright browsers need to be installed:',
'',
' pnpm exec playwright install chromium --with-deps',
'',
'This downloads ~200MB. Please wait...'
]
info(instructions)
return {
name: 'Playwright browsers',
ok: false,
installInstructions: instructions
}
} catch {
fail('Playwright', 'not installed')
const instructions = [
'Playwright is a project dependency. Run:',
'',
' pnpm install',
' pnpm exec playwright install chromium --with-deps'
]
info(instructions)
return {
name: 'Playwright',
ok: false,
installInstructions: instructions
}
}
}
}

View File

@@ -0,0 +1,25 @@
import { execSync } from 'node:child_process'
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkPnpm(): Promise<CheckResult> {
try {
const version = execSync('pnpm --version', { encoding: 'utf-8' }).trim()
pass('pnpm', version)
return { name: 'pnpm', ok: true, version }
} catch {
fail('pnpm', 'not installed')
const instructions = [
'Install pnpm via corepack (comes with Node.js):',
'',
' corepack enable',
' corepack prepare pnpm@latest --activate',
'',
'Or install directly:',
'',
' npm install -g pnpm'
]
info(instructions)
return { name: 'pnpm', ok: false, installInstructions: instructions }
}
}

View File

@@ -0,0 +1,39 @@
import { execSync } from 'node:child_process'
import { detectPlatform } from './platform'
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkPython(): Promise<CheckResult> {
for (const cmd of ['python3', 'python']) {
try {
const version = execSync(`${cmd} --version`, { encoding: 'utf-8' }).trim()
pass('Python', version)
return { name: 'Python', ok: true, version }
} catch {
continue
}
}
fail('Python 3', 'not installed')
const platform = detectPlatform()
const instructions =
platform === 'macos'
? [
'Python 3 is needed for the ComfyUI backend.',
'',
' brew install python3',
'',
'Or download from: https://www.python.org/downloads/'
]
: platform === 'windows'
? [
'Download Python from: https://www.python.org/downloads/',
'Check "Add Python to PATH" during install.'
]
: [
' sudo apt install python3 # Debian/Ubuntu',
' sudo dnf install python3 # Fedora'
]
info(instructions)
return { name: 'Python 3', ok: false, installInstructions: instructions }
}

View File

@@ -0,0 +1,7 @@
export interface CheckResult {
name: string
ok: boolean
version?: string
optional?: boolean
installInstructions?: string[]
}

View File

@@ -0,0 +1,34 @@
import { execSync } from 'node:child_process'
import { detectPlatform } from './platform'
import { pass, fail, info } from '../ui/logger'
import type { CheckResult } from './types'
export async function checkXcode(): Promise<CheckResult> {
if (detectPlatform() !== 'macos') {
return { name: 'Xcode CLI Tools', ok: true, version: 'n/a (not macOS)' }
}
try {
const path = execSync('xcode-select -p', { encoding: 'utf-8' }).trim()
pass('Xcode CLI Tools', path)
return { name: 'Xcode CLI Tools', ok: true, version: path }
} catch {
fail('Xcode CLI Tools', 'not installed')
const instructions = [
'Xcode Command Line Tools are required for git and build tools.',
'',
'To install, run this in Terminal:',
'',
' xcode-select --install',
'',
'A popup will appear — click "Install" and wait (~5 min).',
'When done, come back here and press Enter.'
]
info(instructions)
return {
name: 'Xcode CLI Tools',
ok: false,
installInstructions: instructions
}
}
}

View File

@@ -0,0 +1,39 @@
import { checkPlatform } from '../checks/platform'
import { checkXcode } from '../checks/xcode'
import { checkGit } from '../checks/git'
import { checkNode } from '../checks/node'
import { checkPnpm } from '../checks/pnpm'
import { checkPython } from '../checks/python'
import { checkPlaywright } from '../checks/playwright'
import { checkGh } from '../checks/gh'
import { checkDevServer } from '../checks/devServer'
import { checkBackend } from '../checks/backend'
import { header } from '../ui/logger'
import type { CheckResult } from '../checks/types'
export async function runChecks(): Promise<{
results: CheckResult[]
allPassed: boolean
}> {
header('Environment Check')
const results: CheckResult[] = []
// System checks (sequential — each depends on prior)
results.push(checkPlatform())
results.push(await checkXcode())
results.push(await checkGit())
results.push(await checkNode())
results.push(await checkPnpm())
results.push(await checkPython())
results.push(await checkPlaywright())
results.push(await checkGh())
header('Services Check')
results.push(await checkBackend())
results.push(await checkDevServer())
const requiredFailed = results.filter((r) => !r.ok && !r.optional)
return { results, allPassed: requiredFailed.length === 0 }
}

View File

@@ -0,0 +1,38 @@
import pc from 'picocolors'
import { listWorkflows, findProjectRoot } from '../recorder/runner'
import { header } from '../ui/logger'
export async function runList(): Promise<void> {
header('Available Workflows')
const projectRoot = findProjectRoot()
const workflows = listWorkflows(projectRoot)
if (workflows.length === 0) {
console.log(pc.dim(' No workflow assets found in browser_tests/assets/'))
return
}
console.log(
pc.dim(` Found ${workflows.length} workflows in browser_tests/assets/:\n`)
)
let currentDir = ''
for (const wf of workflows) {
const parts = wf.split('/')
if (parts.length > 1) {
const dir = parts.slice(0, -1).join('/')
if (dir !== currentDir) {
currentDir = dir
console.log(pc.bold(` ${dir}/`))
}
console.log(` ${pc.cyan(parts[parts.length - 1])}`)
} else {
if (currentDir !== '') {
currentDir = ''
console.log()
}
console.log(` ${pc.cyan(wf)}`)
}
}
}

View File

@@ -0,0 +1,243 @@
import { createInterface } from 'node:readline'
import { spawnSync } from 'node:child_process'
import { writeFileSync, mkdirSync, readFileSync } from 'node:fs'
import { join } from 'node:path'
import {
text,
select,
multiselect,
confirm,
isCancel,
cancel,
spinner
} from '@clack/prompts'
import pc from 'picocolors'
import { runChecks } from './check'
import {
runRecording,
findProjectRoot,
listWorkflows
} from '../recorder/runner'
import { transform, formatTransformSummary } from '../transform/engine'
import { stepHeader } from '../ui/steps'
import { pass, fail, info, blank, box } from '../ui/logger'
import { checkGhAvailable, createPr } from '../pr/gh'
import { printManualInstructions } from '../pr/manual'
import { copyToClipboard } from '../pr/clipboard'
function toSlug(description: string): string {
return description
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
}
function readMultiline(): Promise<string> {
return new Promise((resolve) => {
const lines: string[] = []
const rl = createInterface({ input: process.stdin })
rl.on('line', (line) => lines.push(line))
rl.on('close', () => resolve(lines.join('\n')))
})
}
export async function runRecord(): Promise<void> {
// ── Step 1: Environment Check ──────────────────────────────
stepHeader(1, 7, 'Environment Check')
const { allPassed } = await runChecks()
if (!allPassed) {
blank()
fail('Some required checks failed. Fix the issues above.')
process.exit(1)
}
// ── Step 2: Project Setup ──────────────────────────────────
stepHeader(2, 7, 'Project Setup')
let projectRoot: string
try {
projectRoot = findProjectRoot()
} catch (err) {
fail(err instanceof Error ? err.message : 'Could not find project root')
process.exit(1)
}
const s = spinner()
s.start('Installing dependencies...')
spawnSync('pnpm', ['install'], { cwd: projectRoot, stdio: 'pipe' })
s.stop('Dependencies installed')
pass('Project ready', projectRoot)
// ── Step 3: Backend & Dev Server ───────────────────────────
stepHeader(3, 7, 'Backend & Dev Server')
pass('Services checked in Step 1')
// ── Step 4: Configure Your Test ────────────────────────────
stepHeader(4, 7, 'Configure Your Test')
const description = await text({
message: 'What are you testing?',
placeholder: 'e.g., adding a KSampler node and queuing'
})
if (isCancel(description)) {
cancel('Operation cancelled')
process.exit(0)
}
let slug = toSlug(description)
const filenameOk = await confirm({
message: `Generated filename: ${slug}.spec.ts — looks good?`
})
if (isCancel(filenameOk)) {
cancel('Operation cancelled')
process.exit(0)
}
if (!filenameOk) {
const customName = await text({
message: 'Enter a custom filename (without .spec.ts):',
placeholder: slug
})
if (isCancel(customName)) {
cancel('Operation cancelled')
process.exit(0)
}
slug = toSlug(customName)
}
const selectedTags = await multiselect({
message: 'Select tags for this test:',
options: [
{ value: '@canvas', label: '@canvas' },
{ value: '@widget', label: '@widget' },
{ value: '@sidebar', label: '@sidebar' },
{ value: '@smoke', label: '@smoke' },
{ value: '@mobile', label: '@mobile' },
{ value: '@screenshot', label: '@screenshot' }
],
initialValues: ['@canvas']
})
if (isCancel(selectedTags)) {
cancel('Operation cancelled')
process.exit(0)
}
const workflows = listWorkflows(projectRoot)
const workflowOptions: {
value: string
label: string
hint?: string
}[] = [
{ value: '', label: '(empty canvas)', hint: 'start fresh' },
...workflows.map((wf) => ({ value: wf, label: wf }))
]
const selectedWorkflow = await select({
message: 'Start with a pre-loaded workflow?',
options: workflowOptions
})
if (isCancel(selectedWorkflow)) {
cancel('Operation cancelled')
process.exit(0)
}
// ── Step 5: Record ─────────────────────────────────────────
stepHeader(5, 7, 'Record')
const result = await runRecording({
testName: slug,
workflow: selectedWorkflow || undefined,
projectRoot
})
if (!result.success) {
fail('Recording failed', result.error)
process.exit(1)
}
// ── Step 6: Paste & Transform ──────────────────────────────
stepHeader(6, 7, 'Paste & Transform')
info([
'Copy the generated code from the Playwright Inspector.',
'',
'Paste your code below, then press ' +
pc.bold('Ctrl+D') +
' (Mac/Linux) or ' +
pc.bold('Ctrl+Z') +
' (Windows) when done:'
])
blank()
const pastedCode = await readMultiline()
if (!pastedCode.trim()) {
blank()
info([
'No code pasted. You can transform later with:',
pc.cyan(` comfy-test transform <file>`)
])
process.exit(0)
}
const transformResult = transform(pastedCode, {
testName: slug,
tags: selectedTags as string[]
})
blank()
const summary = formatTransformSummary(transformResult)
for (const line of summary) {
console.log(` ${line}`)
}
const testsDir = join(projectRoot, 'browser_tests', 'tests')
mkdirSync(testsDir, { recursive: true })
const outputPath = join(testsDir, `${slug}.spec.ts`)
writeFileSync(outputPath, transformResult.code)
blank()
pass('Test saved', outputPath)
// ── Step 7: Finalize ───────────────────────────────────────
stepHeader(7, 7, 'Finalize')
box([
'Run your test:',
pc.cyan(` pnpm exec playwright test ${slug} --headed`),
'',
'Review in UI mode:',
pc.cyan(' pnpm exec playwright test --ui')
])
blank()
const wantPr = await confirm({
message: 'Create a Pull Request now?'
})
if (isCancel(wantPr)) {
cancel('Operation cancelled')
process.exit(0)
}
if (wantPr) {
const gh = await checkGhAvailable()
if (gh.available && gh.authenticated) {
await createPr({
testFilePath: outputPath,
testName: slug,
description: description as string
})
} else {
const fileContents = readFileSync(outputPath, 'utf-8')
await copyToClipboard(fileContents)
const relativePath = `browser_tests/tests/${slug}.spec.ts`
printManualInstructions({
testFilePath: outputPath,
testName: slug,
relativePath
})
}
} else {
blank()
info(['You can create a PR later.', pc.dim(`Test file: ${outputPath}`)])
}
}

View File

@@ -0,0 +1,48 @@
import { readFileSync, writeFileSync } from 'node:fs'
import pc from 'picocolors'
import { transform, formatTransformSummary } from '../transform/engine'
import { header } from '../ui/logger'
export async function runTransform(
filePath: string,
options: {
testName?: string
tags?: string[]
output?: string
} = {}
): Promise<void> {
header('Transform Codegen → Conventions')
// Read input
let rawCode: string
try {
rawCode = readFileSync(filePath, 'utf-8')
} catch {
console.log(pc.red(` Could not read file: ${filePath}`))
process.exit(1)
}
console.log(pc.dim(` Input: ${filePath}`))
console.log()
// Transform
const testName =
options.testName ??
filePath.split('/').pop()?.replace('.raw.spec.ts', '') ??
'test'
const tags = options.tags ?? ['@canvas']
const result = transform(rawCode, { testName, tags })
// Print summary
console.log(' Transforms applied:')
for (const line of formatTransformSummary(result)) {
console.log(` ${line}`)
}
console.log()
// Write output
const outputPath =
options.output ?? filePath.replace('.raw.spec.ts', '.spec.ts')
writeFileSync(outputPath, result.code)
console.log(pc.green(` ✅ Saved: ${outputPath}`))
}

View File

@@ -0,0 +1,65 @@
#!/usr/bin/env node
import { intro, outro } from '@clack/prompts'
import pc from 'picocolors'
const args = process.argv.slice(2)
const command = args[0]
intro(pc.bgCyan(pc.black(' 🎭 ComfyUI Test Recorder ')))
switch (command) {
case 'record': {
const { runRecord } = await import('./commands/record')
await runRecord()
break
}
case 'transform': {
const filePath = args[1]
if (!filePath) {
console.log(pc.red(' Usage: comfy-test transform <file>'))
process.exit(1)
}
const { runTransform } = await import('./commands/transform')
await runTransform(filePath, {
testName: args[2],
tags: args.slice(3)
})
break
}
case 'check': {
const { runChecks } = await import('./commands/check')
const { allPassed } = await runChecks()
if (!allPassed) {
console.log()
console.log(
pc.red(
' Some required checks failed. Fix the issues above and try again.'
)
)
}
break
}
case 'list': {
const { runList } = await import('./commands/list')
await runList()
break
}
default:
console.log(`
Usage: comfy-test <command>
Commands:
record Record a new browser test interactively
transform Transform raw codegen output to conventions
check Check environment prerequisites
list List available test workflows
Options:
--help Show help
--version Show version
`)
break
}
outro(pc.dim('https://github.com/Comfy-Org/ComfyUI_frontend'))

View File

@@ -0,0 +1,36 @@
import { spawnSync } from 'node:child_process'
import { detectPlatform } from '../checks/platform'
export async function copyToClipboard(content: string): Promise<boolean> {
const platform = detectPlatform()
let cmd: string
let args: string[]
if (platform === 'macos') {
cmd = 'pbcopy'
args = []
} else if (platform === 'windows') {
cmd = 'clip.exe'
args = []
} else {
// Linux: try xclip first, fall back to xsel
const xclip = spawnSync('xclip', ['-selection', 'clipboard'], {
input: content,
stdio: 'pipe'
})
if (xclip.status === 0) return true
const xsel = spawnSync('xsel', ['--clipboard', '--input'], {
input: content,
stdio: 'pipe'
})
return xsel.status === 0
}
const result = spawnSync(cmd, args, {
input: content,
stdio: 'pipe'
})
return result.status === 0
}

View File

@@ -0,0 +1,90 @@
import { execSync, spawnSync } from 'node:child_process'
import { pass, fail } from '../ui/logger'
interface PrOptions {
testFilePath: string
testName: string
description: string
branchName?: string
}
interface PrResult {
success: boolean
url?: string
error?: string
}
export async function checkGhAvailable(): Promise<{
available: boolean
authenticated: boolean
}> {
try {
execSync('gh --version', { stdio: 'pipe' })
} catch {
return { available: false, authenticated: false }
}
try {
execSync('gh auth status', {
stdio: 'pipe',
encoding: 'utf-8'
})
return { available: true, authenticated: true }
} catch {
return { available: true, authenticated: false }
}
}
export async function createPr(options: PrOptions): Promise<PrResult> {
const branchName = options.branchName ?? `test/${options.testName}`
const commitMsg = `test: add ${options.testName} e2e test\n\n${options.description}`
const prTitle = `test: add ${options.testName} e2e test`
const prBody =
`${options.description}\n\n---\n\n` + 'Recorded with `comfy-test record`'
const checkout = spawnSync('git', ['checkout', '-b', branchName], {
encoding: 'utf-8',
stdio: 'pipe'
})
if (checkout.status !== 0) {
fail('Branch creation failed', checkout.stderr.trim())
return {
success: false,
error: checkout.stderr.trim()
}
}
pass('Created branch', branchName)
const add = spawnSync('git', ['add', options.testFilePath], {
encoding: 'utf-8',
stdio: 'pipe'
})
if (add.status !== 0) {
fail('Git add failed', add.stderr.trim())
return { success: false, error: add.stderr.trim() }
}
const commit = spawnSync('git', ['commit', '-m', commitMsg], {
encoding: 'utf-8',
stdio: 'pipe'
})
if (commit.status !== 0) {
fail('Git commit failed', commit.stderr.trim())
return { success: false, error: commit.stderr.trim() }
}
pass('Committed test file')
const pr = spawnSync(
'gh',
['pr', 'create', '--title', prTitle, '--body', prBody, '--fill'],
{ encoding: 'utf-8', stdio: 'pipe' }
)
if (pr.status !== 0) {
fail('PR creation failed', pr.stderr.trim())
return { success: false, error: pr.stderr.trim() }
}
const url = pr.stdout.trim()
pass('Pull request created', url)
return { success: true, url }
}

View File

@@ -0,0 +1,29 @@
import { info } from '../ui/logger'
interface ManualPrOptions {
testFilePath: string
testName: string
relativePath: string
repoUrl?: string
}
const DEFAULT_REPO = 'https://github.com/Comfy-Org/ComfyUI_frontend'
export function printManualInstructions(options: ManualPrOptions): void {
const repo = options.repoUrl ?? DEFAULT_REPO
info([
'Create a PR manually:',
'',
` 1. Go to: ${repo}`,
' 2. Click "Add file" → "Create new file"',
` 3. Set path to: ${options.relativePath}`,
' 4. Paste the contents (copied to your clipboard ✅)',
` 5. Write commit message: "test: add ${options.testName} e2e test"`,
' 6. Select "Create a new branch" → click "Propose new file"',
' 7. Click "Create pull request"',
'',
'Or, ask an AI agent: "Create a PR with this test file"',
`(The file is at ${options.testFilePath})`
])
}

View File

@@ -0,0 +1,147 @@
import { spawnSync } from 'node:child_process'
import { existsSync, readdirSync, statSync } from 'node:fs'
import { join } from 'node:path'
import pc from 'picocolors'
import { generateRecordingTemplate, cleanupRecordingTemplate } from './template'
import { box } from '../ui/logger'
interface RunnerOptions {
testName: string
workflow?: string
projectRoot: string
}
interface RecordingResult {
success: boolean
rawOutputPath?: string
error?: string
}
/**
* Find the project root by looking for playwright.config.ts
*/
export function findProjectRoot(): string {
let dir = process.cwd()
while (dir !== '/') {
if (existsSync(join(dir, 'playwright.config.ts'))) {
return dir
}
dir = join(dir, '..')
}
throw new Error(
'Could not find project root (no playwright.config.ts found). ' +
'Run this command from the ComfyUI_frontend directory.'
)
}
/**
* List available workflow assets.
*/
export function listWorkflows(projectRoot: string): string[] {
const assetsDir = join(projectRoot, 'browser_tests', 'assets')
const results: string[] = []
function walk(dir: string, prefix: string) {
try {
for (const entry of readdirSync(dir)) {
const fullPath = join(dir, entry)
const relPath = prefix ? `${prefix}/${entry}` : entry
const stat = statSync(fullPath)
if (stat.isDirectory()) {
walk(fullPath, relPath)
} else if (entry.endsWith('.json')) {
results.push(relPath.replace(/\.json$/, ''))
}
}
} catch {
// Directory might not exist
}
}
walk(assetsDir, '')
return results.sort()
}
/**
* Run the recording session.
*
* 1. Generate the temporary test file with page.pause()
* 2. Run it in headed mode — this opens the Playwright Inspector
* 3. User records their actions
* 4. User closes the browser
* 5. We save the output as *.raw.spec.ts
*/
export async function runRecording(
options: RunnerOptions
): Promise<RecordingResult> {
const browserTestsDir = join(options.projectRoot, 'browser_tests')
// Generate the recording template
const tempFile = generateRecordingTemplate(
{ testName: options.testName, workflow: options.workflow },
browserTestsDir
)
console.log()
box([
'A browser window will open.',
'',
'👉 Perform your test actions:',
' • Click, type, drag — everything is recorded',
' • Use toolbar buttons to add assertions',
' • When done, close the browser window',
'',
'The Playwright Inspector shows generated code.'
])
console.log()
try {
// Run the test in headed mode with PWDEBUG to force inspector
spawnSync(
'pnpm',
[
'exec',
'playwright',
'test',
'_recording-session',
'--headed',
'--project=chromium',
'--timeout=0'
],
{
cwd: options.projectRoot,
stdio: 'inherit',
env: {
...process.env,
PWDEBUG: '1',
PLAYWRIGHT_LOCAL: '1'
}
}
)
// The user will have copied code from the Inspector.
// We can't automatically capture Inspector output, so we'll
// prompt the user to paste it or check if they saved a file.
console.log()
console.log(pc.green(' ✅ Recording session complete.'))
console.log()
// Save a placeholder raw file — the user pastes codegen output here
// or the record command handles prompting for it
const rawOutputPath = join(
browserTestsDir,
'tests',
`${options.testName}.raw.spec.ts`
)
return { success: true, rawOutputPath }
} catch (err) {
return {
success: false,
error: `Recording failed: ${err instanceof Error ? err.message : String(err)}`
}
} finally {
// Always clean up the temp recording file
cleanupRecordingTemplate(browserTestsDir)
}
}

View File

@@ -0,0 +1,70 @@
import { writeFileSync, mkdirSync, unlinkSync } from 'node:fs'
import { join, dirname } from 'node:path'
interface TemplateOptions {
workflow?: string
testName: string
}
/**
* Generates a temporary test file that uses page.pause() to open
* the Playwright Inspector with codegen controls.
*
* The test file:
* 1. Uses comfyPageFixture to get full fixture context
* 2. Optionally loads a workflow
* 3. Calls page.pause() to open the Inspector
*/
export function generateRecordingTemplate(
options: TemplateOptions,
browserTestsDir: string
): string {
const filePath = join(browserTestsDir, 'tests', `_recording-session.spec.ts`)
const workflowLine = options.workflow
? ` // Load the selected workflow\n await comfyPage.workflow.loadWorkflow('${options.workflow}')\n await comfyPage.nextFrame()\n`
: ''
const code = `/**
* Auto-generated recording session.
* This file is temporary — it will be deleted after recording.
*
* DO NOT COMMIT THIS FILE.
*/
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test('recording: ${options.testName}', async ({ comfyPage }) => {
${workflowLine}
// ┌────────────────────────────────────────────────────────┐
// │ The Playwright Inspector will open. │
// │ │
// │ 1. Click the Record button (red circle) to start │
// │ 2. Perform your test actions in the browser │
// │ 3. Use toolbar buttons to add assertions │
// │ 4. Click Stop when done │
// │ 5. Copy the generated code from the Inspector │
// │ 6. Close the browser window │
// └────────────────────────────────────────────────────────┘
await comfyPage.page.pause()
})
`
mkdirSync(dirname(filePath), { recursive: true })
writeFileSync(filePath, code)
return filePath
}
/**
* Clean up the temporary recording file.
*/
export function cleanupRecordingTemplate(browserTestsDir: string): void {
const filePath = join(browserTestsDir, 'tests', '_recording-session.spec.ts')
try {
unlinkSync(filePath)
} catch {
// File might already be deleted
}
}

View File

@@ -0,0 +1,89 @@
import { transformRules, structuralTransforms } from './rules'
interface TransformResult {
code: string
appliedRules: { name: string; description: string }[]
warnings: string[]
}
/**
* Transform raw Playwright codegen output into ComfyUI conventions.
*/
export function transform(
rawCode: string,
options: {
testName?: string
tags?: string[]
} = {}
): TransformResult {
const testName = options.testName ?? 'unnamed-test'
const tags = options.tags ?? ['@canvas']
let code = rawCode
const appliedRules: { name: string; description: string }[] = []
const warnings: string[] = []
// Phase 1: Apply regex-based rules
for (const rule of transformRules) {
const before = code
if (typeof rule.replacement === 'string') {
code = code.replace(rule.pattern, rule.replacement)
} else {
code = code.replace(
rule.pattern,
rule.replacement as (...args: string[]) => string
)
}
if (code !== before) {
appliedRules.push({ name: rule.name, description: rule.description })
}
}
// Phase 2: Clean up empty lines from removed statements
code = code.replace(/\n{3,}/g, '\n\n')
// Phase 3: Apply structural transforms
for (const transform of structuralTransforms) {
const before = code
code = transform.apply(code, testName, tags)
if (code !== before) {
appliedRules.push({
name: transform.name,
description: transform.description
})
}
}
// Phase 4: Check for remaining issues and warn
if (code.includes('waitForTimeout')) {
warnings.push(
'Still contains waitForTimeout — replace with comfyPage.nextFrame() or retrying assertions'
)
}
if (code.includes("from '@playwright/test'")) {
warnings.push(
'Still imports from @playwright/test — should use ../fixtures/ComfyPage'
)
}
if (/position:\s*\{\s*x:\s*\d+,\s*y:\s*\d+/.test(code)) {
warnings.push(
'Contains pixel coordinates — consider replacing with node references (comfyPage.nodeOps.*) where possible'
)
}
return { code: code.trim() + '\n', appliedRules, warnings }
}
/**
* Get a human-readable summary of what was transformed.
*/
export function formatTransformSummary(result: TransformResult): string[] {
const lines: string[] = []
for (const rule of result.appliedRules) {
lines.push(`${rule.description}`)
}
for (const warning of result.warnings) {
lines.push(`⚠️ ${warning}`)
}
return lines
}

View File

@@ -0,0 +1,123 @@
interface TransformRule {
name: string
description: string
pattern: RegExp
replacement: string | ((match: string, ...groups: string[]) => string)
category: 'import' | 'fixture' | 'locator' | 'wait' | 'structure' | 'cleanup'
}
export const transformRules: TransformRule[] = [
// === Import transforms ===
{
name: 'replace-test-import',
description: 'Use comfyPageFixture instead of @playwright/test',
pattern:
/import\s*\{\s*test\s*,?\s*expect\s*\}\s*from\s*['"]@playwright\/test['"]/,
replacement: `import {\n comfyPageFixture as test,\n comfyExpect as expect\n} from '../fixtures/ComfyPage'`,
category: 'import'
},
{
name: 'replace-test-only-import',
description: 'Use comfyPageFixture when only test is imported',
pattern: /import\s*\{\s*test\s*\}\s*from\s*['"]@playwright\/test['"]/,
replacement: `import { comfyPageFixture as test } from '../fixtures/ComfyPage'`,
category: 'import'
},
{
name: 'replace-expect-only-import',
description: 'Use comfyExpect when only expect is imported',
pattern: /import\s*\{\s*expect\s*\}\s*from\s*['"]@playwright\/test['"]/,
replacement: `import { comfyExpect as expect } from '../fixtures/ComfyPage'`,
category: 'import'
},
// === Fixture transforms ===
{
name: 'replace-page-destructure',
description: 'Use comfyPage fixture instead of page',
pattern: /async\s*\(\s*\{\s*page\s*\}\s*\)/g,
replacement: 'async ({ comfyPage })',
category: 'fixture'
},
// === Remove page.goto ===
{
name: 'remove-goto',
description: 'Remove page.goto — fixture handles navigation',
pattern: /^\s*await\s+page\.goto\s*\([^)]*\)\s*;?\s*$/gm,
replacement: '',
category: 'locator'
},
// === Locator transforms ===
{
name: 'replace-canvas-locator',
description: 'Use comfyPage.canvas instead of page.locator("canvas")',
pattern: /page\.locator\(\s*['"]canvas['"]\s*\)/g,
replacement: 'comfyPage.canvas',
category: 'locator'
},
{
name: 'replace-search-placeholder',
description: 'Use comfyPage.searchBox for search input',
pattern: /page\.getByPlaceholder\(\s*['"]Search\s+Nodes\.{0,3}['"]\s*\)/g,
replacement: 'comfyPage.searchBox.input',
category: 'locator'
},
{
name: 'replace-bare-page',
description: 'Replace bare page references with comfyPage.page',
pattern: /(?<!\w)page\./g,
replacement: 'comfyPage.page.',
category: 'locator'
},
// === Wait transforms ===
{
name: 'replace-waitForTimeout',
description: 'Use comfyPage.nextFrame() instead of arbitrary waits',
pattern:
/await\s+(?:comfyPage\.)?page\.waitForTimeout\s*\(\s*\d+\s*\)\s*;?/g,
replacement: 'await comfyPage.nextFrame()',
category: 'wait'
}
]
/**
* Rules that need structural changes (not just regex replacement).
* These are applied by the engine after regex rules.
*/
interface StructuralTransform {
name: string
description: string
apply: (code: string, testName: string, tags: string[]) => string
}
export const structuralTransforms: StructuralTransform[] = [
{
name: 'wrap-in-describe',
description: 'Wrap test in test.describe with tags and afterEach',
apply: (code: string, testName: string, tags: string[]) => {
// If already has test.describe, skip
if (code.includes('test.describe')) return code
const tagStr = tags.map((t) => `'${t}'`).join(', ')
const descName = testName.replace(/[-_]/g, ' ').replace(/\.spec\.ts$/, '')
// Find the test() call and wrap it
const testMatch = code.match(/^(import[\s\S]*?\n\n?)(test\s*\([\s\S]*)$/m)
if (!testMatch) return code
const imports = testMatch[1]
const testBody = testMatch[2]
return `${imports}test.describe('${descName}', { tag: [${tagStr}] }, () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.canvasOps.resetView()
})
${testBody.replace(/^/gm, ' ').trimStart()}
})\n`
}
}
]

View File

@@ -0,0 +1,46 @@
import pc from 'picocolors'
export function pass(label: string, detail?: string) {
const d = detail ? pc.dim(` ${detail}`) : ''
console.log(` ${pc.green('✅')} ${label}${d}`)
}
export function fail(label: string, detail?: string) {
const d = detail ? pc.dim(` ${detail}`) : ''
console.log(` ${pc.red('❌')} ${label}${d}`)
}
export function warn(label: string, detail?: string) {
const d = detail ? pc.dim(` ${detail}`) : ''
console.log(` ${pc.yellow('⚠️')} ${label}${d}`)
}
function wait(label: string) {
console.log(` ${pc.blue('⏳')} ${label}`)
}
export function info(lines: string[]) {
for (const line of lines) {
console.log(` ${pc.dim('┃')} ${line}`)
}
}
export function blank() {
console.log()
}
export function header(text: string) {
console.log()
console.log(pc.bold(` ━━━ ${text} ━━━`))
console.log()
}
export function box(lines: string[]) {
const maxLen = Math.max(...lines.map((l) => l.length))
const border = '─'.repeat(maxLen + 4)
console.log(`${border}`)
for (const line of lines) {
console.log(`${line.padEnd(maxLen + 2)}`)
}
console.log(`${border}`)
}

View File

@@ -0,0 +1,7 @@
import pc from 'picocolors'
export function stepHeader(current: number, total: number, label: string) {
console.log()
console.log(pc.bold(` ━━━ Step ${current} of ${total}: ${label} ━━━`))
console.log()
}

View File

@@ -0,0 +1,15 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"outDir": "./dist",
"rootDir": "./src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"declaration": true,
"sourceMap": true
},
"include": ["src/**/*.ts"]
}