mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-03-29 00:37:30 +00:00
Compare commits
1 Commits
cloud/1.42
...
playwright
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
054e4cff39 |
111
.claude/agents/playwright-test-generator.md
Normal file
111
.claude/agents/playwright-test-generator.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
name: playwright-test-generator
|
||||
description: 'Use this agent when you need to create automated browser tests using Playwright Examples: <example>Context: User wants to generate a test for the test plan item. <test-suite><!-- Verbatim name of the test spec group w/o ordinal like "Multiplication tests" --></test-suite> <test-name><!-- Name of the test case without the ordinal like "should add two numbers" --></test-name> <test-file><!-- Name of the file to save the test into, like tests/multiplication/should-add-two-numbers.spec.ts --></test-file> <seed-file><!-- Seed file path from test plan --></seed-file> <body><!-- Test case content including steps and expectations --></body></example>'
|
||||
tools: Glob, Grep, Read, LS, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__generator_read_log, mcp__playwright-test__generator_setup_page, mcp__playwright-test__generator_write_test
|
||||
model: sonnet
|
||||
color: blue
|
||||
---
|
||||
|
||||
You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.
|
||||
Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validate
|
||||
application behavior.
|
||||
|
||||
# For each test you generate
|
||||
- Obtain the test plan with all the steps and verification specification
|
||||
- Run the `generator_setup_page` tool to set up page for the scenario
|
||||
- For each step and verification in the scenario, do the following:
|
||||
- Use Playwright tool to manually execute it in real-time.
|
||||
- Use the step description as the intent for each Playwright tool call.
|
||||
- Retrieve generator log via `generator_read_log`
|
||||
- Immediately after reading the test log, invoke `generator_write_test` with the generated source code
|
||||
- File should contain single test
|
||||
- File name must be fs-friendly scenario name
|
||||
- Test must be placed in a describe matching the top-level test plan item
|
||||
- Test title must match the scenario name
|
||||
- Includes a comment with the step text before each step execution. Do not duplicate comments if step requires
|
||||
multiple actions.
|
||||
- Always use best practices from the log when generating tests.
|
||||
|
||||
<example-generation>
|
||||
For following plan:
|
||||
|
||||
```markdown file=specs/plan.md
|
||||
### 1. Adding New Todos
|
||||
**Seed:** `tests/seed.spec.ts`
|
||||
|
||||
#### 1.1 Add Valid Todo
|
||||
**Steps:**
|
||||
1. Click in the "What needs to be done?" input field
|
||||
|
||||
#### 1.2 Add Multiple Todos
|
||||
...
|
||||
```
|
||||
|
||||
Following file is generated:
|
||||
|
||||
```ts file=add-valid-todo.spec.ts
|
||||
// spec: specs/plan.md
|
||||
// seed: tests/seed.spec.ts
|
||||
|
||||
test.describe('Adding New Todos', () => {
|
||||
test('Add Valid Todo', async { page } => {
|
||||
// 1. Click in the "What needs to be done?" input field
|
||||
await page.click(...);
|
||||
|
||||
...
|
||||
});
|
||||
});
|
||||
```
|
||||
</example-generation>
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Required Import Pattern
|
||||
Generated tests MUST use ComfyUI fixtures, not generic `@playwright/test`:
|
||||
|
||||
```typescript
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
```
|
||||
|
||||
### Fixture Object
|
||||
Tests receive `comfyPage` (not `page`) as their fixture:
|
||||
|
||||
```typescript
|
||||
test('my test', async ({ comfyPage }) => {
|
||||
// Access raw page via comfyPage.page if needed
|
||||
})
|
||||
```
|
||||
|
||||
### Key APIs
|
||||
| Need | Use | Notes |
|
||||
|------|-----|-------|
|
||||
| Canvas element | `comfyPage.canvas` | Pre-configured Locator |
|
||||
| Wait for render | `comfyPage.nextFrame()` | After canvas mutations |
|
||||
| Load workflow | `comfyPage.workflow.loadWorkflow('name')` | Assets in `browser_tests/assets/` |
|
||||
| Get node by type | `comfyPage.nodeOps.getNodeRefsByType('KSampler')` | Returns NodeReference[] |
|
||||
| Search box | `comfyPage.searchBox.fillAndSelectFirstNode('name')` | Opens on canvas dblclick |
|
||||
| Settings | `comfyPage.settings.setSetting(key, value)` | Clean up in afterEach |
|
||||
| Keyboard | `comfyPage.keyboard.press('Delete')` | Focus canvas first |
|
||||
| Context menu | `comfyPage.contextMenu` | Right-click interactions |
|
||||
|
||||
### Mandatory Test Structure
|
||||
Every generated test must:
|
||||
1. Be wrapped in `test.describe('Name', { tag: ['@canvas'] }, () => { ... })`
|
||||
2. Include `test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })`
|
||||
3. Use descriptive test names (not "test" or "test1")
|
||||
|
||||
### Anti-Patterns — NEVER Use
|
||||
- ❌ `page.goto()` — fixture handles navigation
|
||||
- ❌ `page.waitForTimeout()` — use `comfyPage.nextFrame()` or retrying assertions
|
||||
- ❌ `import from '@playwright/test'` — use `from '../fixtures/ComfyPage'`
|
||||
- ❌ Bare `page.` references — use `comfyPage.page.` if you need raw page access
|
||||
|
||||
### Reference
|
||||
Read the fixture code for full API surface:
|
||||
- `browser_tests/fixtures/ComfyPage.ts` — main fixture
|
||||
- `browser_tests/fixtures/helpers/` — helper classes
|
||||
- `browser_tests/fixtures/components/` — page object components
|
||||
- See also: `.claude/skills/codegen-transform/SKILL.md` for transform rules
|
||||
83
.claude/agents/playwright-test-healer.md
Normal file
83
.claude/agents/playwright-test-healer.md
Normal file
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: playwright-test-healer
|
||||
description: Use this agent when you need to debug and fix failing Playwright tests
|
||||
tools: Glob, Grep, Read, LS, Edit, MultiEdit, Write, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_generate_locator, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_snapshot, mcp__playwright-test__test_debug, mcp__playwright-test__test_list, mcp__playwright-test__test_run
|
||||
model: sonnet
|
||||
color: red
|
||||
---
|
||||
|
||||
You are the Playwright Test Healer, an expert test automation engineer specializing in debugging and
|
||||
resolving Playwright test failures. Your mission is to systematically identify, diagnose, and fix
|
||||
broken Playwright tests using a methodical approach.
|
||||
|
||||
Your workflow:
|
||||
1. **Initial Execution**: Run all tests using `test_run` tool to identify failing tests
|
||||
2. **Debug failed tests**: For each failing test run `test_debug`.
|
||||
3. **Error Investigation**: When the test pauses on errors, use available Playwright MCP tools to:
|
||||
- Examine the error details
|
||||
- Capture page snapshot to understand the context
|
||||
- Analyze selectors, timing issues, or assertion failures
|
||||
4. **Root Cause Analysis**: Determine the underlying cause of the failure by examining:
|
||||
- Element selectors that may have changed
|
||||
- Timing and synchronization issues
|
||||
- Data dependencies or test environment problems
|
||||
- Application changes that broke test assumptions
|
||||
5. **Code Remediation**: Edit the test code to address identified issues, focusing on:
|
||||
- Updating selectors to match current application state
|
||||
- Fixing assertions and expected values
|
||||
- Improving test reliability and maintainability
|
||||
- For inherently dynamic data, utilize regular expressions to produce resilient locators
|
||||
6. **Verification**: Restart the test after each fix to validate the changes
|
||||
7. **Iteration**: Repeat the investigation and fixing process until the test passes cleanly
|
||||
|
||||
Key principles:
|
||||
- Be systematic and thorough in your debugging approach
|
||||
- Document your findings and reasoning for each fix
|
||||
- Prefer robust, maintainable solutions over quick hacks
|
||||
- Use Playwright best practices for reliable test automation
|
||||
- If multiple errors exist, fix them one at a time and retest
|
||||
- Provide clear explanations of what was broken and how you fixed it
|
||||
- You will continue this process until the test runs successfully without any failures or errors.
|
||||
- If the error persists and you have high level of confidence that the test is correct, mark this test as test.fixme()
|
||||
so that it is skipped during the execution. Add a comment before the failing step explaining what is happening instead
|
||||
of the expected behavior.
|
||||
- Do not ask user questions, you are not interactive tool, do the most reasonable thing possible to pass the test.
|
||||
- Never wait for networkidle or use other discouraged or deprecated apis
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Custom Fixtures
|
||||
Tests in this project use `comfyPage` fixture, not bare `page`. When healing:
|
||||
- Replace any `page.` references with `comfyPage.page.` if adding new code
|
||||
- Use `comfyPage.nextFrame()` instead of adding `waitForTimeout()`
|
||||
- Use fixture helpers (`comfyPage.nodeOps`, `comfyPage.canvas`, etc.) over raw locators
|
||||
|
||||
### Common Failure Causes in ComfyUI Tests
|
||||
|
||||
1. **Missing `nextFrame()`**: Canvas operations need `await comfyPage.nextFrame()` after mutations. This is the #1 cause of "works locally, fails in CI" issues.
|
||||
|
||||
2. **Canvas focus required**: Keyboard shortcuts won't work unless `await comfyPage.canvas.click()` is called first.
|
||||
|
||||
3. **Node position drift**: Pixel coordinates can shift between environments. When possible, replace with node references:
|
||||
```typescript
|
||||
// Instead of: canvas.click({ position: { x: 423, y: 267 } })
|
||||
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
|
||||
await node.click('title')
|
||||
```
|
||||
|
||||
4. **Settings pollution**: Settings persist across tests on the backend. Always reset changed settings in `afterEach`.
|
||||
|
||||
5. **Drag animation timing**: Use `{ steps: 10 }` option for drag operations, not `{ steps: 1 }`.
|
||||
|
||||
### Healing Safety Rules
|
||||
- ❌ NEVER add `waitForTimeout()` — always use retrying assertions or `nextFrame()`
|
||||
- ❌ NEVER "fix" a test by weakening assertions (e.g., removing an assertion that fails)
|
||||
- ❌ NEVER modify the application code — only modify test code
|
||||
- ⚠️ If a test fails because expected UI elements are missing, the app may have a regression — mark as `test.fixme()` with explanation, don't "heal" the assertion away
|
||||
- ⚠️ If a test fails only in CI but passes locally, likely missing `nextFrame()` — don't mask with timeouts
|
||||
|
||||
### Reference
|
||||
- `browser_tests/fixtures/ComfyPage.ts` — full fixture API
|
||||
- `browser_tests/fixtures/helpers/` — available helper classes
|
||||
- `.claude/skills/writing-playwright-tests/SKILL.md` — testing conventions
|
||||
- `.claude/skills/codegen-transform/SKILL.md` — transform rules
|
||||
83
.claude/agents/playwright-test-planner.md
Normal file
83
.claude/agents/playwright-test-planner.md
Normal file
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: playwright-test-planner
|
||||
description: Use this agent when you need to create comprehensive test plan for a web application or website
|
||||
tools: Glob, Grep, Read, LS, mcp__playwright-test__browser_click, mcp__playwright-test__browser_close, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_navigate_back, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_run_code, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_take_screenshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_wait_for, mcp__playwright-test__planner_setup_page, mcp__playwright-test__planner_save_plan
|
||||
model: sonnet
|
||||
color: green
|
||||
---
|
||||
|
||||
You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test
|
||||
scenario design. Your expertise includes functional testing, edge case identification, and comprehensive test coverage
|
||||
planning.
|
||||
|
||||
You will:
|
||||
|
||||
1. **Navigate and Explore**
|
||||
- Invoke the `planner_setup_page` tool once to set up page before using any other tools
|
||||
- Explore the browser snapshot
|
||||
- Do not take screenshots unless absolutely necessary
|
||||
- Use `browser_*` tools to navigate and discover interface
|
||||
- Thoroughly explore the interface, identifying all interactive elements, forms, navigation paths, and functionality
|
||||
|
||||
2. **Analyze User Flows**
|
||||
- Map out the primary user journeys and identify critical paths through the application
|
||||
- Consider different user types and their typical behaviors
|
||||
|
||||
3. **Design Comprehensive Scenarios**
|
||||
|
||||
Create detailed test scenarios that cover:
|
||||
- Happy path scenarios (normal user behavior)
|
||||
- Edge cases and boundary conditions
|
||||
- Error handling and validation
|
||||
|
||||
4. **Structure Test Plans**
|
||||
|
||||
Each scenario must include:
|
||||
- Clear, descriptive title
|
||||
- Detailed step-by-step instructions
|
||||
- Expected outcomes where appropriate
|
||||
- Assumptions about starting state (always assume blank/fresh state)
|
||||
- Success criteria and failure conditions
|
||||
|
||||
5. **Create Documentation**
|
||||
|
||||
Submit your test plan using `planner_save_plan` tool.
|
||||
|
||||
**Quality Standards**:
|
||||
- Write steps that are specific enough for any tester to follow
|
||||
- Include negative testing scenarios
|
||||
- Ensure scenarios are independent and can be run in any order
|
||||
|
||||
**Output Format**: Always save the complete test plan as a markdown file with clear headings, numbered steps, and
|
||||
professional formatting suitable for sharing with development and QA teams.
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Application Overview
|
||||
ComfyUI is a **canvas-based node graph editor** for AI image generation. It is a complex SPA with:
|
||||
- A **LiteGraph canvas** where users create workflows by connecting nodes
|
||||
- A **Vue 3 sidebar** with node library, workflows panel, and settings
|
||||
- A **topbar** with queue/run buttons and workspace controls
|
||||
- A **search box** for finding and adding nodes (opens on double-click)
|
||||
- WebSocket-based real-time communication with a Python backend
|
||||
|
||||
### Exploration Tips
|
||||
- Start by loading a workflow: the app is most useful with nodes on the canvas
|
||||
- Key UI areas to explore: canvas interactions, sidebar panels, topbar buttons, search box, context menus, settings dialog
|
||||
- Double-click the canvas to open the node search box
|
||||
- Right-click nodes/canvas for context menus
|
||||
- The bottom panel shows job queue and execution logs
|
||||
|
||||
### Test Environment
|
||||
- The seed test uses `comfyPageFixture` which provides a `comfyPage` object with extensive helpers
|
||||
- Workflows (JSON files) are loaded via `comfyPage.workflow.loadWorkflow('name')`
|
||||
- Available workflow assets are in `browser_tests/assets/`
|
||||
- The backend MUST be running with `--multi-user` flag
|
||||
- A Vite dev server runs on `:5173`
|
||||
|
||||
### When Creating Test Plans
|
||||
- Reference specific workflow assets when a scenario needs a starting state
|
||||
- Note that canvas interactions use pixel coordinates — these may vary across environments
|
||||
- Distinguish between "canvas tests" (LiteGraph) and "UI tests" (Vue components)
|
||||
- Include tags in your plans: `@canvas`, `@widget`, `@sidebar`, `@smoke`, `@screenshot`
|
||||
- Reference `browser_tests/fixtures/ComfyPage.ts` for available test helpers
|
||||
174
.claude/skills/codegen-transform/SKILL.md
Normal file
174
.claude/skills/codegen-transform/SKILL.md
Normal file
@@ -0,0 +1,174 @@
|
||||
---
|
||||
name: codegen-transform
|
||||
description: 'Transforms raw Playwright codegen output into ComfyUI convention-compliant tests. Use when: user pastes raw codegen, asks to convert raw Playwright code, refactor recorded tests, or rewrite to project conventions. Triggers on: transform codegen, convert raw test, rewrite to conventions, codegen output, raw playwright.'
|
||||
---
|
||||
|
||||
# Codegen → Convention Transform
|
||||
|
||||
Transform raw Playwright codegen output into tests that follow ComfyUI conventions.
|
||||
|
||||
## When to Use
|
||||
|
||||
- QA tester recorded a test with `pnpm comfy-test record` and wants refinement
|
||||
- Developer pasted raw `npx playwright codegen` output
|
||||
- Agent needs to post-process Playwright test agent output
|
||||
- Reviewing a test that uses raw `page.*` calls instead of fixture helpers
|
||||
|
||||
## Transform Rules
|
||||
|
||||
Apply these replacements in order:
|
||||
|
||||
| Raw codegen | Convention replacement | Why |
|
||||
|---|---|---|
|
||||
| `import { test, expect } from '@playwright/test'` | `import { comfyPageFixture as test, comfyExpect as expect } from '../fixtures/ComfyPage'` | Use custom fixtures with ComfyUI helpers |
|
||||
| `test('test', async ({ page }) =>` | `test('descriptive-name', async ({ comfyPage }) =>` | Use comfyPage fixture, descriptive names |
|
||||
| `await page.goto('http://...')` | **Remove entirely** | Fixture handles navigation automatically |
|
||||
| `page.locator('canvas')` | `comfyPage.canvas` | Pre-configured canvas locator |
|
||||
| `page.waitForTimeout(N)` | `comfyPage.nextFrame()` | Never use arbitrary waits |
|
||||
| `page.getByPlaceholder('Search Nodes...')` | `comfyPage.searchBox.input` | Use search box page object |
|
||||
| `page` (bare reference) | `comfyPage.page` | Access raw page through fixture |
|
||||
| Bare `test(...)` | `test.describe('Feature', { tag: ['@canvas'] }, () => { test(...) })` | All tests need describe + tags |
|
||||
| No cleanup | Add `test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })` | Canvas tests need cleanup |
|
||||
|
||||
## Fixture API Quick Reference
|
||||
|
||||
| Need | Use | Notes |
|
||||
|------|-----|-------|
|
||||
| Canvas element | `comfyPage.canvas` | Pre-configured Locator |
|
||||
| Wait for render | `comfyPage.nextFrame()` | After canvas mutations. NOT needed after `loadWorkflow()` |
|
||||
| Load workflow | `comfyPage.workflow.loadWorkflow('name')` | Assets in `browser_tests/assets/` |
|
||||
| Get node by type | `comfyPage.nodeOps.getNodeRefsByType('KSampler')` | Returns array of NodeReference |
|
||||
| Get node by title | `comfyPage.nodeOps.getNodeRefsByTitle('My Node')` | Returns array of NodeReference |
|
||||
| Search box | `comfyPage.searchBox` | Has `.input`, `.fillAndSelectFirstNode()` |
|
||||
| Settings | `comfyPage.settings.setSetting(key, value)` | Persistent — clean up in afterEach |
|
||||
| Keyboard | `comfyPage.keyboard.press('Delete')` | Focus canvas first |
|
||||
| Drag & drop | `comfyPage.dragDrop.*` | Use `{ steps: 10 }` for reliability |
|
||||
| Context menu | `comfyPage.contextMenu.*` | Right-click interactions |
|
||||
| Toast messages | `comfyPage.toast.*` | Notification assertions |
|
||||
| Subgraph | `comfyPage.subgraph.*` | Subgraph/group node operations |
|
||||
| Vue Nodes | `comfyPage.vueNodes.*` | Requires opt-in: `setSetting('Comfy.VueNodes.Enabled', true)` |
|
||||
| Mouse ops | `comfyPage.page` + `ComfyMouse` | For precise canvas mouse interactions |
|
||||
| Bottom panel | `comfyPage.bottomPanel.*` | Job queue, logs panel |
|
||||
| Commands | `comfyPage.command.*` | Command palette interactions |
|
||||
| Clipboard | `comfyPage.clipboard.*` | Copy/paste operations |
|
||||
|
||||
## Canvas Coordinates → Node References
|
||||
|
||||
Raw codegen records fragile pixel coordinates. Replace with node references when possible:
|
||||
|
||||
```typescript
|
||||
// ❌ Raw codegen — fragile pixel coordinates
|
||||
await page.locator('canvas').click({ position: { x: 423, y: 267 } })
|
||||
|
||||
// ✅ If clicking a specific node
|
||||
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
|
||||
await node.click('title')
|
||||
|
||||
// ✅ If double-clicking canvas to open search
|
||||
await comfyPage.canvas.dblclick({ position: { x: 500, y: 400 } })
|
||||
await comfyPage.searchBox.fillAndSelectFirstNode('KSampler')
|
||||
```
|
||||
|
||||
**When to keep coordinates**: Canvas background clicks (pan, zoom), empty area clicks to deselect. These are inherently position-based.
|
||||
|
||||
## Complete Before/After Example
|
||||
|
||||
### Raw codegen output
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test'
|
||||
|
||||
test('test', async ({ page }) => {
|
||||
await page.goto('http://localhost:5173/')
|
||||
await page.locator('canvas').dblclick({ position: { x: 500, y: 400 } })
|
||||
await page.getByPlaceholder('Search Nodes...').fill('KSampler')
|
||||
await page.getByPlaceholder('Search Nodes...').press('Enter')
|
||||
await page.locator('canvas').click({ position: { x: 600, y: 300 } })
|
||||
await page.waitForTimeout(1000)
|
||||
await page.getByRole('button', { name: 'Queue' }).click()
|
||||
})
|
||||
```
|
||||
|
||||
### Convention-compliant output
|
||||
|
||||
```typescript
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
test.describe('Queue workflow with KSampler', { tag: ['@canvas'] }, () => {
|
||||
test.afterEach(async ({ comfyPage }) => {
|
||||
await comfyPage.canvasOps.resetView()
|
||||
})
|
||||
|
||||
test('should add KSampler node and queue', async ({ comfyPage }) => {
|
||||
// Open search and add KSampler
|
||||
await comfyPage.canvas.dblclick({ position: { x: 500, y: 400 } })
|
||||
await comfyPage.searchBox.fillAndSelectFirstNode('KSampler')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Queue the workflow
|
||||
await comfyPage.menu.topbar.runButton.click()
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
### What changed and why
|
||||
|
||||
1. **Imports**: `@playwright/test` → `../fixtures/ComfyPage` (custom fixtures)
|
||||
2. **Fixture**: `{ page }` → `{ comfyPage }` (access all helpers)
|
||||
3. **goto removed**: Fixture navigates automatically
|
||||
4. **Search box**: Raw locator → `comfyPage.searchBox.fillAndSelectFirstNode()`
|
||||
5. **waitForTimeout**: Replaced with `comfyPage.nextFrame()`
|
||||
6. **Queue button**: Used `comfyPage.menu.topbar.runButton` page object
|
||||
7. **Structure**: Wrapped in `describe` with `@canvas` tag and `afterEach` cleanup
|
||||
8. **Test name**: Generic "test" → descriptive name
|
||||
|
||||
## Decision Guide
|
||||
|
||||
| Question | Answer |
|
||||
|----------|--------|
|
||||
| Canvas or DOM interaction? | Canvas: `comfyPage.nodeOps.*`. DOM: `comfyPage.vueNodes.*` (needs opt-in) |
|
||||
| Need `nextFrame()`? | Yes after canvas mutations. No after `loadWorkflow()`, no after DOM clicks |
|
||||
| Which tag? | `@canvas` for canvas tests, `@widget` for widget tests, `@screenshot` for visual regression |
|
||||
| Need cleanup? | Yes for canvas tests (`resetView`), yes if changing settings (`setSetting` back) |
|
||||
| Keep pixel coords? | Only for empty canvas clicks. Replace with node refs for node interactions |
|
||||
| Use `page` directly? | Only via `comfyPage.page` for Playwright APIs not wrapped by fixtures |
|
||||
|
||||
## Tags Reference
|
||||
|
||||
| Tag | When to use |
|
||||
|-----|-------------|
|
||||
| `@canvas` | Any test interacting with the canvas |
|
||||
| `@widget` | Testing widget inputs |
|
||||
| `@smoke` | Quick essential tests |
|
||||
| `@screenshot` | Visual regression (Linux CI only) |
|
||||
| `@mobile` | Mobile viewport (runs on Pixel 5) |
|
||||
| `@2x` | HiDPI tests (2x scale) |
|
||||
| `@0.5x` | Low-DPI tests (0.5x scale) |
|
||||
| `@slow` | Tests taking >10 seconds |
|
||||
| `@perf` | Performance measurement tests |
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
1. **Never use `waitForTimeout`** → use `nextFrame()` or retrying assertions
|
||||
2. **Never use `page.goto`** → fixture handles navigation
|
||||
3. **Never import from `@playwright/test`** → use `../fixtures/ComfyPage`
|
||||
4. **Never use bare CSS selectors** → use test IDs or semantic locators
|
||||
5. **Never share state between tests** → each test is independent
|
||||
6. **Never commit local screenshots** → Linux CI generates baselines
|
||||
|
||||
## For Deeper Reference
|
||||
|
||||
Read fixture code directly — it's the source of truth:
|
||||
|
||||
| Purpose | Path |
|
||||
|---------|------|
|
||||
| Main fixture | `browser_tests/fixtures/ComfyPage.ts` |
|
||||
| Helper classes | `browser_tests/fixtures/helpers/` |
|
||||
| Component objects | `browser_tests/fixtures/components/` |
|
||||
| Test selectors | `browser_tests/fixtures/selectors.ts` |
|
||||
| Vue Node helpers | `browser_tests/fixtures/VueNodeHelpers.ts` |
|
||||
| Existing tests | `browser_tests/tests/` |
|
||||
| Test assets | `browser_tests/assets/` |
|
||||
12
.mcp.json
Normal file
12
.mcp.json
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"playwright-test": {
|
||||
"command": "pnpm",
|
||||
"args": [
|
||||
"exec",
|
||||
"playwright",
|
||||
"run-test-mcp-server"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -10,6 +10,8 @@
|
||||
"dist/*",
|
||||
"packages/registry-types/src/comfyRegistryTypes.ts",
|
||||
"playwright-report/*",
|
||||
"scripts/patch-playwright-agents.js",
|
||||
"tools/test-recorder/*",
|
||||
"src/extensions/core/*",
|
||||
"src/scripts/*",
|
||||
"src/types/generatedManagerTypes.ts",
|
||||
|
||||
@@ -49,6 +49,10 @@ This project uses **pnpm**. Always prefer scripts defined in `package.json` (e.g
|
||||
- `pnpm preview`: Preview the production build locally
|
||||
- `pnpm test:unit`: Run Vitest unit tests
|
||||
- `pnpm test:browser:local`: Run Playwright E2E tests (`browser_tests/`)
|
||||
- `pnpm comfy-test record`: Interactive test recorder (guided setup for non-devs)
|
||||
- `pnpm comfy-test transform <file>`: Transform raw codegen to conventions
|
||||
- `pnpm comfy-test check`: Check environment prerequisites
|
||||
- `pnpm comfy-test list`: List available test workflows
|
||||
- `pnpm lint` / `pnpm lint:fix`: Lint (ESLint)
|
||||
- `pnpm format` / `pnpm format:check`: oxfmt
|
||||
- `pnpm typecheck`: Vue TSC type checking
|
||||
|
||||
@@ -41,3 +41,49 @@ browser_tests/
|
||||
A Playwright test-writing skill exists at `.claude/skills/writing-playwright-tests/SKILL.md`.
|
||||
|
||||
The skill documents **meta-level guidance only** (gotchas, anti-patterns, decision guides). It does **not** duplicate fixture APIs - agents should read the fixture code directly in `browser_tests/fixtures/`.
|
||||
|
||||
## AI-Assisted Test Creation
|
||||
|
||||
Three systems work together for test authoring:
|
||||
|
||||
### 1. Interactive Recorder CLI (`comfy-test`)
|
||||
|
||||
For QA testers and non-developers. Guides through the full flow:
|
||||
|
||||
```bash
|
||||
pnpm comfy-test record # Interactive 7-step recording flow
|
||||
pnpm comfy-test transform # Transform raw codegen to conventions
|
||||
pnpm comfy-test check # Verify environment prerequisites
|
||||
pnpm comfy-test list # List available workflow assets
|
||||
```
|
||||
|
||||
Source: `tools/test-recorder/`
|
||||
|
||||
### 2. Codegen Transform Skill
|
||||
|
||||
For AI agents transforming raw Playwright codegen output. See `.claude/skills/codegen-transform/SKILL.md`.
|
||||
|
||||
Key transforms:
|
||||
- `@playwright/test` → `../fixtures/ComfyPage` imports
|
||||
- `page` destructure → `comfyPage` fixture
|
||||
- `page.goto()` → removed (fixture handles navigation)
|
||||
- `page.locator('canvas')` → `comfyPage.canvas`
|
||||
- `waitForTimeout()` → `comfyPage.nextFrame()`
|
||||
- Wraps in `test.describe` with tags and `afterEach` cleanup
|
||||
|
||||
### 3. Playwright AI Agents
|
||||
|
||||
Three agents in `.claude/agents/` are patched with ComfyUI context:
|
||||
|
||||
- **planner** — explores the app and creates test plans in `browser_tests/specs/`
|
||||
- **generator** — converts test plans into executable `.spec.ts` files
|
||||
- **healer** — debugs and fixes failing tests
|
||||
|
||||
To regenerate after Playwright updates: `bash scripts/update-playwright-agents.sh`
|
||||
|
||||
### MCP Server
|
||||
|
||||
The `.mcp.json` configures `playwright-test` MCP server for agent browser interaction:
|
||||
```bash
|
||||
pnpm exec playwright run-test-mcp-server
|
||||
```
|
||||
|
||||
@@ -70,6 +70,30 @@ await comfyPage.setup({ mockReleases: false })
|
||||
|
||||
For tests that specifically need to test release functionality, see the example in `tests/releaseNotifications.spec.ts`.
|
||||
|
||||
## Recording Tests (For Non-Developers)
|
||||
|
||||
If you're a QA tester or non-developer, use the interactive recorder:
|
||||
|
||||
```bash
|
||||
pnpm comfy-test record
|
||||
```
|
||||
|
||||
This guides you through a 7-step flow:
|
||||
1. **Environment check** — verifies all tools are installed (with install instructions if not)
|
||||
2. **Project setup** — installs dependencies
|
||||
3. **Backend check** — ensures ComfyUI is running
|
||||
4. **Configure** — set test name, tags, and starting workflow
|
||||
5. **Record** — opens browser with Playwright Inspector for recording
|
||||
6. **Transform** — paste recorded code, auto-transforms to project conventions
|
||||
7. **PR creation** — creates a PR via `gh` CLI or gives manual instructions
|
||||
|
||||
Other commands:
|
||||
```bash
|
||||
pnpm comfy-test check # Just run environment checks
|
||||
pnpm comfy-test transform <file> # Transform a raw codegen file
|
||||
pnpm comfy-test list # List available workflow assets
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
**Always use UI mode for development:**
|
||||
|
||||
3
browser_tests/specs/README.md
Normal file
3
browser_tests/specs/README.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Specs
|
||||
|
||||
This is a directory for test plans.
|
||||
26
browser_tests/tests/seed.spec.ts
Normal file
26
browser_tests/tests/seed.spec.ts
Normal file
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Seed test for Playwright AI agents.
|
||||
*
|
||||
* This test bootstraps the ComfyUI environment for agent exploration.
|
||||
* When agents (Planner, Generator, Healer) run, they execute this test
|
||||
* first to set up the browser state, then use it as a template for
|
||||
* generated tests.
|
||||
*
|
||||
* Usage:
|
||||
* - Planner: Runs this to explore the app, then generates a test plan
|
||||
* - Generator: Uses this as an import/fixture template
|
||||
* - Healer: Runs this to establish baseline state
|
||||
*/
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
test('seed', async ({ comfyPage }) => {
|
||||
// Load the default workflow — gives agents a realistic starting state
|
||||
await comfyPage.workflow.loadWorkflow('default')
|
||||
await comfyPage.nextFrame()
|
||||
|
||||
// Verify the app is ready
|
||||
await expect(comfyPage.canvas).toBeVisible()
|
||||
})
|
||||
@@ -26,6 +26,9 @@ const config: KnipConfig = {
|
||||
},
|
||||
'packages/registry-types': {
|
||||
project: ['src/**/*.{js,ts}']
|
||||
},
|
||||
'tools/test-recorder': {
|
||||
project: ['src/**/*.ts']
|
||||
}
|
||||
},
|
||||
ignoreBinaries: ['python3', 'gh'],
|
||||
|
||||
@@ -43,6 +43,7 @@
|
||||
"storybook:desktop": "nx run @comfyorg/desktop-ui:storybook",
|
||||
"stylelint:fix": "stylelint --cache --fix '{apps,packages,src}/**/*.{css,vue}'",
|
||||
"stylelint": "stylelint --cache '{apps,packages,src}/**/*.{css,vue}'",
|
||||
"comfy-test": "tsx tools/test-recorder/src/index.ts",
|
||||
"test:browser": "pnpm exec nx e2e",
|
||||
"test:browser:local": "cross-env PLAYWRIGHT_LOCAL=1 pnpm test:browser",
|
||||
"test:unit": "nx run test",
|
||||
|
||||
104
pnpm-lock.yaml
generated
104
pnpm-lock.yaml
generated
@@ -910,6 +910,22 @@ importers:
|
||||
specifier: 'catalog:'
|
||||
version: 5.9.3
|
||||
|
||||
tools/test-recorder:
|
||||
dependencies:
|
||||
'@clack/prompts':
|
||||
specifier: ^0.9.1
|
||||
version: 0.9.1
|
||||
picocolors:
|
||||
specifier: 'catalog:'
|
||||
version: 1.1.1
|
||||
devDependencies:
|
||||
'@types/node':
|
||||
specifier: 'catalog:'
|
||||
version: 24.10.4
|
||||
typescript:
|
||||
specifier: 'catalog:'
|
||||
version: 5.9.3
|
||||
|
||||
packages:
|
||||
|
||||
'@acemir/cssom@0.9.30':
|
||||
@@ -1549,6 +1565,12 @@ packages:
|
||||
'@cacheable/utils@2.3.2':
|
||||
resolution: {integrity: sha512-8kGE2P+HjfY8FglaOiW+y8qxcaQAfAhVML+i66XJR3YX5FtyDqn6Txctr3K2FrbxLKixRRYYBWMbuGciOhYNDg==}
|
||||
|
||||
'@clack/core@0.4.1':
|
||||
resolution: {integrity: sha512-Pxhij4UXg8KSr7rPek6Zowm+5M22rbd2g1nfojHJkxp5YkFqiZ2+YLEM/XGVIzvGOcM0nqjIFxrpDwWRZYWYjA==}
|
||||
|
||||
'@clack/prompts@0.9.1':
|
||||
resolution: {integrity: sha512-JIpyaboYZeWYlyP0H+OoPPxd6nqueG/CmN6ixBiNFsIDHREevjIf0n0Ohh5gr5C8pEDknzgvz+pIJ8dMhzWIeg==}
|
||||
|
||||
'@comfyorg/comfyui-electron-types@0.6.2':
|
||||
resolution: {integrity: sha512-r3By5Wbizq8jagUrhtcym79HYUTinsvoBnYkFFWbUmrURBWIaC0HduFVkRkI1PNdI76piW+JSOJJnw00YCVXeg==}
|
||||
|
||||
@@ -2457,21 +2479,25 @@ packages:
|
||||
resolution: {integrity: sha512-D+tPXB0tkSuDPsuXvyQIsF3f3PBWfAwIe9FkBWtVoDVYqE+jbz+tVGsjQMNWGafLE4sC8ZQdjhsxyT8I53Anbw==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@nx/nx-linux-arm64-musl@22.5.2':
|
||||
resolution: {integrity: sha512-UbO527qqa8KLBi13uXto5SmxcZv1Smer7sPexJonshDlmrJsyvx5m8nm6tcSv04W5yQEL90vPlTux8dNvEDWrw==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@nx/nx-linux-x64-gnu@22.5.2':
|
||||
resolution: {integrity: sha512-wR6596Vr/Z+blUAmjLHG2TCQMs4O1oi9JXK1J/PoPeO9UqdHwStCJBAd61zDFSUYJe0x+dkeRQu96fE5BW8Kcg==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@nx/nx-linux-x64-musl@22.5.2':
|
||||
resolution: {integrity: sha512-MBXOw4AH4FWl4orwVykj/e75awTNDePogrl3pXNX9NcQLdj6JzS4e2jaALQeRBQLxQzeFvFQV/W4PBzoPV6/NA==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@nx/nx-win32-arm64-msvc@22.5.2':
|
||||
resolution: {integrity: sha512-SaWSZkRH5uV8vP2lj6RRv+kw2IzaIDXkutReOXpooshIWZl9KjrQELNTCZTYyhLDsMlcyhSvLFlTiA4NkZ8udw==}
|
||||
@@ -2637,41 +2663,49 @@ packages:
|
||||
resolution: {integrity: sha512-SVjjjtMW66Mza76PBGJLqB0KKyFTBnxmtDXLJPbL6ZPGSctcXVmujz7/WAc0rb9m2oV0cHQTtVjnq6orQnI/jg==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxc-resolver/binding-linux-arm64-musl@11.15.0':
|
||||
resolution: {integrity: sha512-JDv2/AycPF2qgzEiDeMJCcSzKNDm3KxNg0KKWipoKEMDFqfM7LxNwwSVyAOGmrYlE4l3dg290hOMsr9xG7jv9g==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxc-resolver/binding-linux-ppc64-gnu@11.15.0':
|
||||
resolution: {integrity: sha512-zbu9FhvBLW4KJxo7ElFvZWbSt4vP685Qc/Gyk/Ns3g2gR9qh2qWXouH8PWySy+Ko/qJ42+HJCLg+ZNcxikERfg==}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxc-resolver/binding-linux-riscv64-gnu@11.15.0':
|
||||
resolution: {integrity: sha512-Kfleehe6B09C2qCnyIU01xLFqFXCHI4ylzkicfX/89j+gNHh9xyNdpEvit88Kq6i5tTGdavVnM6DQfOE2qNtlg==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxc-resolver/binding-linux-riscv64-musl@11.15.0':
|
||||
resolution: {integrity: sha512-J7LPiEt27Tpm8P+qURDwNc8q45+n+mWgyys4/V6r5A8v5gDentHRGUx3iVk5NxdKhgoGulrzQocPTZVosq25Eg==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxc-resolver/binding-linux-s390x-gnu@11.15.0':
|
||||
resolution: {integrity: sha512-+8/d2tAScPjVJNyqa7GPGnqleTB/XW9dZJQ2D/oIM3wpH3TG+DaFEXBbk4QFJ9K9AUGBhvQvWU2mQyhK/yYn3Q==}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxc-resolver/binding-linux-x64-gnu@11.15.0':
|
||||
resolution: {integrity: sha512-xtvSzH7Nr5MCZI2FKImmOdTl9kzuQ51RPyLh451tvD2qnkg3BaqI9Ox78bTk57YJhlXPuxWSOL5aZhKAc9J6qg==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxc-resolver/binding-linux-x64-musl@11.15.0':
|
||||
resolution: {integrity: sha512-14YL1zuXj06+/tqsuUZuzL0T425WA/I4nSVN1kBXeC5WHxem6lQ+2HGvG+crjeJEqHgZUT62YIgj88W+8E7eyg==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxc-resolver/binding-openharmony-arm64@11.15.0':
|
||||
resolution: {integrity: sha512-/7Qli+1Wk93coxnrQaU8ySlICYN8HsgyIrzqjgIkQEpI//9eUeaeIHZptNl2fMvBGeXa7k2QgLbRNaBRgpnvMw==}
|
||||
@@ -2745,48 +2779,56 @@ packages:
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxfmt/binding-linux-arm64-musl@0.34.0':
|
||||
resolution: {integrity: sha512-H+F8+71gHQoGTFPPJ6z4dD0Fzfzi0UP8Zx94h5kUmIFThLvMq5K1Y/bUUubiXwwHfwb5C3MPjUpYijiy0rj51Q==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxfmt/binding-linux-ppc64-gnu@0.34.0':
|
||||
resolution: {integrity: sha512-dIGnzTNhCXqQD5pzBwduLg8pClm+t8R53qaE9i5h8iua1iaFAJyLffh4847CNZSlASb7gn1Ofuv7KoG/EpoGZg==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxfmt/binding-linux-riscv64-gnu@0.34.0':
|
||||
resolution: {integrity: sha512-FGQ2GTTooilDte/ogwWwkHuuL3lGtcE3uKM2EcC7kOXNWdUfMY6Jx3JCodNVVbFoybv4A+HuCj8WJji2uu1Ceg==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxfmt/binding-linux-riscv64-musl@0.34.0':
|
||||
resolution: {integrity: sha512-2dGbGneJ7ptOIVKMwEIHdCkdZEomh74X3ggo4hCzEXL/rl9HwfsZDR15MkqfQqAs6nVXMvtGIOMxjDYa5lwKaA==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxfmt/binding-linux-s390x-gnu@0.34.0':
|
||||
resolution: {integrity: sha512-cCtGgmrTrxq3OeSG0UAO+w6yLZTMeOF4XM9SAkNrRUxYhRQELSDQ/iNPCLyHhYNi38uHJQbS5RQweLUDpI4ajA==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxfmt/binding-linux-x64-gnu@0.34.0':
|
||||
resolution: {integrity: sha512-7AvMzmeX+k7GdgitXp99GQoIV/QZIpAS7rwxQvC/T541yWC45nwvk4mpnU8N+V6dE5SPEObnqfhCjO80s7qIsg==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxfmt/binding-linux-x64-musl@0.34.0':
|
||||
resolution: {integrity: sha512-uNiglhcmivJo1oDMh3hoN/Z0WsbEXOpRXZdQ3W/IkOpyV8WF308jFjSC1ZxajdcNRXWej0zgge9QXba58Owt+g==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxfmt/binding-openharmony-arm64@0.34.0':
|
||||
resolution: {integrity: sha512-5eFsTjCyji25j6zznzlMc+wQAZJoL9oWy576xhqd2efv+N4g1swIzuSDcb1dz4gpcVC6veWe9pAwD7HnrGjLwg==}
|
||||
@@ -2889,48 +2931,56 @@ packages:
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxlint/binding-linux-arm64-musl@1.49.0':
|
||||
resolution: {integrity: sha512-xeqkMOARgGBlEg9BQuPDf6ZW711X6BT5qjDyeM5XNowCJeTSdmMhpePJjTEiVbbr3t21sIlK8RE6X5bc04nWyQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxlint/binding-linux-ppc64-gnu@1.49.0':
|
||||
resolution: {integrity: sha512-uvcqRO6PnlJGbL7TeePhTK5+7/JXbxGbN+C6FVmfICDeeRomgQqrfVjf0lUrVpUU8ii8TSkIbNdft3M+oNlOsQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxlint/binding-linux-riscv64-gnu@1.49.0':
|
||||
resolution: {integrity: sha512-Dw1HkdXAwHNH+ZDserHP2RzXQmhHtpsYYI0hf8fuGAVCIVwvS6w1+InLxpPMY25P8ASRNiFN3hADtoh6lI+4lg==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxlint/binding-linux-riscv64-musl@1.49.0':
|
||||
resolution: {integrity: sha512-EPlMYaA05tJ9km/0dI9K57iuMq3Tw+nHst7TNIegAJZrBPtsOtYaMFZEaWj02HA8FI5QvSnRHMt+CI+RIhXJBQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxlint/binding-linux-s390x-gnu@1.49.0':
|
||||
resolution: {integrity: sha512-yZiQL9qEwse34aMbnMb5VqiAWfDY+fLFuoJbHOuzB1OaJZbN1MRF9Nk+W89PIpGr5DNPDipwjZb8+Q7wOywoUQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxlint/binding-linux-x64-gnu@1.49.0':
|
||||
resolution: {integrity: sha512-CcCDwMMXSchNkhdgvhVn3DLZ4EnBXAD8o8+gRzahg+IdSt/72y19xBgShJgadIRF0TsRcV/MhDUMwL5N/W54aQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@oxlint/binding-linux-x64-musl@1.49.0':
|
||||
resolution: {integrity: sha512-u3HfKV8BV6t6UCCbN0RRiyqcymhrnpunVmLFI8sEa5S/EBu+p/0bJ3D7LZ2KT6PsBbrB71SWq4DeFrskOVgIZg==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@oxlint/binding-openharmony-arm64@1.49.0':
|
||||
resolution: {integrity: sha512-dRDpH9fw+oeUMpM4br0taYCFpW6jQtOuEIec89rOgDA1YhqwmeRcx0XYeCv7U48p57qJ1XZHeMGM9LdItIjfzA==}
|
||||
@@ -3099,24 +3149,28 @@ packages:
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rolldown/binding-linux-arm64-musl@1.0.0-rc.3':
|
||||
resolution: {integrity: sha512-Z03/wrqau9Bicfgb3Dbs6SYTHliELk2PM2LpG2nFd+cGupTMF5kanLEcj2vuuJLLhptNyS61rtk7SOZ+lPsTUA==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rolldown/binding-linux-x64-gnu@1.0.0-rc.3':
|
||||
resolution: {integrity: sha512-iSXXZsQp08CSilff/DCTFZHSVEpEwdicV3W8idHyrByrcsRDVh9sGC3sev6d8BygSGj3vt8GvUKBPCoyMA4tgQ==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rolldown/binding-linux-x64-musl@1.0.0-rc.3':
|
||||
resolution: {integrity: sha512-qaj+MFudtdCv9xZo9znFvkgoajLdc+vwf0Kz5N44g+LU5XMe+IsACgn3UG7uTRlCCvhMAGXm1XlpEA5bZBrOcw==}
|
||||
engines: {node: ^20.19.0 || >=22.12.0}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rolldown/binding-openharmony-arm64@1.0.0-rc.3':
|
||||
resolution: {integrity: sha512-U662UnMETyjT65gFmG9ma+XziENrs7BBnENi/27swZPYagubfHRirXHG2oMl+pEax2WvO7Kb9gHZmMakpYqBHQ==}
|
||||
@@ -3194,56 +3248,67 @@ packages:
|
||||
resolution: {integrity: sha512-dV3T9MyAf0w8zPVLVBptVlzaXxka6xg1f16VAQmjg+4KMSTWDvhimI/Y6mp8oHwNrmnmVl9XxJ/w/mO4uIQONA==}
|
||||
cpu: [arm]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-arm-musleabihf@4.53.5':
|
||||
resolution: {integrity: sha512-wIGYC1x/hyjP+KAu9+ewDI+fi5XSNiUi9Bvg6KGAh2TsNMA3tSEs+Sh6jJ/r4BV/bx/CyWu2ue9kDnIdRyafcQ==}
|
||||
cpu: [arm]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rollup/rollup-linux-arm64-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-Y+qVA0D9d0y2FRNiG9oM3Hut/DgODZbU9I8pLLPwAsU0tUKZ49cyV1tzmB/qRbSzGvY8lpgGkJuMyuhH7Ma+Vg==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-arm64-musl@4.53.5':
|
||||
resolution: {integrity: sha512-juaC4bEgJsyFVfqhtGLz8mbopaWD+WeSOYr5E16y+1of6KQjc0BpwZLuxkClqY1i8sco+MdyoXPNiCkQou09+g==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rollup/rollup-linux-loong64-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-rIEC0hZ17A42iXtHX+EPJVL/CakHo+tT7W0pbzdAGuWOt2jxDFh7A/lRhsNHBcqL4T36+UiAgwO8pbmn3dE8wA==}
|
||||
cpu: [loong64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-ppc64-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-T7l409NhUE552RcAOcmJHj3xyZ2h7vMWzcwQI0hvn5tqHh3oSoclf9WgTl+0QqffWFG8MEVZZP1/OBglKZx52Q==}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-riscv64-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-7OK5/GhxbnrMcxIFoYfhV/TkknarkYC1hqUw1wU2xUN3TVRLNT5FmBv4KkheSG2xZ6IEbRAhTooTV2+R5Tk0lQ==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-riscv64-musl@4.53.5':
|
||||
resolution: {integrity: sha512-GwuDBE/PsXaTa76lO5eLJTyr2k8QkPipAyOrs4V/KJufHCZBJ495VCGJol35grx9xryk4V+2zd3Ri+3v7NPh+w==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rollup/rollup-linux-s390x-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-IAE1Ziyr1qNfnmiQLHBURAD+eh/zH1pIeJjeShleII7Vj8kyEm2PF77o+lf3WTHDpNJcu4IXJxNO0Zluro8bOw==}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-x64-gnu@4.53.5':
|
||||
resolution: {integrity: sha512-Pg6E+oP7GvZ4XwgRJBuSXZjcqpIW3yCBhK4BcsANvb47qMvAbCjR6E+1a/U2WXz1JJxp9/4Dno3/iSJLcm5auw==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@rollup/rollup-linux-x64-musl@4.53.5':
|
||||
resolution: {integrity: sha512-txGtluxDKTxaMDzUduGP0wdfng24y1rygUMnmlUJ88fzCCULCLn7oE5kb2+tRB+MWq1QDZT6ObT5RrR8HFRKqg==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@rollup/rollup-openharmony-arm64@4.53.5':
|
||||
resolution: {integrity: sha512-3DFiLPnTxiOQV993fMc+KO8zXHTcIjgaInrqlG8zDp1TlhYl6WgrOHuJkJQ6M8zHEcntSJsUp1XFZSY8C1DYbg==}
|
||||
@@ -3519,24 +3584,28 @@ packages:
|
||||
engines: {node: '>= 20'}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@tailwindcss/oxide-linux-arm64-musl@4.2.0':
|
||||
resolution: {integrity: sha512-XKcSStleEVnbH6W/9DHzZv1YhjE4eSS6zOu2eRtYAIh7aV4o3vIBs+t/B15xlqoxt6ef/0uiqJVB6hkHjWD/0A==}
|
||||
engines: {node: '>= 20'}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@tailwindcss/oxide-linux-x64-gnu@4.2.0':
|
||||
resolution: {integrity: sha512-/hlXCBqn9K6fi7eAM0RsobHwJYa5V/xzWspVTzxnX+Ft9v6n+30Pz8+RxCn7sQL/vRHHLS30iQPrHQunu6/vJA==}
|
||||
engines: {node: '>= 20'}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@tailwindcss/oxide-linux-x64-musl@4.2.0':
|
||||
resolution: {integrity: sha512-lKUaygq4G7sWkhQbfdRRBkaq4LY39IriqBQ+Gk6l5nKq6Ay2M2ZZb1tlIyRNgZKS8cbErTwuYSor0IIULC0SHw==}
|
||||
engines: {node: '>= 20'}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@tailwindcss/oxide-wasm32-wasi@4.2.0':
|
||||
resolution: {integrity: sha512-xuDjhAsFdUuFP5W9Ze4k/o4AskUtI8bcAGU4puTYprr89QaYFmhYOPfP+d1pH+k9ets6RoE23BXZM1X1jJqoyw==}
|
||||
@@ -4012,41 +4081,49 @@ packages:
|
||||
resolution: {integrity: sha512-34gw7PjDGB9JgePJEmhEqBhWvCiiWCuXsL9hYphDF7crW7UgI05gyBAi6MF58uGcMOiOqSJ2ybEeCvHcq0BCmQ==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@unrs/resolver-binding-linux-arm64-musl@1.11.1':
|
||||
resolution: {integrity: sha512-RyMIx6Uf53hhOtJDIamSbTskA99sPHS96wxVE/bJtePJJtpdKGXO1wY90oRdXuYOGOTuqjT8ACccMc4K6QmT3w==}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@unrs/resolver-binding-linux-ppc64-gnu@1.11.1':
|
||||
resolution: {integrity: sha512-D8Vae74A4/a+mZH0FbOkFJL9DSK2R6TFPC9M+jCWYia/q2einCubX10pecpDiTmkJVUH+y8K3BZClycD8nCShA==}
|
||||
cpu: [ppc64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@unrs/resolver-binding-linux-riscv64-gnu@1.11.1':
|
||||
resolution: {integrity: sha512-frxL4OrzOWVVsOc96+V3aqTIQl1O2TjgExV4EKgRY09AJ9leZpEg8Ak9phadbuX0BA4k8U5qtvMSQQGGmaJqcQ==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@unrs/resolver-binding-linux-riscv64-musl@1.11.1':
|
||||
resolution: {integrity: sha512-mJ5vuDaIZ+l/acv01sHoXfpnyrNKOk/3aDoEdLO/Xtn9HuZlDD6jKxHlkN8ZhWyLJsRBxfv9GYM2utQ1SChKew==}
|
||||
cpu: [riscv64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@unrs/resolver-binding-linux-s390x-gnu@1.11.1':
|
||||
resolution: {integrity: sha512-kELo8ebBVtb9sA7rMe1Cph4QHreByhaZ2QEADd9NzIQsYNQpt9UkM9iqr2lhGr5afh885d/cB5QeTXSbZHTYPg==}
|
||||
cpu: [s390x]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@unrs/resolver-binding-linux-x64-gnu@1.11.1':
|
||||
resolution: {integrity: sha512-C3ZAHugKgovV5YvAMsxhq0gtXuwESUKc5MhEtjBpLoHPLYM+iuwSj3lflFwK3DPm68660rZ7G8BMcwSro7hD5w==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
'@unrs/resolver-binding-linux-x64-musl@1.11.1':
|
||||
resolution: {integrity: sha512-rV0YSoyhK2nZ4vEswT/QwqzqQXw5I6CjoaYMOX0TqBlWhojUf8P94mvI7nuJTeaCkkds3QE4+zS8Ko+GdXuZtA==}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
'@unrs/resolver-binding-wasm32-wasi@1.11.1':
|
||||
resolution: {integrity: sha512-5u4RkfxJm+Ng7IWgkzi3qrFOvLvQYnPBmjmZQ8+szTK/b31fQCnleNl1GgEt7nIsZRIf5PLhPwT0WM+q45x/UQ==}
|
||||
@@ -6426,24 +6503,28 @@ packages:
|
||||
engines: {node: '>= 12.0.0'}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
lightningcss-linux-arm64-musl@1.31.1:
|
||||
resolution: {integrity: sha512-mVZ7Pg2zIbe3XlNbZJdjs86YViQFoJSpc41CbVmKBPiGmC4YrfeOyz65ms2qpAobVd7WQsbW4PdsSJEMymyIMg==}
|
||||
engines: {node: '>= 12.0.0'}
|
||||
cpu: [arm64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
lightningcss-linux-x64-gnu@1.31.1:
|
||||
resolution: {integrity: sha512-xGlFWRMl+0KvUhgySdIaReQdB4FNudfUTARn7q0hh/V67PVGCs3ADFjw+6++kG1RNd0zdGRlEKa+T13/tQjPMA==}
|
||||
engines: {node: '>= 12.0.0'}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [glibc]
|
||||
|
||||
lightningcss-linux-x64-musl@1.31.1:
|
||||
resolution: {integrity: sha512-eowF8PrKHw9LpoZii5tdZwnBcYDxRw2rRCyvAXLi34iyeYfqCQNA9rmUM0ce62NlPhCvof1+9ivRaTY6pSKDaA==}
|
||||
engines: {node: '>= 12.0.0'}
|
||||
cpu: [x64]
|
||||
os: [linux]
|
||||
libc: [musl]
|
||||
|
||||
lightningcss-win32-arm64-msvc@1.31.1:
|
||||
resolution: {integrity: sha512-aJReEbSEQzx1uBlQizAOBSjcmr9dCdL3XuC/6HLXAxmtErsj2ICo5yYggg1qOODQMtnjNQv2UHb9NpOuFtYe4w==}
|
||||
@@ -7669,6 +7750,9 @@ packages:
|
||||
resolution: {integrity: sha512-2wcC/oGxHis/BoHkkPwldgiPSYcpZK3JU28WoMVv55yHJgcZ8rlXvuG9iZggz+sU1d4bRgIGASwyWqjxu3FM0g==}
|
||||
engines: {node: '>=18'}
|
||||
|
||||
sisteransi@1.0.5:
|
||||
resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==}
|
||||
|
||||
slash@3.0.0:
|
||||
resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==}
|
||||
engines: {node: '>=8'}
|
||||
@@ -8402,6 +8486,9 @@ packages:
|
||||
vue-component-type-helpers@3.2.5:
|
||||
resolution: {integrity: sha512-tkvNr+bU8+xD/onAThIe7CHFvOJ/BO6XCOrxMzeytJq40nTfpGDJuVjyCM8ccGZKfAbGk2YfuZyDMXM56qheZQ==}
|
||||
|
||||
vue-component-type-helpers@3.2.6:
|
||||
resolution: {integrity: sha512-O02tnvIfOQVmnvoWwuSydwRoHjZVt8UEBR+2p4rT35p8GAy5VTlWP8o5qXfJR/GWCN0nVZoYWsVUvx2jwgdBmQ==}
|
||||
|
||||
vue-demi@0.14.10:
|
||||
resolution: {integrity: sha512-nMZBOwuzabUO0nLgIcc6rycZEebF6eeUfaiQx9+WSk8e29IbLvPU9feI6tqW4kTo3hvoYAJkMh8n8D0fuISphg==}
|
||||
engines: {node: '>=12'}
|
||||
@@ -9585,6 +9672,17 @@ snapshots:
|
||||
hashery: 1.3.0
|
||||
keyv: 5.5.5
|
||||
|
||||
'@clack/core@0.4.1':
|
||||
dependencies:
|
||||
picocolors: 1.1.1
|
||||
sisteransi: 1.0.5
|
||||
|
||||
'@clack/prompts@0.9.1':
|
||||
dependencies:
|
||||
'@clack/core': 0.4.1
|
||||
picocolors: 1.1.1
|
||||
sisteransi: 1.0.5
|
||||
|
||||
'@comfyorg/comfyui-electron-types@0.6.2': {}
|
||||
|
||||
'@csstools/color-helpers@5.1.0': {}
|
||||
@@ -11485,7 +11583,7 @@ snapshots:
|
||||
storybook: 10.2.10(@testing-library/dom@10.4.1)(prettier@3.7.4)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
|
||||
type-fest: 2.19.0
|
||||
vue: 3.5.13(typescript@5.9.3)
|
||||
vue-component-type-helpers: 3.2.5
|
||||
vue-component-type-helpers: 3.2.6
|
||||
|
||||
'@swc/helpers@0.5.17':
|
||||
dependencies:
|
||||
@@ -16547,6 +16645,8 @@ snapshots:
|
||||
mrmime: 2.0.1
|
||||
totalist: 3.0.1
|
||||
|
||||
sisteransi@1.0.5: {}
|
||||
|
||||
slash@3.0.0: {}
|
||||
|
||||
slice-ansi@4.0.0:
|
||||
@@ -17500,6 +17600,8 @@ snapshots:
|
||||
|
||||
vue-component-type-helpers@3.2.5: {}
|
||||
|
||||
vue-component-type-helpers@3.2.6: {}
|
||||
|
||||
vue-demi@0.14.10(vue@3.5.13(typescript@5.9.3)):
|
||||
dependencies:
|
||||
vue: 3.5.13(typescript@5.9.3)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
packages:
|
||||
- apps/**
|
||||
- packages/**
|
||||
- tools/**
|
||||
|
||||
catalog:
|
||||
'@alloc/quick-lru': ^5.2.0
|
||||
|
||||
154
scripts/patch-playwright-agents.js
Normal file
154
scripts/patch-playwright-agents.js
Normal file
@@ -0,0 +1,154 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { readFileSync, writeFileSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
|
||||
const agentsDir = join(process.cwd(), '.claude', 'agents')
|
||||
|
||||
const patches = {
|
||||
'playwright-test-planner.md': `
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Application Overview
|
||||
ComfyUI is a **canvas-based node graph editor** for AI image generation. It is a complex SPA with:
|
||||
- A **LiteGraph canvas** where users create workflows by connecting nodes
|
||||
- A **Vue 3 sidebar** with node library, workflows panel, and settings
|
||||
- A **topbar** with queue/run buttons and workspace controls
|
||||
- A **search box** for finding and adding nodes (opens on double-click)
|
||||
- WebSocket-based real-time communication with a Python backend
|
||||
|
||||
### Exploration Tips
|
||||
- Start by loading a workflow: the app is most useful with nodes on the canvas
|
||||
- Key UI areas to explore: canvas interactions, sidebar panels, topbar buttons, search box, context menus, settings dialog
|
||||
- Double-click the canvas to open the node search box
|
||||
- Right-click nodes/canvas for context menus
|
||||
- The bottom panel shows job queue and execution logs
|
||||
|
||||
### Test Environment
|
||||
- The seed test uses \`comfyPageFixture\` which provides a \`comfyPage\` object with extensive helpers
|
||||
- Workflows (JSON files) are loaded via \`comfyPage.workflow.loadWorkflow('name')\`
|
||||
- Available workflow assets are in \`browser_tests/assets/\`
|
||||
- The backend MUST be running with \`--multi-user\` flag
|
||||
- A Vite dev server runs on \`:5173\`
|
||||
|
||||
### When Creating Test Plans
|
||||
- Reference specific workflow assets when a scenario needs a starting state
|
||||
- Note that canvas interactions use pixel coordinates — these may vary across environments
|
||||
- Distinguish between "canvas tests" (LiteGraph) and "UI tests" (Vue components)
|
||||
- Include tags in your plans: \`@canvas\`, \`@widget\`, \`@sidebar\`, \`@smoke\`, \`@screenshot\`
|
||||
- Reference \`browser_tests/fixtures/ComfyPage.ts\` for available test helpers`,
|
||||
|
||||
'playwright-test-generator.md': `
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Required Import Pattern
|
||||
Generated tests MUST use ComfyUI fixtures, not generic \`@playwright/test\`:
|
||||
|
||||
\`\`\`typescript
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
\`\`\`
|
||||
|
||||
### Fixture Object
|
||||
Tests receive \`comfyPage\` (not \`page\`) as their fixture:
|
||||
|
||||
\`\`\`typescript
|
||||
test('my test', async ({ comfyPage }) => {
|
||||
// Access raw page via comfyPage.page if needed
|
||||
})
|
||||
\`\`\`
|
||||
|
||||
### Key APIs
|
||||
| Need | Use | Notes |
|
||||
|------|-----|-------|
|
||||
| Canvas element | \`comfyPage.canvas\` | Pre-configured Locator |
|
||||
| Wait for render | \`comfyPage.nextFrame()\` | After canvas mutations |
|
||||
| Load workflow | \`comfyPage.workflow.loadWorkflow('name')\` | Assets in \`browser_tests/assets/\` |
|
||||
| Get node by type | \`comfyPage.nodeOps.getNodeRefsByType('KSampler')\` | Returns NodeReference[] |
|
||||
| Search box | \`comfyPage.searchBox.fillAndSelectFirstNode('name')\` | Opens on canvas dblclick |
|
||||
| Settings | \`comfyPage.settings.setSetting(key, value)\` | Clean up in afterEach |
|
||||
| Keyboard | \`comfyPage.keyboard.press('Delete')\` | Focus canvas first |
|
||||
| Context menu | \`comfyPage.contextMenu\` | Right-click interactions |
|
||||
|
||||
### Mandatory Test Structure
|
||||
Every generated test must:
|
||||
1. Be wrapped in \`test.describe('Name', { tag: ['@canvas'] }, () => { ... })\`
|
||||
2. Include \`test.afterEach(async ({ comfyPage }) => { await comfyPage.canvasOps.resetView() })\`
|
||||
3. Use descriptive test names (not "test" or "test1")
|
||||
|
||||
### Anti-Patterns — NEVER Use
|
||||
- ❌ \`page.goto()\` — fixture handles navigation
|
||||
- ❌ \`page.waitForTimeout()\` — use \`comfyPage.nextFrame()\` or retrying assertions
|
||||
- ❌ \`import from '@playwright/test'\` — use \`from '../fixtures/ComfyPage'\`
|
||||
- ❌ Bare \`page.\` references — use \`comfyPage.page.\` if you need raw page access
|
||||
|
||||
### Reference
|
||||
Read the fixture code for full API surface:
|
||||
- \`browser_tests/fixtures/ComfyPage.ts\` — main fixture
|
||||
- \`browser_tests/fixtures/helpers/\` — helper classes
|
||||
- \`browser_tests/fixtures/components/\` — page object components
|
||||
- See also: \`.claude/skills/codegen-transform/SKILL.md\` for transform rules`,
|
||||
|
||||
'playwright-test-healer.md': `
|
||||
|
||||
## ComfyUI Project Context
|
||||
|
||||
### Custom Fixtures
|
||||
Tests in this project use \`comfyPage\` fixture, not bare \`page\`. When healing:
|
||||
- Replace any \`page.\` references with \`comfyPage.page.\` if adding new code
|
||||
- Use \`comfyPage.nextFrame()\` instead of adding \`waitForTimeout()\`
|
||||
- Use fixture helpers (\`comfyPage.nodeOps\`, \`comfyPage.canvas\`, etc.) over raw locators
|
||||
|
||||
### Common Failure Causes in ComfyUI Tests
|
||||
|
||||
1. **Missing \`nextFrame()\`**: Canvas operations need \`await comfyPage.nextFrame()\` after mutations. This is the #1 cause of "works locally, fails in CI" issues.
|
||||
|
||||
2. **Canvas focus required**: Keyboard shortcuts won't work unless \`await comfyPage.canvas.click()\` is called first.
|
||||
|
||||
3. **Node position drift**: Pixel coordinates can shift between environments. When possible, replace with node references:
|
||||
\`\`\`typescript
|
||||
// Instead of: canvas.click({ position: { x: 423, y: 267 } })
|
||||
const node = (await comfyPage.nodeOps.getNodeRefsByType('KSampler'))[0]
|
||||
await node.click('title')
|
||||
\`\`\`
|
||||
|
||||
4. **Settings pollution**: Settings persist across tests on the backend. Always reset changed settings in \`afterEach\`.
|
||||
|
||||
5. **Drag animation timing**: Use \`{ steps: 10 }\` option for drag operations, not \`{ steps: 1 }\`.
|
||||
|
||||
### Healing Safety Rules
|
||||
- ❌ NEVER add \`waitForTimeout()\` — always use retrying assertions or \`nextFrame()\`
|
||||
- ❌ NEVER "fix" a test by weakening assertions (e.g., removing an assertion that fails)
|
||||
- ❌ NEVER modify the application code — only modify test code
|
||||
- ⚠️ If a test fails because expected UI elements are missing, the app may have a regression — mark as \`test.fixme()\` with explanation, don't "heal" the assertion away
|
||||
- ⚠️ If a test fails only in CI but passes locally, likely missing \`nextFrame()\` — don't mask with timeouts
|
||||
|
||||
### Reference
|
||||
- \`browser_tests/fixtures/ComfyPage.ts\` — full fixture API
|
||||
- \`browser_tests/fixtures/helpers/\` — available helper classes
|
||||
- \`.claude/skills/writing-playwright-tests/SKILL.md\` — testing conventions
|
||||
- \`.claude/skills/codegen-transform/SKILL.md\` — transform rules`
|
||||
}
|
||||
|
||||
const CONTEXT_HEADING = '## ComfyUI Project Context'
|
||||
|
||||
for (const [filename, patch] of Object.entries(patches)) {
|
||||
const filePath = join(agentsDir, filename)
|
||||
let content = readFileSync(filePath, 'utf-8')
|
||||
|
||||
// Strip existing ComfyUI context section (heading to EOF)
|
||||
const idx = content.indexOf(CONTEXT_HEADING)
|
||||
if (idx !== -1) {
|
||||
// Trim trailing whitespace before the heading too
|
||||
content = content.substring(0, idx).trimEnd()
|
||||
console.log(` ♻️ ${filename}: stripped existing ComfyUI context`)
|
||||
}
|
||||
|
||||
content += patch
|
||||
writeFileSync(filePath, content, 'utf-8')
|
||||
console.log(` ✅ ${filename}: patched`)
|
||||
}
|
||||
28
scripts/update-playwright-agents.sh
Executable file
28
scripts/update-playwright-agents.sh
Executable file
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Navigate to repo root (script location relative)
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
echo "🎭 Updating Playwright agent definitions..."
|
||||
echo ""
|
||||
|
||||
# Step 1: Regenerate base agent files
|
||||
echo "Step 1: Running init-agents..."
|
||||
cd "$REPO_ROOT"
|
||||
pnpm exec playwright init-agents --loop=claude
|
||||
|
||||
echo ""
|
||||
echo "Step 2: Applying ComfyUI patches..."
|
||||
node "$SCRIPT_DIR/patch-playwright-agents.js"
|
||||
|
||||
echo ""
|
||||
echo "✅ Agent files updated and patched."
|
||||
echo ""
|
||||
echo "Files modified:"
|
||||
echo " .claude/agents/playwright-test-planner.md"
|
||||
echo " .claude/agents/playwright-test-generator.md"
|
||||
echo " .claude/agents/playwright-test-healer.md"
|
||||
echo ""
|
||||
echo "Review changes with: git diff .claude/agents/"
|
||||
26
tools/test-recorder/README.md
Normal file
26
tools/test-recorder/README.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# @comfyorg/test-recorder
|
||||
|
||||
Interactive CLI for recording and transforming Playwright browser tests for ComfyUI.
|
||||
|
||||
## Usage
|
||||
|
||||
From the repo root:
|
||||
|
||||
```bash
|
||||
pnpm comfy-test record # Record a new test
|
||||
pnpm comfy-test transform # Transform raw codegen to conventions
|
||||
pnpm comfy-test check # Check environment prerequisites
|
||||
pnpm comfy-test list # List available workflows
|
||||
```
|
||||
|
||||
## For QA Testers
|
||||
|
||||
See the [Browser Tests README](../../browser_tests/README.md) for full setup instructions.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
cd tools/test-recorder
|
||||
pnpm build # Compile TypeScript
|
||||
pnpm dev # Watch mode
|
||||
```
|
||||
24
tools/test-recorder/package.json
Normal file
24
tools/test-recorder/package.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "@comfyorg/test-recorder",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"description": "Interactive CLI for recording and transforming Playwright browser tests",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"comfy-test": "./dist/index.js"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"dev": "tsc --watch",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@clack/prompts": "^0.9.1",
|
||||
"picocolors": "catalog:"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "catalog:",
|
||||
"typescript": "catalog:"
|
||||
},
|
||||
"packageManager": "pnpm@10.17.1"
|
||||
}
|
||||
33
tools/test-recorder/src/checks/backend.ts
Normal file
33
tools/test-recorder/src/checks/backend.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkBackend(port = 8188): Promise<CheckResult> {
|
||||
const url = `http://localhost:${port}`
|
||||
try {
|
||||
const res = await fetch(`${url}/system_stats`, {
|
||||
signal: AbortSignal.timeout(3000)
|
||||
})
|
||||
if (res.ok) {
|
||||
pass('ComfyUI backend', url)
|
||||
return { name: 'ComfyUI backend', ok: true, version: url }
|
||||
}
|
||||
throw new Error(`Status ${res.status}`)
|
||||
} catch {
|
||||
fail('ComfyUI backend', `not running on :${port}`)
|
||||
const instructions = [
|
||||
'ComfyUI backend must be running for browser tests.',
|
||||
'',
|
||||
'In a separate terminal, navigate to your ComfyUI folder and run:',
|
||||
'',
|
||||
' python main.py --multi-user',
|
||||
'',
|
||||
'⚠️ The --multi-user flag is REQUIRED for parallel test support.'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'ComfyUI backend',
|
||||
ok: false,
|
||||
installInstructions: instructions
|
||||
}
|
||||
}
|
||||
}
|
||||
25
tools/test-recorder/src/checks/devServer.ts
Normal file
25
tools/test-recorder/src/checks/devServer.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkDevServer(port = 5173): Promise<CheckResult> {
|
||||
const url = `http://localhost:${port}`
|
||||
try {
|
||||
const res = await fetch(url, { signal: AbortSignal.timeout(3000) })
|
||||
if (res.ok || res.status === 304) {
|
||||
pass('Dev server', url)
|
||||
return { name: 'Dev server', ok: true, version: url }
|
||||
}
|
||||
throw new Error(`Status ${res.status}`)
|
||||
} catch {
|
||||
fail('Dev server', `not running on :${port}`)
|
||||
const instructions = [
|
||||
'Start the Vite dev server in another terminal:',
|
||||
'',
|
||||
' pnpm dev',
|
||||
'',
|
||||
`Then wait for it to show "Local: http://localhost:${port}"`
|
||||
]
|
||||
info(instructions)
|
||||
return { name: 'Dev server', ok: false, installInstructions: instructions }
|
||||
}
|
||||
}
|
||||
55
tools/test-recorder/src/checks/gh.ts
Normal file
55
tools/test-recorder/src/checks/gh.ts
Normal file
@@ -0,0 +1,55 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { detectPlatform } from './platform'
|
||||
import { pass, warn, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkGh(): Promise<CheckResult> {
|
||||
try {
|
||||
const version = execSync('gh --version', { encoding: 'utf-8' })
|
||||
.split('\n')[0]
|
||||
.trim()
|
||||
// Check if authenticated
|
||||
try {
|
||||
execSync('gh auth status', { encoding: 'utf-8', stdio: 'pipe' })
|
||||
pass('GitHub CLI (gh)', version.replace('gh version ', ''))
|
||||
return { name: 'GitHub CLI', ok: true, optional: true, version }
|
||||
} catch {
|
||||
warn('GitHub CLI (gh)', 'installed but not authenticated')
|
||||
info([
|
||||
'Run `gh auth login` to authenticate.',
|
||||
"Without auth, we'll help you create PRs manually."
|
||||
])
|
||||
return { name: 'GitHub CLI', ok: false, optional: true }
|
||||
}
|
||||
} catch {
|
||||
warn('GitHub CLI (gh)', 'not installed (optional)')
|
||||
const platform = detectPlatform()
|
||||
const instructions =
|
||||
platform === 'macos'
|
||||
? [
|
||||
"gh CLI lets us auto-create PRs. Without it, we'll help",
|
||||
'you create the PR manually on github.com.',
|
||||
'',
|
||||
'To install (optional):',
|
||||
' brew install gh',
|
||||
' gh auth login'
|
||||
]
|
||||
: platform === 'windows'
|
||||
? [
|
||||
'gh CLI is optional. Install from: https://cli.github.com/',
|
||||
"Without it, we'll help you create PRs via github.com."
|
||||
]
|
||||
: [
|
||||
'gh CLI is optional. Install:',
|
||||
' sudo apt install gh # Debian/Ubuntu',
|
||||
' gh auth login'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'GitHub CLI',
|
||||
ok: false,
|
||||
optional: true,
|
||||
installInstructions: instructions
|
||||
}
|
||||
}
|
||||
}
|
||||
37
tools/test-recorder/src/checks/git.ts
Normal file
37
tools/test-recorder/src/checks/git.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { detectPlatform } from './platform'
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkGit(): Promise<CheckResult> {
|
||||
try {
|
||||
const version = execSync('git --version', { encoding: 'utf-8' }).trim()
|
||||
const v = version.replace('git version ', '')
|
||||
pass('Git', v)
|
||||
return { name: 'Git', ok: true, version: v }
|
||||
} catch {
|
||||
fail('Git', 'not installed')
|
||||
const platform = detectPlatform()
|
||||
const instructions =
|
||||
platform === 'macos'
|
||||
? [
|
||||
'Git is included with Xcode CLI Tools. Install them first:',
|
||||
'',
|
||||
' xcode-select --install'
|
||||
]
|
||||
: platform === 'windows'
|
||||
? [
|
||||
'Download Git from: https://git-scm.com/download/win',
|
||||
'Run the installer with default settings.'
|
||||
]
|
||||
: [
|
||||
'Install git using your package manager:',
|
||||
'',
|
||||
' sudo apt install git # Debian/Ubuntu',
|
||||
' sudo dnf install git # Fedora',
|
||||
' sudo pacman -S git # Arch'
|
||||
]
|
||||
info(instructions)
|
||||
return { name: 'Git', ok: false, installInstructions: instructions }
|
||||
}
|
||||
}
|
||||
59
tools/test-recorder/src/checks/node.ts
Normal file
59
tools/test-recorder/src/checks/node.ts
Normal file
@@ -0,0 +1,59 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { detectPlatform } from './platform'
|
||||
import { pass, fail, warn, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkNode(): Promise<CheckResult> {
|
||||
try {
|
||||
const version = execSync('node --version', { encoding: 'utf-8' }).trim()
|
||||
const major = parseInt(version.replace('v', '').split('.')[0])
|
||||
if (major < 20) {
|
||||
warn('Node.js', `${version} (need v20+)`)
|
||||
const instructions = [
|
||||
`Node.js ${version} is too old. You need v20 or later.`,
|
||||
'',
|
||||
'Update via nvm:',
|
||||
' nvm install 20',
|
||||
' nvm use 20'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'Node.js',
|
||||
ok: false,
|
||||
version,
|
||||
installInstructions: instructions
|
||||
}
|
||||
}
|
||||
pass('Node.js', version)
|
||||
return { name: 'Node.js', ok: true, version }
|
||||
} catch {
|
||||
fail('Node.js', 'not installed')
|
||||
const platform = detectPlatform()
|
||||
const instructions =
|
||||
platform === 'macos'
|
||||
? [
|
||||
'Install Node.js via nvm (recommended):',
|
||||
'',
|
||||
' curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash',
|
||||
'',
|
||||
'Close and reopen Terminal, then run:',
|
||||
'',
|
||||
' nvm install 20'
|
||||
]
|
||||
: platform === 'windows'
|
||||
? [
|
||||
'Download Node.js from: https://nodejs.org/',
|
||||
'Choose the LTS version (v20+).',
|
||||
'Run the installer with default settings.'
|
||||
]
|
||||
: [
|
||||
'Install Node.js via nvm:',
|
||||
'',
|
||||
' curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash',
|
||||
' source ~/.bashrc',
|
||||
' nvm install 20'
|
||||
]
|
||||
info(instructions)
|
||||
return { name: 'Node.js', ok: false, installInstructions: instructions }
|
||||
}
|
||||
}
|
||||
24
tools/test-recorder/src/checks/platform.ts
Normal file
24
tools/test-recorder/src/checks/platform.ts
Normal file
@@ -0,0 +1,24 @@
|
||||
import os from 'node:os'
|
||||
import { pass } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
type Platform = 'macos' | 'windows' | 'linux'
|
||||
|
||||
export function detectPlatform(): Platform {
|
||||
switch (os.platform()) {
|
||||
case 'darwin':
|
||||
return 'macos'
|
||||
case 'win32':
|
||||
return 'windows'
|
||||
default:
|
||||
return 'linux'
|
||||
}
|
||||
}
|
||||
|
||||
export function checkPlatform(): CheckResult {
|
||||
const p = os.platform()
|
||||
const arch = os.arch()
|
||||
const release = os.release()
|
||||
pass('Operating System', `${p} ${release} (${arch})`)
|
||||
return { name: 'Operating System', ok: true, version: `${p} ${arch}` }
|
||||
}
|
||||
51
tools/test-recorder/src/checks/playwright.ts
Normal file
51
tools/test-recorder/src/checks/playwright.ts
Normal file
@@ -0,0 +1,51 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { pass, fail, warn, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkPlaywright(): Promise<CheckResult> {
|
||||
try {
|
||||
// Check if chromium browser is installed
|
||||
execSync('pnpm exec playwright install --dry-run chromium', {
|
||||
encoding: 'utf-8',
|
||||
stdio: 'pipe'
|
||||
})
|
||||
pass('Playwright browsers', 'chromium installed')
|
||||
return { name: 'Playwright browsers', ok: true, version: 'chromium' }
|
||||
} catch {
|
||||
// Browser might not be installed, try to check another way
|
||||
try {
|
||||
const result = execSync('pnpm exec playwright --version', {
|
||||
encoding: 'utf-8',
|
||||
stdio: 'pipe'
|
||||
}).trim()
|
||||
warn('Playwright', `${result} (browsers may need installing)`)
|
||||
const instructions = [
|
||||
'Playwright browsers need to be installed:',
|
||||
'',
|
||||
' pnpm exec playwright install chromium --with-deps',
|
||||
'',
|
||||
'This downloads ~200MB. Please wait...'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'Playwright browsers',
|
||||
ok: false,
|
||||
installInstructions: instructions
|
||||
}
|
||||
} catch {
|
||||
fail('Playwright', 'not installed')
|
||||
const instructions = [
|
||||
'Playwright is a project dependency. Run:',
|
||||
'',
|
||||
' pnpm install',
|
||||
' pnpm exec playwright install chromium --with-deps'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'Playwright',
|
||||
ok: false,
|
||||
installInstructions: instructions
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
25
tools/test-recorder/src/checks/pnpm.ts
Normal file
25
tools/test-recorder/src/checks/pnpm.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkPnpm(): Promise<CheckResult> {
|
||||
try {
|
||||
const version = execSync('pnpm --version', { encoding: 'utf-8' }).trim()
|
||||
pass('pnpm', version)
|
||||
return { name: 'pnpm', ok: true, version }
|
||||
} catch {
|
||||
fail('pnpm', 'not installed')
|
||||
const instructions = [
|
||||
'Install pnpm via corepack (comes with Node.js):',
|
||||
'',
|
||||
' corepack enable',
|
||||
' corepack prepare pnpm@latest --activate',
|
||||
'',
|
||||
'Or install directly:',
|
||||
'',
|
||||
' npm install -g pnpm'
|
||||
]
|
||||
info(instructions)
|
||||
return { name: 'pnpm', ok: false, installInstructions: instructions }
|
||||
}
|
||||
}
|
||||
39
tools/test-recorder/src/checks/python.ts
Normal file
39
tools/test-recorder/src/checks/python.ts
Normal file
@@ -0,0 +1,39 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { detectPlatform } from './platform'
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkPython(): Promise<CheckResult> {
|
||||
for (const cmd of ['python3', 'python']) {
|
||||
try {
|
||||
const version = execSync(`${cmd} --version`, { encoding: 'utf-8' }).trim()
|
||||
pass('Python', version)
|
||||
return { name: 'Python', ok: true, version }
|
||||
} catch {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
fail('Python 3', 'not installed')
|
||||
const platform = detectPlatform()
|
||||
const instructions =
|
||||
platform === 'macos'
|
||||
? [
|
||||
'Python 3 is needed for the ComfyUI backend.',
|
||||
'',
|
||||
' brew install python3',
|
||||
'',
|
||||
'Or download from: https://www.python.org/downloads/'
|
||||
]
|
||||
: platform === 'windows'
|
||||
? [
|
||||
'Download Python from: https://www.python.org/downloads/',
|
||||
'Check "Add Python to PATH" during install.'
|
||||
]
|
||||
: [
|
||||
' sudo apt install python3 # Debian/Ubuntu',
|
||||
' sudo dnf install python3 # Fedora'
|
||||
]
|
||||
info(instructions)
|
||||
return { name: 'Python 3', ok: false, installInstructions: instructions }
|
||||
}
|
||||
7
tools/test-recorder/src/checks/types.ts
Normal file
7
tools/test-recorder/src/checks/types.ts
Normal file
@@ -0,0 +1,7 @@
|
||||
export interface CheckResult {
|
||||
name: string
|
||||
ok: boolean
|
||||
version?: string
|
||||
optional?: boolean
|
||||
installInstructions?: string[]
|
||||
}
|
||||
34
tools/test-recorder/src/checks/xcode.ts
Normal file
34
tools/test-recorder/src/checks/xcode.ts
Normal file
@@ -0,0 +1,34 @@
|
||||
import { execSync } from 'node:child_process'
|
||||
import { detectPlatform } from './platform'
|
||||
import { pass, fail, info } from '../ui/logger'
|
||||
import type { CheckResult } from './types'
|
||||
|
||||
export async function checkXcode(): Promise<CheckResult> {
|
||||
if (detectPlatform() !== 'macos') {
|
||||
return { name: 'Xcode CLI Tools', ok: true, version: 'n/a (not macOS)' }
|
||||
}
|
||||
|
||||
try {
|
||||
const path = execSync('xcode-select -p', { encoding: 'utf-8' }).trim()
|
||||
pass('Xcode CLI Tools', path)
|
||||
return { name: 'Xcode CLI Tools', ok: true, version: path }
|
||||
} catch {
|
||||
fail('Xcode CLI Tools', 'not installed')
|
||||
const instructions = [
|
||||
'Xcode Command Line Tools are required for git and build tools.',
|
||||
'',
|
||||
'To install, run this in Terminal:',
|
||||
'',
|
||||
' xcode-select --install',
|
||||
'',
|
||||
'A popup will appear — click "Install" and wait (~5 min).',
|
||||
'When done, come back here and press Enter.'
|
||||
]
|
||||
info(instructions)
|
||||
return {
|
||||
name: 'Xcode CLI Tools',
|
||||
ok: false,
|
||||
installInstructions: instructions
|
||||
}
|
||||
}
|
||||
}
|
||||
39
tools/test-recorder/src/commands/check.ts
Normal file
39
tools/test-recorder/src/commands/check.ts
Normal file
@@ -0,0 +1,39 @@
|
||||
import { checkPlatform } from '../checks/platform'
|
||||
import { checkXcode } from '../checks/xcode'
|
||||
import { checkGit } from '../checks/git'
|
||||
import { checkNode } from '../checks/node'
|
||||
import { checkPnpm } from '../checks/pnpm'
|
||||
import { checkPython } from '../checks/python'
|
||||
import { checkPlaywright } from '../checks/playwright'
|
||||
import { checkGh } from '../checks/gh'
|
||||
import { checkDevServer } from '../checks/devServer'
|
||||
import { checkBackend } from '../checks/backend'
|
||||
import { header } from '../ui/logger'
|
||||
import type { CheckResult } from '../checks/types'
|
||||
|
||||
export async function runChecks(): Promise<{
|
||||
results: CheckResult[]
|
||||
allPassed: boolean
|
||||
}> {
|
||||
header('Environment Check')
|
||||
|
||||
const results: CheckResult[] = []
|
||||
|
||||
// System checks (sequential — each depends on prior)
|
||||
results.push(checkPlatform())
|
||||
results.push(await checkXcode())
|
||||
results.push(await checkGit())
|
||||
results.push(await checkNode())
|
||||
results.push(await checkPnpm())
|
||||
results.push(await checkPython())
|
||||
results.push(await checkPlaywright())
|
||||
results.push(await checkGh())
|
||||
|
||||
header('Services Check')
|
||||
|
||||
results.push(await checkBackend())
|
||||
results.push(await checkDevServer())
|
||||
|
||||
const requiredFailed = results.filter((r) => !r.ok && !r.optional)
|
||||
return { results, allPassed: requiredFailed.length === 0 }
|
||||
}
|
||||
38
tools/test-recorder/src/commands/list.ts
Normal file
38
tools/test-recorder/src/commands/list.ts
Normal file
@@ -0,0 +1,38 @@
|
||||
import pc from 'picocolors'
|
||||
import { listWorkflows, findProjectRoot } from '../recorder/runner'
|
||||
import { header } from '../ui/logger'
|
||||
|
||||
export async function runList(): Promise<void> {
|
||||
header('Available Workflows')
|
||||
|
||||
const projectRoot = findProjectRoot()
|
||||
const workflows = listWorkflows(projectRoot)
|
||||
|
||||
if (workflows.length === 0) {
|
||||
console.log(pc.dim(' No workflow assets found in browser_tests/assets/'))
|
||||
return
|
||||
}
|
||||
|
||||
console.log(
|
||||
pc.dim(` Found ${workflows.length} workflows in browser_tests/assets/:\n`)
|
||||
)
|
||||
|
||||
let currentDir = ''
|
||||
for (const wf of workflows) {
|
||||
const parts = wf.split('/')
|
||||
if (parts.length > 1) {
|
||||
const dir = parts.slice(0, -1).join('/')
|
||||
if (dir !== currentDir) {
|
||||
currentDir = dir
|
||||
console.log(pc.bold(` ${dir}/`))
|
||||
}
|
||||
console.log(` ${pc.cyan(parts[parts.length - 1])}`)
|
||||
} else {
|
||||
if (currentDir !== '') {
|
||||
currentDir = ''
|
||||
console.log()
|
||||
}
|
||||
console.log(` ${pc.cyan(wf)}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
243
tools/test-recorder/src/commands/record.ts
Normal file
243
tools/test-recorder/src/commands/record.ts
Normal file
@@ -0,0 +1,243 @@
|
||||
import { createInterface } from 'node:readline'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { writeFileSync, mkdirSync, readFileSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
import {
|
||||
text,
|
||||
select,
|
||||
multiselect,
|
||||
confirm,
|
||||
isCancel,
|
||||
cancel,
|
||||
spinner
|
||||
} from '@clack/prompts'
|
||||
import pc from 'picocolors'
|
||||
import { runChecks } from './check'
|
||||
import {
|
||||
runRecording,
|
||||
findProjectRoot,
|
||||
listWorkflows
|
||||
} from '../recorder/runner'
|
||||
import { transform, formatTransformSummary } from '../transform/engine'
|
||||
import { stepHeader } from '../ui/steps'
|
||||
import { pass, fail, info, blank, box } from '../ui/logger'
|
||||
import { checkGhAvailable, createPr } from '../pr/gh'
|
||||
import { printManualInstructions } from '../pr/manual'
|
||||
import { copyToClipboard } from '../pr/clipboard'
|
||||
|
||||
function toSlug(description: string): string {
|
||||
return description
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
}
|
||||
|
||||
function readMultiline(): Promise<string> {
|
||||
return new Promise((resolve) => {
|
||||
const lines: string[] = []
|
||||
const rl = createInterface({ input: process.stdin })
|
||||
rl.on('line', (line) => lines.push(line))
|
||||
rl.on('close', () => resolve(lines.join('\n')))
|
||||
})
|
||||
}
|
||||
|
||||
export async function runRecord(): Promise<void> {
|
||||
// ── Step 1: Environment Check ──────────────────────────────
|
||||
stepHeader(1, 7, 'Environment Check')
|
||||
const { allPassed } = await runChecks()
|
||||
if (!allPassed) {
|
||||
blank()
|
||||
fail('Some required checks failed. Fix the issues above.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Step 2: Project Setup ──────────────────────────────────
|
||||
stepHeader(2, 7, 'Project Setup')
|
||||
|
||||
let projectRoot: string
|
||||
try {
|
||||
projectRoot = findProjectRoot()
|
||||
} catch (err) {
|
||||
fail(err instanceof Error ? err.message : 'Could not find project root')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const s = spinner()
|
||||
s.start('Installing dependencies...')
|
||||
spawnSync('pnpm', ['install'], { cwd: projectRoot, stdio: 'pipe' })
|
||||
s.stop('Dependencies installed')
|
||||
pass('Project ready', projectRoot)
|
||||
|
||||
// ── Step 3: Backend & Dev Server ───────────────────────────
|
||||
stepHeader(3, 7, 'Backend & Dev Server')
|
||||
pass('Services checked in Step 1')
|
||||
|
||||
// ── Step 4: Configure Your Test ────────────────────────────
|
||||
stepHeader(4, 7, 'Configure Your Test')
|
||||
|
||||
const description = await text({
|
||||
message: 'What are you testing?',
|
||||
placeholder: 'e.g., adding a KSampler node and queuing'
|
||||
})
|
||||
if (isCancel(description)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
let slug = toSlug(description)
|
||||
|
||||
const filenameOk = await confirm({
|
||||
message: `Generated filename: ${slug}.spec.ts — looks good?`
|
||||
})
|
||||
if (isCancel(filenameOk)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
if (!filenameOk) {
|
||||
const customName = await text({
|
||||
message: 'Enter a custom filename (without .spec.ts):',
|
||||
placeholder: slug
|
||||
})
|
||||
if (isCancel(customName)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
slug = toSlug(customName)
|
||||
}
|
||||
|
||||
const selectedTags = await multiselect({
|
||||
message: 'Select tags for this test:',
|
||||
options: [
|
||||
{ value: '@canvas', label: '@canvas' },
|
||||
{ value: '@widget', label: '@widget' },
|
||||
{ value: '@sidebar', label: '@sidebar' },
|
||||
{ value: '@smoke', label: '@smoke' },
|
||||
{ value: '@mobile', label: '@mobile' },
|
||||
{ value: '@screenshot', label: '@screenshot' }
|
||||
],
|
||||
initialValues: ['@canvas']
|
||||
})
|
||||
if (isCancel(selectedTags)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const workflows = listWorkflows(projectRoot)
|
||||
const workflowOptions: {
|
||||
value: string
|
||||
label: string
|
||||
hint?: string
|
||||
}[] = [
|
||||
{ value: '', label: '(empty canvas)', hint: 'start fresh' },
|
||||
...workflows.map((wf) => ({ value: wf, label: wf }))
|
||||
]
|
||||
|
||||
const selectedWorkflow = await select({
|
||||
message: 'Start with a pre-loaded workflow?',
|
||||
options: workflowOptions
|
||||
})
|
||||
if (isCancel(selectedWorkflow)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
// ── Step 5: Record ─────────────────────────────────────────
|
||||
stepHeader(5, 7, 'Record')
|
||||
|
||||
const result = await runRecording({
|
||||
testName: slug,
|
||||
workflow: selectedWorkflow || undefined,
|
||||
projectRoot
|
||||
})
|
||||
if (!result.success) {
|
||||
fail('Recording failed', result.error)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// ── Step 6: Paste & Transform ──────────────────────────────
|
||||
stepHeader(6, 7, 'Paste & Transform')
|
||||
|
||||
info([
|
||||
'Copy the generated code from the Playwright Inspector.',
|
||||
'',
|
||||
'Paste your code below, then press ' +
|
||||
pc.bold('Ctrl+D') +
|
||||
' (Mac/Linux) or ' +
|
||||
pc.bold('Ctrl+Z') +
|
||||
' (Windows) when done:'
|
||||
])
|
||||
blank()
|
||||
|
||||
const pastedCode = await readMultiline()
|
||||
|
||||
if (!pastedCode.trim()) {
|
||||
blank()
|
||||
info([
|
||||
'No code pasted. You can transform later with:',
|
||||
pc.cyan(` comfy-test transform <file>`)
|
||||
])
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const transformResult = transform(pastedCode, {
|
||||
testName: slug,
|
||||
tags: selectedTags as string[]
|
||||
})
|
||||
|
||||
blank()
|
||||
const summary = formatTransformSummary(transformResult)
|
||||
for (const line of summary) {
|
||||
console.log(` ${line}`)
|
||||
}
|
||||
|
||||
const testsDir = join(projectRoot, 'browser_tests', 'tests')
|
||||
mkdirSync(testsDir, { recursive: true })
|
||||
const outputPath = join(testsDir, `${slug}.spec.ts`)
|
||||
writeFileSync(outputPath, transformResult.code)
|
||||
|
||||
blank()
|
||||
pass('Test saved', outputPath)
|
||||
|
||||
// ── Step 7: Finalize ───────────────────────────────────────
|
||||
stepHeader(7, 7, 'Finalize')
|
||||
|
||||
box([
|
||||
'Run your test:',
|
||||
pc.cyan(` pnpm exec playwright test ${slug} --headed`),
|
||||
'',
|
||||
'Review in UI mode:',
|
||||
pc.cyan(' pnpm exec playwright test --ui')
|
||||
])
|
||||
blank()
|
||||
|
||||
const wantPr = await confirm({
|
||||
message: 'Create a Pull Request now?'
|
||||
})
|
||||
if (isCancel(wantPr)) {
|
||||
cancel('Operation cancelled')
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
if (wantPr) {
|
||||
const gh = await checkGhAvailable()
|
||||
if (gh.available && gh.authenticated) {
|
||||
await createPr({
|
||||
testFilePath: outputPath,
|
||||
testName: slug,
|
||||
description: description as string
|
||||
})
|
||||
} else {
|
||||
const fileContents = readFileSync(outputPath, 'utf-8')
|
||||
await copyToClipboard(fileContents)
|
||||
const relativePath = `browser_tests/tests/${slug}.spec.ts`
|
||||
printManualInstructions({
|
||||
testFilePath: outputPath,
|
||||
testName: slug,
|
||||
relativePath
|
||||
})
|
||||
}
|
||||
} else {
|
||||
blank()
|
||||
info(['You can create a PR later.', pc.dim(`Test file: ${outputPath}`)])
|
||||
}
|
||||
}
|
||||
48
tools/test-recorder/src/commands/transform.ts
Normal file
48
tools/test-recorder/src/commands/transform.ts
Normal file
@@ -0,0 +1,48 @@
|
||||
import { readFileSync, writeFileSync } from 'node:fs'
|
||||
import pc from 'picocolors'
|
||||
import { transform, formatTransformSummary } from '../transform/engine'
|
||||
import { header } from '../ui/logger'
|
||||
|
||||
export async function runTransform(
|
||||
filePath: string,
|
||||
options: {
|
||||
testName?: string
|
||||
tags?: string[]
|
||||
output?: string
|
||||
} = {}
|
||||
): Promise<void> {
|
||||
header('Transform Codegen → Conventions')
|
||||
|
||||
// Read input
|
||||
let rawCode: string
|
||||
try {
|
||||
rawCode = readFileSync(filePath, 'utf-8')
|
||||
} catch {
|
||||
console.log(pc.red(` Could not read file: ${filePath}`))
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(pc.dim(` Input: ${filePath}`))
|
||||
console.log()
|
||||
|
||||
// Transform
|
||||
const testName =
|
||||
options.testName ??
|
||||
filePath.split('/').pop()?.replace('.raw.spec.ts', '') ??
|
||||
'test'
|
||||
const tags = options.tags ?? ['@canvas']
|
||||
const result = transform(rawCode, { testName, tags })
|
||||
|
||||
// Print summary
|
||||
console.log(' Transforms applied:')
|
||||
for (const line of formatTransformSummary(result)) {
|
||||
console.log(` ${line}`)
|
||||
}
|
||||
console.log()
|
||||
|
||||
// Write output
|
||||
const outputPath =
|
||||
options.output ?? filePath.replace('.raw.spec.ts', '.spec.ts')
|
||||
writeFileSync(outputPath, result.code)
|
||||
console.log(pc.green(` ✅ Saved: ${outputPath}`))
|
||||
}
|
||||
65
tools/test-recorder/src/index.ts
Normal file
65
tools/test-recorder/src/index.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { intro, outro } from '@clack/prompts'
|
||||
import pc from 'picocolors'
|
||||
|
||||
const args = process.argv.slice(2)
|
||||
const command = args[0]
|
||||
|
||||
intro(pc.bgCyan(pc.black(' 🎭 ComfyUI Test Recorder ')))
|
||||
|
||||
switch (command) {
|
||||
case 'record': {
|
||||
const { runRecord } = await import('./commands/record')
|
||||
await runRecord()
|
||||
break
|
||||
}
|
||||
case 'transform': {
|
||||
const filePath = args[1]
|
||||
if (!filePath) {
|
||||
console.log(pc.red(' Usage: comfy-test transform <file>'))
|
||||
process.exit(1)
|
||||
}
|
||||
const { runTransform } = await import('./commands/transform')
|
||||
await runTransform(filePath, {
|
||||
testName: args[2],
|
||||
tags: args.slice(3)
|
||||
})
|
||||
break
|
||||
}
|
||||
case 'check': {
|
||||
const { runChecks } = await import('./commands/check')
|
||||
const { allPassed } = await runChecks()
|
||||
if (!allPassed) {
|
||||
console.log()
|
||||
console.log(
|
||||
pc.red(
|
||||
' Some required checks failed. Fix the issues above and try again.'
|
||||
)
|
||||
)
|
||||
}
|
||||
break
|
||||
}
|
||||
case 'list': {
|
||||
const { runList } = await import('./commands/list')
|
||||
await runList()
|
||||
break
|
||||
}
|
||||
default:
|
||||
console.log(`
|
||||
Usage: comfy-test <command>
|
||||
|
||||
Commands:
|
||||
record Record a new browser test interactively
|
||||
transform Transform raw codegen output to conventions
|
||||
check Check environment prerequisites
|
||||
list List available test workflows
|
||||
|
||||
Options:
|
||||
--help Show help
|
||||
--version Show version
|
||||
`)
|
||||
break
|
||||
}
|
||||
|
||||
outro(pc.dim('https://github.com/Comfy-Org/ComfyUI_frontend'))
|
||||
36
tools/test-recorder/src/pr/clipboard.ts
Normal file
36
tools/test-recorder/src/pr/clipboard.ts
Normal file
@@ -0,0 +1,36 @@
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { detectPlatform } from '../checks/platform'
|
||||
|
||||
export async function copyToClipboard(content: string): Promise<boolean> {
|
||||
const platform = detectPlatform()
|
||||
|
||||
let cmd: string
|
||||
let args: string[]
|
||||
|
||||
if (platform === 'macos') {
|
||||
cmd = 'pbcopy'
|
||||
args = []
|
||||
} else if (platform === 'windows') {
|
||||
cmd = 'clip.exe'
|
||||
args = []
|
||||
} else {
|
||||
// Linux: try xclip first, fall back to xsel
|
||||
const xclip = spawnSync('xclip', ['-selection', 'clipboard'], {
|
||||
input: content,
|
||||
stdio: 'pipe'
|
||||
})
|
||||
if (xclip.status === 0) return true
|
||||
|
||||
const xsel = spawnSync('xsel', ['--clipboard', '--input'], {
|
||||
input: content,
|
||||
stdio: 'pipe'
|
||||
})
|
||||
return xsel.status === 0
|
||||
}
|
||||
|
||||
const result = spawnSync(cmd, args, {
|
||||
input: content,
|
||||
stdio: 'pipe'
|
||||
})
|
||||
return result.status === 0
|
||||
}
|
||||
90
tools/test-recorder/src/pr/gh.ts
Normal file
90
tools/test-recorder/src/pr/gh.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
import { execSync, spawnSync } from 'node:child_process'
|
||||
import { pass, fail } from '../ui/logger'
|
||||
|
||||
interface PrOptions {
|
||||
testFilePath: string
|
||||
testName: string
|
||||
description: string
|
||||
branchName?: string
|
||||
}
|
||||
|
||||
interface PrResult {
|
||||
success: boolean
|
||||
url?: string
|
||||
error?: string
|
||||
}
|
||||
|
||||
export async function checkGhAvailable(): Promise<{
|
||||
available: boolean
|
||||
authenticated: boolean
|
||||
}> {
|
||||
try {
|
||||
execSync('gh --version', { stdio: 'pipe' })
|
||||
} catch {
|
||||
return { available: false, authenticated: false }
|
||||
}
|
||||
|
||||
try {
|
||||
execSync('gh auth status', {
|
||||
stdio: 'pipe',
|
||||
encoding: 'utf-8'
|
||||
})
|
||||
return { available: true, authenticated: true }
|
||||
} catch {
|
||||
return { available: true, authenticated: false }
|
||||
}
|
||||
}
|
||||
|
||||
export async function createPr(options: PrOptions): Promise<PrResult> {
|
||||
const branchName = options.branchName ?? `test/${options.testName}`
|
||||
const commitMsg = `test: add ${options.testName} e2e test\n\n${options.description}`
|
||||
const prTitle = `test: add ${options.testName} e2e test`
|
||||
const prBody =
|
||||
`${options.description}\n\n---\n\n` + 'Recorded with `comfy-test record`'
|
||||
|
||||
const checkout = spawnSync('git', ['checkout', '-b', branchName], {
|
||||
encoding: 'utf-8',
|
||||
stdio: 'pipe'
|
||||
})
|
||||
if (checkout.status !== 0) {
|
||||
fail('Branch creation failed', checkout.stderr.trim())
|
||||
return {
|
||||
success: false,
|
||||
error: checkout.stderr.trim()
|
||||
}
|
||||
}
|
||||
pass('Created branch', branchName)
|
||||
|
||||
const add = spawnSync('git', ['add', options.testFilePath], {
|
||||
encoding: 'utf-8',
|
||||
stdio: 'pipe'
|
||||
})
|
||||
if (add.status !== 0) {
|
||||
fail('Git add failed', add.stderr.trim())
|
||||
return { success: false, error: add.stderr.trim() }
|
||||
}
|
||||
|
||||
const commit = spawnSync('git', ['commit', '-m', commitMsg], {
|
||||
encoding: 'utf-8',
|
||||
stdio: 'pipe'
|
||||
})
|
||||
if (commit.status !== 0) {
|
||||
fail('Git commit failed', commit.stderr.trim())
|
||||
return { success: false, error: commit.stderr.trim() }
|
||||
}
|
||||
pass('Committed test file')
|
||||
|
||||
const pr = spawnSync(
|
||||
'gh',
|
||||
['pr', 'create', '--title', prTitle, '--body', prBody, '--fill'],
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
)
|
||||
if (pr.status !== 0) {
|
||||
fail('PR creation failed', pr.stderr.trim())
|
||||
return { success: false, error: pr.stderr.trim() }
|
||||
}
|
||||
|
||||
const url = pr.stdout.trim()
|
||||
pass('Pull request created', url)
|
||||
return { success: true, url }
|
||||
}
|
||||
29
tools/test-recorder/src/pr/manual.ts
Normal file
29
tools/test-recorder/src/pr/manual.ts
Normal file
@@ -0,0 +1,29 @@
|
||||
import { info } from '../ui/logger'
|
||||
|
||||
interface ManualPrOptions {
|
||||
testFilePath: string
|
||||
testName: string
|
||||
relativePath: string
|
||||
repoUrl?: string
|
||||
}
|
||||
|
||||
const DEFAULT_REPO = 'https://github.com/Comfy-Org/ComfyUI_frontend'
|
||||
|
||||
export function printManualInstructions(options: ManualPrOptions): void {
|
||||
const repo = options.repoUrl ?? DEFAULT_REPO
|
||||
|
||||
info([
|
||||
'Create a PR manually:',
|
||||
'',
|
||||
` 1. Go to: ${repo}`,
|
||||
' 2. Click "Add file" → "Create new file"',
|
||||
` 3. Set path to: ${options.relativePath}`,
|
||||
' 4. Paste the contents (copied to your clipboard ✅)',
|
||||
` 5. Write commit message: "test: add ${options.testName} e2e test"`,
|
||||
' 6. Select "Create a new branch" → click "Propose new file"',
|
||||
' 7. Click "Create pull request"',
|
||||
'',
|
||||
'Or, ask an AI agent: "Create a PR with this test file"',
|
||||
`(The file is at ${options.testFilePath})`
|
||||
])
|
||||
}
|
||||
147
tools/test-recorder/src/recorder/runner.ts
Normal file
147
tools/test-recorder/src/recorder/runner.ts
Normal file
@@ -0,0 +1,147 @@
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { existsSync, readdirSync, statSync } from 'node:fs'
|
||||
import { join } from 'node:path'
|
||||
import pc from 'picocolors'
|
||||
import { generateRecordingTemplate, cleanupRecordingTemplate } from './template'
|
||||
import { box } from '../ui/logger'
|
||||
|
||||
interface RunnerOptions {
|
||||
testName: string
|
||||
workflow?: string
|
||||
projectRoot: string
|
||||
}
|
||||
|
||||
interface RecordingResult {
|
||||
success: boolean
|
||||
rawOutputPath?: string
|
||||
error?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the project root by looking for playwright.config.ts
|
||||
*/
|
||||
export function findProjectRoot(): string {
|
||||
let dir = process.cwd()
|
||||
while (dir !== '/') {
|
||||
if (existsSync(join(dir, 'playwright.config.ts'))) {
|
||||
return dir
|
||||
}
|
||||
dir = join(dir, '..')
|
||||
}
|
||||
throw new Error(
|
||||
'Could not find project root (no playwright.config.ts found). ' +
|
||||
'Run this command from the ComfyUI_frontend directory.'
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* List available workflow assets.
|
||||
*/
|
||||
export function listWorkflows(projectRoot: string): string[] {
|
||||
const assetsDir = join(projectRoot, 'browser_tests', 'assets')
|
||||
const results: string[] = []
|
||||
|
||||
function walk(dir: string, prefix: string) {
|
||||
try {
|
||||
for (const entry of readdirSync(dir)) {
|
||||
const fullPath = join(dir, entry)
|
||||
const relPath = prefix ? `${prefix}/${entry}` : entry
|
||||
const stat = statSync(fullPath)
|
||||
if (stat.isDirectory()) {
|
||||
walk(fullPath, relPath)
|
||||
} else if (entry.endsWith('.json')) {
|
||||
results.push(relPath.replace(/\.json$/, ''))
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Directory might not exist
|
||||
}
|
||||
}
|
||||
|
||||
walk(assetsDir, '')
|
||||
return results.sort()
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the recording session.
|
||||
*
|
||||
* 1. Generate the temporary test file with page.pause()
|
||||
* 2. Run it in headed mode — this opens the Playwright Inspector
|
||||
* 3. User records their actions
|
||||
* 4. User closes the browser
|
||||
* 5. We save the output as *.raw.spec.ts
|
||||
*/
|
||||
export async function runRecording(
|
||||
options: RunnerOptions
|
||||
): Promise<RecordingResult> {
|
||||
const browserTestsDir = join(options.projectRoot, 'browser_tests')
|
||||
|
||||
// Generate the recording template
|
||||
const tempFile = generateRecordingTemplate(
|
||||
{ testName: options.testName, workflow: options.workflow },
|
||||
browserTestsDir
|
||||
)
|
||||
|
||||
console.log()
|
||||
box([
|
||||
'A browser window will open.',
|
||||
'',
|
||||
'👉 Perform your test actions:',
|
||||
' • Click, type, drag — everything is recorded',
|
||||
' • Use toolbar buttons to add assertions',
|
||||
' • When done, close the browser window',
|
||||
'',
|
||||
'The Playwright Inspector shows generated code.'
|
||||
])
|
||||
console.log()
|
||||
|
||||
try {
|
||||
// Run the test in headed mode with PWDEBUG to force inspector
|
||||
spawnSync(
|
||||
'pnpm',
|
||||
[
|
||||
'exec',
|
||||
'playwright',
|
||||
'test',
|
||||
'_recording-session',
|
||||
'--headed',
|
||||
'--project=chromium',
|
||||
'--timeout=0'
|
||||
],
|
||||
{
|
||||
cwd: options.projectRoot,
|
||||
stdio: 'inherit',
|
||||
env: {
|
||||
...process.env,
|
||||
PWDEBUG: '1',
|
||||
PLAYWRIGHT_LOCAL: '1'
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
// The user will have copied code from the Inspector.
|
||||
// We can't automatically capture Inspector output, so we'll
|
||||
// prompt the user to paste it or check if they saved a file.
|
||||
console.log()
|
||||
console.log(pc.green(' ✅ Recording session complete.'))
|
||||
console.log()
|
||||
|
||||
// Save a placeholder raw file — the user pastes codegen output here
|
||||
// or the record command handles prompting for it
|
||||
const rawOutputPath = join(
|
||||
browserTestsDir,
|
||||
'tests',
|
||||
`${options.testName}.raw.spec.ts`
|
||||
)
|
||||
|
||||
return { success: true, rawOutputPath }
|
||||
} catch (err) {
|
||||
return {
|
||||
success: false,
|
||||
error: `Recording failed: ${err instanceof Error ? err.message : String(err)}`
|
||||
}
|
||||
} finally {
|
||||
// Always clean up the temp recording file
|
||||
cleanupRecordingTemplate(browserTestsDir)
|
||||
}
|
||||
}
|
||||
70
tools/test-recorder/src/recorder/template.ts
Normal file
70
tools/test-recorder/src/recorder/template.ts
Normal file
@@ -0,0 +1,70 @@
|
||||
import { writeFileSync, mkdirSync, unlinkSync } from 'node:fs'
|
||||
import { join, dirname } from 'node:path'
|
||||
|
||||
interface TemplateOptions {
|
||||
workflow?: string
|
||||
testName: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates a temporary test file that uses page.pause() to open
|
||||
* the Playwright Inspector with codegen controls.
|
||||
*
|
||||
* The test file:
|
||||
* 1. Uses comfyPageFixture to get full fixture context
|
||||
* 2. Optionally loads a workflow
|
||||
* 3. Calls page.pause() to open the Inspector
|
||||
*/
|
||||
export function generateRecordingTemplate(
|
||||
options: TemplateOptions,
|
||||
browserTestsDir: string
|
||||
): string {
|
||||
const filePath = join(browserTestsDir, 'tests', `_recording-session.spec.ts`)
|
||||
|
||||
const workflowLine = options.workflow
|
||||
? ` // Load the selected workflow\n await comfyPage.workflow.loadWorkflow('${options.workflow}')\n await comfyPage.nextFrame()\n`
|
||||
: ''
|
||||
|
||||
const code = `/**
|
||||
* Auto-generated recording session.
|
||||
* This file is temporary — it will be deleted after recording.
|
||||
*
|
||||
* DO NOT COMMIT THIS FILE.
|
||||
*/
|
||||
import {
|
||||
comfyPageFixture as test,
|
||||
comfyExpect as expect
|
||||
} from '../fixtures/ComfyPage'
|
||||
|
||||
test('recording: ${options.testName}', async ({ comfyPage }) => {
|
||||
${workflowLine}
|
||||
// ┌────────────────────────────────────────────────────────┐
|
||||
// │ The Playwright Inspector will open. │
|
||||
// │ │
|
||||
// │ 1. Click the Record button (red circle) to start │
|
||||
// │ 2. Perform your test actions in the browser │
|
||||
// │ 3. Use toolbar buttons to add assertions │
|
||||
// │ 4. Click Stop when done │
|
||||
// │ 5. Copy the generated code from the Inspector │
|
||||
// │ 6. Close the browser window │
|
||||
// └────────────────────────────────────────────────────────┘
|
||||
await comfyPage.page.pause()
|
||||
})
|
||||
`
|
||||
|
||||
mkdirSync(dirname(filePath), { recursive: true })
|
||||
writeFileSync(filePath, code)
|
||||
return filePath
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up the temporary recording file.
|
||||
*/
|
||||
export function cleanupRecordingTemplate(browserTestsDir: string): void {
|
||||
const filePath = join(browserTestsDir, 'tests', '_recording-session.spec.ts')
|
||||
try {
|
||||
unlinkSync(filePath)
|
||||
} catch {
|
||||
// File might already be deleted
|
||||
}
|
||||
}
|
||||
89
tools/test-recorder/src/transform/engine.ts
Normal file
89
tools/test-recorder/src/transform/engine.ts
Normal file
@@ -0,0 +1,89 @@
|
||||
import { transformRules, structuralTransforms } from './rules'
|
||||
|
||||
interface TransformResult {
|
||||
code: string
|
||||
appliedRules: { name: string; description: string }[]
|
||||
warnings: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Transform raw Playwright codegen output into ComfyUI conventions.
|
||||
*/
|
||||
export function transform(
|
||||
rawCode: string,
|
||||
options: {
|
||||
testName?: string
|
||||
tags?: string[]
|
||||
} = {}
|
||||
): TransformResult {
|
||||
const testName = options.testName ?? 'unnamed-test'
|
||||
const tags = options.tags ?? ['@canvas']
|
||||
|
||||
let code = rawCode
|
||||
const appliedRules: { name: string; description: string }[] = []
|
||||
const warnings: string[] = []
|
||||
|
||||
// Phase 1: Apply regex-based rules
|
||||
for (const rule of transformRules) {
|
||||
const before = code
|
||||
if (typeof rule.replacement === 'string') {
|
||||
code = code.replace(rule.pattern, rule.replacement)
|
||||
} else {
|
||||
code = code.replace(
|
||||
rule.pattern,
|
||||
rule.replacement as (...args: string[]) => string
|
||||
)
|
||||
}
|
||||
if (code !== before) {
|
||||
appliedRules.push({ name: rule.name, description: rule.description })
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 2: Clean up empty lines from removed statements
|
||||
code = code.replace(/\n{3,}/g, '\n\n')
|
||||
|
||||
// Phase 3: Apply structural transforms
|
||||
for (const transform of structuralTransforms) {
|
||||
const before = code
|
||||
code = transform.apply(code, testName, tags)
|
||||
if (code !== before) {
|
||||
appliedRules.push({
|
||||
name: transform.name,
|
||||
description: transform.description
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 4: Check for remaining issues and warn
|
||||
if (code.includes('waitForTimeout')) {
|
||||
warnings.push(
|
||||
'Still contains waitForTimeout — replace with comfyPage.nextFrame() or retrying assertions'
|
||||
)
|
||||
}
|
||||
if (code.includes("from '@playwright/test'")) {
|
||||
warnings.push(
|
||||
'Still imports from @playwright/test — should use ../fixtures/ComfyPage'
|
||||
)
|
||||
}
|
||||
if (/position:\s*\{\s*x:\s*\d+,\s*y:\s*\d+/.test(code)) {
|
||||
warnings.push(
|
||||
'Contains pixel coordinates — consider replacing with node references (comfyPage.nodeOps.*) where possible'
|
||||
)
|
||||
}
|
||||
|
||||
return { code: code.trim() + '\n', appliedRules, warnings }
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a human-readable summary of what was transformed.
|
||||
*/
|
||||
export function formatTransformSummary(result: TransformResult): string[] {
|
||||
const lines: string[] = []
|
||||
for (const rule of result.appliedRules) {
|
||||
lines.push(`✅ ${rule.description}`)
|
||||
}
|
||||
for (const warning of result.warnings) {
|
||||
lines.push(`⚠️ ${warning}`)
|
||||
}
|
||||
return lines
|
||||
}
|
||||
123
tools/test-recorder/src/transform/rules.ts
Normal file
123
tools/test-recorder/src/transform/rules.ts
Normal file
@@ -0,0 +1,123 @@
|
||||
interface TransformRule {
|
||||
name: string
|
||||
description: string
|
||||
pattern: RegExp
|
||||
replacement: string | ((match: string, ...groups: string[]) => string)
|
||||
category: 'import' | 'fixture' | 'locator' | 'wait' | 'structure' | 'cleanup'
|
||||
}
|
||||
|
||||
export const transformRules: TransformRule[] = [
|
||||
// === Import transforms ===
|
||||
{
|
||||
name: 'replace-test-import',
|
||||
description: 'Use comfyPageFixture instead of @playwright/test',
|
||||
pattern:
|
||||
/import\s*\{\s*test\s*,?\s*expect\s*\}\s*from\s*['"]@playwright\/test['"]/,
|
||||
replacement: `import {\n comfyPageFixture as test,\n comfyExpect as expect\n} from '../fixtures/ComfyPage'`,
|
||||
category: 'import'
|
||||
},
|
||||
{
|
||||
name: 'replace-test-only-import',
|
||||
description: 'Use comfyPageFixture when only test is imported',
|
||||
pattern: /import\s*\{\s*test\s*\}\s*from\s*['"]@playwright\/test['"]/,
|
||||
replacement: `import { comfyPageFixture as test } from '../fixtures/ComfyPage'`,
|
||||
category: 'import'
|
||||
},
|
||||
{
|
||||
name: 'replace-expect-only-import',
|
||||
description: 'Use comfyExpect when only expect is imported',
|
||||
pattern: /import\s*\{\s*expect\s*\}\s*from\s*['"]@playwright\/test['"]/,
|
||||
replacement: `import { comfyExpect as expect } from '../fixtures/ComfyPage'`,
|
||||
category: 'import'
|
||||
},
|
||||
|
||||
// === Fixture transforms ===
|
||||
{
|
||||
name: 'replace-page-destructure',
|
||||
description: 'Use comfyPage fixture instead of page',
|
||||
pattern: /async\s*\(\s*\{\s*page\s*\}\s*\)/g,
|
||||
replacement: 'async ({ comfyPage })',
|
||||
category: 'fixture'
|
||||
},
|
||||
|
||||
// === Remove page.goto ===
|
||||
{
|
||||
name: 'remove-goto',
|
||||
description: 'Remove page.goto — fixture handles navigation',
|
||||
pattern: /^\s*await\s+page\.goto\s*\([^)]*\)\s*;?\s*$/gm,
|
||||
replacement: '',
|
||||
category: 'locator'
|
||||
},
|
||||
|
||||
// === Locator transforms ===
|
||||
{
|
||||
name: 'replace-canvas-locator',
|
||||
description: 'Use comfyPage.canvas instead of page.locator("canvas")',
|
||||
pattern: /page\.locator\(\s*['"]canvas['"]\s*\)/g,
|
||||
replacement: 'comfyPage.canvas',
|
||||
category: 'locator'
|
||||
},
|
||||
{
|
||||
name: 'replace-search-placeholder',
|
||||
description: 'Use comfyPage.searchBox for search input',
|
||||
pattern: /page\.getByPlaceholder\(\s*['"]Search\s+Nodes\.{0,3}['"]\s*\)/g,
|
||||
replacement: 'comfyPage.searchBox.input',
|
||||
category: 'locator'
|
||||
},
|
||||
{
|
||||
name: 'replace-bare-page',
|
||||
description: 'Replace bare page references with comfyPage.page',
|
||||
pattern: /(?<!\w)page\./g,
|
||||
replacement: 'comfyPage.page.',
|
||||
category: 'locator'
|
||||
},
|
||||
|
||||
// === Wait transforms ===
|
||||
{
|
||||
name: 'replace-waitForTimeout',
|
||||
description: 'Use comfyPage.nextFrame() instead of arbitrary waits',
|
||||
pattern:
|
||||
/await\s+(?:comfyPage\.)?page\.waitForTimeout\s*\(\s*\d+\s*\)\s*;?/g,
|
||||
replacement: 'await comfyPage.nextFrame()',
|
||||
category: 'wait'
|
||||
}
|
||||
]
|
||||
|
||||
/**
|
||||
* Rules that need structural changes (not just regex replacement).
|
||||
* These are applied by the engine after regex rules.
|
||||
*/
|
||||
interface StructuralTransform {
|
||||
name: string
|
||||
description: string
|
||||
apply: (code: string, testName: string, tags: string[]) => string
|
||||
}
|
||||
|
||||
export const structuralTransforms: StructuralTransform[] = [
|
||||
{
|
||||
name: 'wrap-in-describe',
|
||||
description: 'Wrap test in test.describe with tags and afterEach',
|
||||
apply: (code: string, testName: string, tags: string[]) => {
|
||||
// If already has test.describe, skip
|
||||
if (code.includes('test.describe')) return code
|
||||
|
||||
const tagStr = tags.map((t) => `'${t}'`).join(', ')
|
||||
const descName = testName.replace(/[-_]/g, ' ').replace(/\.spec\.ts$/, '')
|
||||
|
||||
// Find the test() call and wrap it
|
||||
const testMatch = code.match(/^(import[\s\S]*?\n\n?)(test\s*\([\s\S]*)$/m)
|
||||
if (!testMatch) return code
|
||||
|
||||
const imports = testMatch[1]
|
||||
const testBody = testMatch[2]
|
||||
|
||||
return `${imports}test.describe('${descName}', { tag: [${tagStr}] }, () => {
|
||||
test.afterEach(async ({ comfyPage }) => {
|
||||
await comfyPage.canvasOps.resetView()
|
||||
})
|
||||
|
||||
${testBody.replace(/^/gm, ' ').trimStart()}
|
||||
})\n`
|
||||
}
|
||||
}
|
||||
]
|
||||
46
tools/test-recorder/src/ui/logger.ts
Normal file
46
tools/test-recorder/src/ui/logger.ts
Normal file
@@ -0,0 +1,46 @@
|
||||
import pc from 'picocolors'
|
||||
|
||||
export function pass(label: string, detail?: string) {
|
||||
const d = detail ? pc.dim(` ${detail}`) : ''
|
||||
console.log(` ${pc.green('✅')} ${label}${d}`)
|
||||
}
|
||||
|
||||
export function fail(label: string, detail?: string) {
|
||||
const d = detail ? pc.dim(` ${detail}`) : ''
|
||||
console.log(` ${pc.red('❌')} ${label}${d}`)
|
||||
}
|
||||
|
||||
export function warn(label: string, detail?: string) {
|
||||
const d = detail ? pc.dim(` ${detail}`) : ''
|
||||
console.log(` ${pc.yellow('⚠️')} ${label}${d}`)
|
||||
}
|
||||
|
||||
function wait(label: string) {
|
||||
console.log(` ${pc.blue('⏳')} ${label}`)
|
||||
}
|
||||
|
||||
export function info(lines: string[]) {
|
||||
for (const line of lines) {
|
||||
console.log(` ${pc.dim('┃')} ${line}`)
|
||||
}
|
||||
}
|
||||
|
||||
export function blank() {
|
||||
console.log()
|
||||
}
|
||||
|
||||
export function header(text: string) {
|
||||
console.log()
|
||||
console.log(pc.bold(` ━━━ ${text} ━━━`))
|
||||
console.log()
|
||||
}
|
||||
|
||||
export function box(lines: string[]) {
|
||||
const maxLen = Math.max(...lines.map((l) => l.length))
|
||||
const border = '─'.repeat(maxLen + 4)
|
||||
console.log(` ┌${border}┐`)
|
||||
for (const line of lines) {
|
||||
console.log(` │ ${line.padEnd(maxLen + 2)}│`)
|
||||
}
|
||||
console.log(` └${border}┘`)
|
||||
}
|
||||
7
tools/test-recorder/src/ui/steps.ts
Normal file
7
tools/test-recorder/src/ui/steps.ts
Normal file
@@ -0,0 +1,7 @@
|
||||
import pc from 'picocolors'
|
||||
|
||||
export function stepHeader(current: number, total: number, label: string) {
|
||||
console.log()
|
||||
console.log(pc.bold(` ━━━ Step ${current} of ${total}: ${label} ━━━`))
|
||||
console.log()
|
||||
}
|
||||
15
tools/test-recorder/tsconfig.json
Normal file
15
tools/test-recorder/tsconfig.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"declaration": true,
|
||||
"sourceMap": true
|
||||
},
|
||||
"include": ["src/**/*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user