diff --git a/.agents/skills/writing-playwright-tests/SKILL.md b/.agents/skills/writing-playwright-tests/SKILL.md new file mode 100644 index 000000000..65b6c972c --- /dev/null +++ b/.agents/skills/writing-playwright-tests/SKILL.md @@ -0,0 +1,45 @@ +--- +name: writing-playwright-tests +description: 'Writes Playwright e2e tests for ComfyUI_frontend. Use when creating, modifying, or debugging browser tests. Triggers on: playwright, e2e test, browser test, spec file.' +--- + +# Writing Playwright Tests for ComfyUI_frontend + +## Golden Rule + +**ALWAYS look at existing tests first.** Search `browser_tests/` for similar patterns before writing new tests. + +## Quick Reference + +| Task | Load This File | +| ------------------------- | -------------------------------------------------- | +| **Start a new test** | [core/setup.md](core/setup.md) | +| **Canvas interactions** | [core/canvas.md](core/canvas.md) | +| **Node operations** | [core/nodes.md](core/nodes.md) | +| **Widget testing** | [testing/widgets.md](testing/widgets.md) | +| **Vue Nodes 2.0** | [features/vue-nodes.md](features/vue-nodes.md) | +| **Commands/keybindings** | [features/commands.md](features/commands.md) | +| **Templates dialog** | [features/templates.md](features/templates.md) | +| **Workflow execution** | [features/execution.md](features/execution.md) | +| **File upload/drag-drop** | [testing/file-upload.md](testing/file-upload.md) | +| **API mocking** | [testing/mocking.md](testing/mocking.md) | +| **Test assets** | [testing/assets.md](testing/assets.md) | +| **Debug flaky tests** | [reference/debugging.md](reference/debugging.md) | +| **All fixture methods** | [reference/fixtures.md](reference/fixtures.md) | +| **Quick cheatsheet** | [reference/cheatsheet.md](reference/cheatsheet.md) | + +## Running Tests + +```bash +pnpm exec playwright test --ui # UI mode (recommended) +pnpm exec playwright test --grep @smoke # Run by tag +pnpm test:browser:local # Local with traces +``` + +## Project Files + +| Purpose | Path | +| ----------------- | ------------------------------------- | +| Main fixture | `browser_tests/fixtures/ComfyPage.ts` | +| Test assets | `browser_tests/assets/` | +| Playwright config | `playwright.config.ts` | diff --git a/.agents/skills/writing-playwright-tests/core/canvas.md b/.agents/skills/writing-playwright-tests/core/canvas.md new file mode 100644 index 000000000..f1c00e5d3 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/core/canvas.md @@ -0,0 +1,144 @@ +# Canvas Interaction Patterns + +## Critical: Always Use nextFrame() + +Canvas changes don't render immediately. **Always** call `nextFrame()` after canvas operations: + +```typescript +await comfyPage.canvas.click(100, 200) +await comfyPage.nextFrame() // ← Required! +``` + +## Focus Before Keyboard + +```typescript +await comfyPage.canvas.focus() // Required before any keyboard events +await comfyPage.page.keyboard.press('Control+a') +``` + +## Click Operations + +```typescript +// Basic click +await comfyPage.canvas.click(x, y) +await comfyPage.nextFrame() + +// Right-click (context menu) +await comfyPage.canvas.click(x, y, { button: 'right' }) +await comfyPage.nextFrame() + +// Double-click (add delay for reliability) +await comfyPage.canvas.dblclick(x, y, { delay: 5 }) +await comfyPage.nextFrame() +``` + +## Drag Operations + +Use `comfyMouse` fixture for reliable drags: + +```typescript +// Basic drag +await comfyMouse.dragFromTo( + { x: 100, y: 100 }, + { x: 300, y: 300 }, + { steps: 10 } // steps prevents flakiness +) +await comfyPage.nextFrame() + +// Drag node +const node = comfyPage.getNodeRefByTitle('KSampler') +await node.drag({ x: 50, y: 50 }) +await comfyPage.nextFrame() +``` + +## Pan and Zoom + +```typescript +// Pan the canvas +await comfyPage.pan({ x: 100, y: 100 }) +await comfyPage.nextFrame() + +// Zoom +await comfyPage.zoom(1.5) // 150% +await comfyPage.nextFrame() + +// Reset view +await comfyPage.resetView() +await comfyPage.nextFrame() +``` + +## Selection + +```typescript +// Select all +await comfyPage.canvas.focus() +await comfyPage.page.keyboard.press('Control+a') +await comfyPage.nextFrame() + +// Box select (marquee) +await comfyMouse.dragFromTo({ x: 0, y: 0 }, { x: 500, y: 500 }) +await comfyPage.nextFrame() +``` + +## Connecting Nodes + +```typescript +// Get output slot +const outputNode = comfyPage.getNodeRefByTitle('CLIP Loader') +const outputSlot = outputNode.getOutputSlot('CLIP') + +// Get input slot +const inputNode = comfyPage.getNodeRefByTitle('CLIP Text Encode') +const inputSlot = inputNode.getInputSlot('clip') + +// Connect via drag +await comfyMouse.dragFromTo( + await outputSlot.getPosition(), + await inputSlot.getPosition(), + { steps: 10 } +) +await comfyPage.nextFrame() +``` + +## Screenshot Testing + +```typescript +// Full canvas screenshot +await expect(comfyPage.canvas).toHaveScreenshot('canvas-state.png') + +// With masking +await expect(comfyPage.canvas).toHaveScreenshot('masked.png', { + mask: [comfyPage.page.locator('.timestamp')] +}) +``` + +## Example: Complete Canvas Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('Canvas Operations', { tag: ['@canvas'] }, () => { + test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() + }) + + test('pans and zooms canvas', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('default') + await comfyPage.nextFrame() + + // Pan + await comfyPage.pan({ x: 100, y: 50 }) + await comfyPage.nextFrame() + + // Zoom + await comfyPage.zoom(1.2) + await comfyPage.nextFrame() + + // Verify canvas state + await expect(comfyPage.canvas).toHaveScreenshot('panned-zoomed.png') + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/core/nodes.md b/.agents/skills/writing-playwright-tests/core/nodes.md new file mode 100644 index 000000000..9ac6733ce --- /dev/null +++ b/.agents/skills/writing-playwright-tests/core/nodes.md @@ -0,0 +1,206 @@ +# Node Patterns + +## Getting Node References + +### By Title (Preferred) + +```typescript +// Stable across positions and reloads +const node = comfyPage.getNodeRefByTitle('KSampler') +``` + +### By ID + +```typescript +// When you know the specific node ID +const node = comfyPage.getNodeRefById(5) +``` + +### First/Last Node + +```typescript +const firstNode = comfyPage.getFirstNode() +const lastNode = comfyPage.getLastNode() +``` + +## Node Operations + +### Click Node + +```typescript +const node = comfyPage.getNodeRefByTitle('KSampler') +await node.click() +await comfyPage.nextFrame() +``` + +### Drag Node + +```typescript +const node = comfyPage.getNodeRefByTitle('KSampler') +await node.drag({ x: 100, y: 50 }) +await comfyPage.nextFrame() +``` + +### Collapse/Expand + +```typescript +await node.collapse() +await comfyPage.nextFrame() + +await node.expand() +await comfyPage.nextFrame() + +// Assert state +await expect(node).toBeCollapsed() +``` + +### Bypass + +```typescript +await node.bypass() +await comfyPage.nextFrame() + +// Assert state +await expect(node).toBeBypassed() +``` + +### Pin + +```typescript +await node.pin() +await comfyPage.nextFrame() + +// Assert state +await expect(node).toBePinned() +``` + +### Delete + +```typescript +await node.click() +await comfyPage.canvas.focus() +await comfyPage.page.keyboard.press('Delete') +await comfyPage.nextFrame() +``` + +## Slots (Inputs/Outputs) + +### Get Slot Reference + +```typescript +// Output slot +const outputSlot = node.getOutputSlot('MODEL') + +// Input slot +const inputSlot = node.getInputSlot('model') +``` + +### Get Slot Position + +```typescript +const position = await outputSlot.getPosition() +// { x: number, y: number } +``` + +### Connect Slots + +```typescript +const sourceNode = comfyPage.getNodeRefByTitle('Load Checkpoint') +const targetNode = comfyPage.getNodeRefByTitle('KSampler') + +const outputSlot = sourceNode.getOutputSlot('MODEL') +const inputSlot = targetNode.getInputSlot('model') + +await comfyMouse.dragFromTo( + await outputSlot.getPosition(), + await inputSlot.getPosition(), + { steps: 10 } +) +await comfyPage.nextFrame() +``` + +## Widgets + +### Get Widget + +```typescript +const widget = node.getWidget('seed') +const stepsWidget = node.getWidget('steps') +``` + +### Set Widget Value + +```typescript +await widget.setValue(12345) +await comfyPage.nextFrame() +``` + +### Get Widget Value + +```typescript +const value = await widget.getValue() +expect(value).toBe(12345) +``` + +### Widget Types + +See [patterns/widgets.md](widgets.md) for type-specific patterns. + +## Node Assertions + +```typescript +// Visibility +await expect(node.locator).toBeVisible() + +// States (custom matchers) +await expect(node).toBeCollapsed() +await expect(node).toBeBypassed() +await expect(node).toBePinned() +``` + +## Example: Complete Node Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('Node Operations', { tag: ['@node'] }, () => { + test.beforeEach(async ({ comfyPage }) => { + await comfyPage.loadWorkflow('nodes/basic') + await comfyPage.nextFrame() + }) + + test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() + }) + + test('collapses and expands node', async ({ comfyPage }) => { + const node = comfyPage.getNodeRefByTitle('KSampler') + + await node.collapse() + await comfyPage.nextFrame() + await expect(node).toBeCollapsed() + + await node.expand() + await comfyPage.nextFrame() + await expect(node).not.toBeCollapsed() + }) + + test('connects two nodes', async ({ comfyPage }) => { + const source = comfyPage.getNodeRefByTitle('Load Checkpoint') + const target = comfyPage.getNodeRefByTitle('KSampler') + + await comfyMouse.dragFromTo( + await source.getOutputSlot('MODEL').getPosition(), + await target.getInputSlot('model').getPosition(), + { steps: 10 } + ) + await comfyPage.nextFrame() + + // Verify connection via screenshot or workflow state + await expect(comfyPage.canvas).toHaveScreenshot('connected.png') + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/core/setup.md b/.agents/skills/writing-playwright-tests/core/setup.md new file mode 100644 index 000000000..9afe45f0b --- /dev/null +++ b/.agents/skills/writing-playwright-tests/core/setup.md @@ -0,0 +1,123 @@ +# Test Setup Patterns + +## Essential Imports + +```typescript +// ALWAYS use these custom fixtures - never vanilla Playwright +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' +``` + +## Quick Start Template + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('FeatureName', { tag: ['@screenshot', '@canvas'] }, () => { + test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.UseNewMenu', 'Disabled') + }) + + test('should do something', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('myWorkflow') + await comfyPage.nextFrame() + + // Test logic here + + await expect(comfyPage.canvas).toHaveScreenshot('expected.png') + }) +}) +``` + +## Test Tags + +Add appropriate tags to every test: + +| Tag | When to Use | +| ------------- | ----------------------- | +| `@smoke` | Quick essential tests | +| `@slow` | Tests > 10 seconds | +| `@screenshot` | Visual regression tests | +| `@canvas` | Canvas interactions | +| `@node` | Node-related | +| `@widget` | Widget-related | +| `@mobile` | Mobile viewport tests | + +```typescript +test.describe('Feature', { tag: ['@screenshot', '@canvas'] }, () => { +``` + +## Common Settings + +```typescript +// Menu mode +await comfyPage.setSetting('Comfy.UseNewMenu', 'Disabled') +await comfyPage.setSetting('Comfy.UseNewMenu', 'Top') + +// Vue Nodes 2.0 +await comfyPage.setSetting('Comfy.VueNodes.Enabled', true) + +// UI elements +await comfyPage.setSetting('Comfy.Minimap.Visible', false) +await comfyPage.setSetting('Comfy.Graph.CanvasMenu', true) + +// Warnings +await comfyPage.setSetting('Comfy.Workflow.ShowMissingModelsWarning', false) + +// Locale +await comfyPage.setSetting('Comfy.Locale', 'fr') +``` + +## Loading Workflows + +```typescript +// Load from browser_tests/assets/ +await comfyPage.loadWorkflow('widgets/combo_widget') +await comfyPage.nextFrame() + +// Always use premade workflows, don't create programmatically +``` + +## Common Gotchas + +### 1. Missing `nextFrame()` + +Canvas changes don't render immediately: + +```typescript +await comfyPage.canvas.click(100, 200) +await comfyPage.nextFrame() // ← Required! +``` + +### 2. Double-Click Reliability + +```typescript +await element.dblclick({ delay: 5 }) +``` + +### 3. Screenshot Tests Are Linux-Only + +Don't commit local screenshots. Use `New Browser Test Expectations` PR label. + +### 4. Focus Before Keyboard + +```typescript +await comfyPage.canvas.focus() +await comfyPage.page.keyboard.press('Delete') +``` + +## Fresh Page Setup + +For tests that need clean state: + +```typescript +test('first-time user experience', async ({ comfyPage }) => { + await comfyPage.setup({ clearStorage: true }) + // localStorage/sessionStorage cleared +}) +``` diff --git a/.agents/skills/writing-playwright-tests/core/workflows.md b/.agents/skills/writing-playwright-tests/core/workflows.md new file mode 100644 index 000000000..ea9cdb57b --- /dev/null +++ b/.agents/skills/writing-playwright-tests/core/workflows.md @@ -0,0 +1,96 @@ +# Workflow Patterns + +## Loading Workflows + +### From Assets (Preferred) + +```typescript +// Loads browser_tests/assets/myWorkflow.json +await comfyPage.loadWorkflow('myWorkflow') + +// Subdirectories supported +await comfyPage.loadWorkflow('widgets/combo_widget') +await comfyPage.loadWorkflow('nodes/reroute') +await comfyPage.loadWorkflow('canvas/pan_zoom') +``` + +### Asset Directory Structure + +``` +browser_tests/assets/ +├── default.json # Basic starting workflow +├── canvas/ # Canvas state tests +├── groups/ # Group-related workflows +├── nodes/ # Node-specific workflows +├── widgets/ # Widget test workflows +├── workflows/ # Complex workflow scenarios +└── images/ # Image assets for drag-drop +``` + +### Creating New Assets + +1. Build the workflow in ComfyUI UI +2. Export as JSON +3. Save to appropriate `assets/` subdirectory +4. Reference by path (without `.json`) + +## Workflow State + +### Check Current Workflow + +```typescript +const workflow = await comfyPage.getWorkflow() +expect(workflow.nodes.length).toBe(3) +``` + +### Reset Workflow + +```typescript +// In afterEach to prevent test pollution +test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() +}) +``` + +### Clear Workflow + +```typescript +await comfyPage.page.keyboard.press('Control+a') +await comfyPage.page.keyboard.press('Delete') +await comfyPage.nextFrame() +``` + +## Workflow Validation + +```typescript +// Check node count +const nodes = await comfyPage.getNodes() +expect(nodes.length).toBe(5) + +// Check specific node exists +const node = comfyPage.getNodeRefByTitle('KSampler') +await expect(node.locator).toBeVisible() +``` + +## Example: Complete Workflow Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('Workflow Operations', { tag: ['@workflow'] }, () => { + test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() + }) + + test('loads workflow from asset', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('widgets/slider_widget') + await comfyPage.nextFrame() + + const node = comfyPage.getNodeRefByTitle('Preview Image') + await expect(node.locator).toBeVisible() + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/features/commands.md b/.agents/skills/writing-playwright-tests/features/commands.md new file mode 100644 index 000000000..182934c38 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/features/commands.md @@ -0,0 +1,112 @@ +# Command Store Patterns + +The command store allows executing and registering commands programmatically. + +## Executing Commands + +```typescript +// Execute a built-in command +await comfyPage.executeCommand('Comfy.QueuePrompt') +await comfyPage.executeCommand('Comfy.NewBlankWorkflow') +await comfyPage.executeCommand('Comfy.BrowseTemplates') +await comfyPage.executeCommand('Comfy.QueueSelectedOutputNodes') +``` + +## Common Built-in Commands + +| Command ID | Description | +| -------------------------------- | ---------------------------------- | +| `Comfy.QueuePrompt` | Queue workflow for execution | +| `Comfy.NewBlankWorkflow` | Clear canvas with new workflow | +| `Comfy.BrowseTemplates` | Open templates dialog | +| `Comfy.QueueSelectedOutputNodes` | Execute only selected output nodes | + +## Registering Custom Commands + +For testing command behavior: + +```typescript +// Register a sync command +await comfyPage.registerCommand('TestCommand', () => { + window['foo'] = true +}) +await comfyPage.executeCommand('TestCommand') +expect(await comfyPage.page.evaluate(() => window['foo'])).toBe(true) + +// Register an async command +await comfyPage.registerCommand('AsyncTestCommand', async () => { + await new Promise((resolve) => + setTimeout(() => { + window['bar'] = true + resolve() + }, 5) + ) +}) +``` + +## Testing Command Errors + +```typescript +await comfyPage.registerCommand('ErrorCommand', () => { + throw new Error('Test error') +}) + +await comfyPage.executeCommand('ErrorCommand') +expect(await comfyPage.getToastErrorCount()).toBe(1) +``` + +## Registering Keybindings + +```typescript +await comfyPage.registerKeybinding({ key: 'KeyT', ctrl: true }, () => { + window['keybindingTriggered'] = true +}) + +await comfyPage.page.keyboard.press('Control+t') +``` + +## How It Works + +Commands are registered via the extension system: + +```typescript +// Under the hood, registerCommand does this: +app.registerExtension({ + name: 'TestExtension_abc123', + commands: [ + { + id: 'TestCommand', + function: () => { + /* your function */ + } + } + ] +}) +``` + +## Example: Command Test + +```typescript +import { expect } from '@playwright/test' +import { comfyPageFixture as test } from '../fixtures/ComfyPage' + +test.describe('Commands', { tag: '@keyboard' }, () => { + test('Should execute command', async ({ comfyPage }) => { + await comfyPage.registerCommand('TestCommand', () => { + window['foo'] = true + }) + + await comfyPage.executeCommand('TestCommand') + expect(await comfyPage.page.evaluate(() => window['foo'])).toBe(true) + }) + + test('Should handle command errors', async ({ comfyPage }) => { + await comfyPage.registerCommand('TestCommand', () => { + throw new Error('Test error') + }) + + await comfyPage.executeCommand('TestCommand') + expect(await comfyPage.getToastErrorCount()).toBe(1) + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/features/execution.md b/.agents/skills/writing-playwright-tests/features/execution.md new file mode 100644 index 000000000..1361d0fdd --- /dev/null +++ b/.agents/skills/writing-playwright-tests/features/execution.md @@ -0,0 +1,177 @@ +# Execution & Workflow Testing Patterns + +Testing workflow execution and reading results. + +## Queueing Prompts + +```typescript +await comfyPage.executeCommand('Comfy.QueuePrompt') +``` + +## Testing Execution Errors + +```typescript +test('Report error on unconnected slot', async ({ comfyPage }) => { + await comfyPage.disconnectEdge() + await comfyPage.clickEmptySpace() + + await comfyPage.executeCommand('Comfy.QueuePrompt') + await expect(comfyPage.page.locator('.comfy-error-report')).toBeVisible() + + // Dismiss error dialog + await comfyPage.page.locator('.p-dialog-close-button').click() + await comfyPage.page.locator('.comfy-error-report').waitFor({ + state: 'hidden' + }) +}) +``` + +## Partial Execution (Selected Outputs) + +```typescript +test('Execute to selected output nodes', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('execution/partial_execution') + + const input = await comfyPage.getNodeRefById(3) + const output1 = await comfyPage.getNodeRefById(1) + const output2 = await comfyPage.getNodeRefById(4) + + // Select only output1 + await output1.click('title') + + // Execute selected outputs only + await comfyPage.executeCommand('Comfy.QueueSelectedOutputNodes') + + // Wait for execution with retry + await expect(async () => { + expect(await (await output1.getWidget(0)).getValue()).toBe('foo') + expect(await (await output2.getWidget(0)).getValue()).toBe('') + }).toPass({ timeout: 2_000 }) +}) +``` + +## Reading Node Outputs + +```typescript +// Access node outputs programmatically +await comfyPage.page.evaluate( + ([loadId, saveId]) => { + // Set output of save node to equal loader node's image + window['app'].nodeOutputs[saveId] = window['app'].nodeOutputs[loadId] + app.canvas.setDirty(true) + }, + [loadNodeId, saveNodeId] +) +``` + +## Simulating Execution Results + +For testing UI responses without actual execution: + +```typescript +// Inject fake output data +await comfyPage.page.evaluate((nodeId) => { + window['app'].nodeOutputs[nodeId] = { + images: [{ filename: 'test.png', subfolder: '', type: 'output' }] + } + app.canvas.setDirty(true) +}, nodeId) +``` + +## Waiting for Execution + +Use retry assertions for async execution results: + +```typescript +await expect(async () => { + const value = await widget.getValue() + expect(value).toBe('expected_result') +}).toPass({ timeout: 2_000 }) +``` + +## Testing Widget Callbacks + +Register callbacks to verify widget updates: + +```typescript +await comfyPage.page.evaluate(() => { + const widget = window['app'].graph.nodes[0].widgets[0] + widget.callback = (value: number) => { + window['widgetValue'] = value + } +}) + +// Perform action that triggers widget +await widget.dragHorizontal(50) + +// Verify callback was called +expect(await comfyPage.page.evaluate(() => window['widgetValue'])).toBeDefined() +``` + +## Graph State Verification + +```typescript +// Check node count +expect(await comfyPage.getGraphNodesCount()).toBe(5) + +// Check selected count +expect(await comfyPage.getSelectedGraphNodesCount()).toBe(2) + +// Check workflow modified status +expect(await comfyPage.isCurrentWorkflowModified()).toBe(false) +``` + +## Refreshing Combo Values + +Testing the 'R' key refresh behavior: + +```typescript +await comfyPage.loadWorkflow('inputs/optional_combo_input') + +const getComboValues = async () => + comfyPage.page.evaluate(() => { + return window['app'].graph.nodes + .find((node) => node.title === 'My Node') + .widgets.find((widget) => widget.name === 'combo_input').options.values + }) + +const initialValues = await getComboValues() + +// Focus canvas and press R to refresh +await comfyPage.page.mouse.click(400, 300) +await comfyPage.page.keyboard.press('r') +await comfyPage.page.waitForTimeout(500) + +const refreshedValues = await getComboValues() +expect(refreshedValues).not.toEqual(initialValues) +``` + +## Example: Complete Execution Test + +```typescript +import { expect } from '@playwright/test' +import { comfyPageFixture as test } from '../fixtures/ComfyPage' + +test.describe('Execution', { tag: ['@smoke', '@workflow'] }, () => { + test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.UseNewMenu', 'Disabled') + }) + + test( + 'Report error on unconnected slot', + { tag: '@screenshot' }, + async ({ comfyPage }) => { + await comfyPage.disconnectEdge() + await comfyPage.clickEmptySpace() + + await comfyPage.executeCommand('Comfy.QueuePrompt') + await expect(comfyPage.page.locator('.comfy-error-report')).toBeVisible() + + await comfyPage.page.locator('.p-dialog-close-button').click() + await expect(comfyPage.canvas).toHaveScreenshot( + 'execution-error-unconnected-slot.png' + ) + } + ) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/features/templates.md b/.agents/skills/writing-playwright-tests/features/templates.md new file mode 100644 index 000000000..dafc9e6bf --- /dev/null +++ b/.agents/skills/writing-playwright-tests/features/templates.md @@ -0,0 +1,166 @@ +# Templates Patterns + +Testing the templates dialog and workflow loading from templates. + +## Opening Templates Dialog + +```typescript +await comfyPage.executeCommand('Comfy.BrowseTemplates') +await expect(comfyPage.templates.content).toBeVisible() +``` + +## Templates API + +Access via `comfyPage.templates`: + +```typescript +// Get all templates +const templates = await comfyPage.templates.getAllTemplates() + +// Load a specific template +await comfyPage.templates.loadTemplate('default') + +// Wait for cards to render +await comfyPage.templates.waitForMinimumCardCount(1) + +// Check content visibility +await expect(comfyPage.templates.content).toBeVisible() +await expect(comfyPage.templates.content).toBeHidden() +``` + +## Template Test Setup + +```typescript +test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.UseNewMenu', 'Top') + await comfyPage.setSetting('Comfy.Workflow.ShowMissingModelsWarning', false) +}) +``` + +## Loading Templates + +```typescript +// Clear workflow first +await comfyPage.menu.workflowsTab.open() +await comfyPage.executeCommand('Comfy.NewBlankWorkflow') +await expect(async () => { + expect(await comfyPage.getGraphNodesCount()).toBe(0) +}).toPass({ timeout: 250 }) + +// Load template +await comfyPage.executeCommand('Comfy.BrowseTemplates') +await comfyPage.page.getByRole('button', { name: 'Getting Started' }).click() +await comfyPage.templates.loadTemplate('default') + +// Verify nodes loaded +await expect(async () => { + expect(await comfyPage.getGraphNodesCount()).toBeGreaterThan(0) +}).toPass({ timeout: 250 }) +``` + +## Testing Template Localization + +```typescript +test('Uses proper locale files', async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.Locale', 'fr') + + await comfyPage.executeCommand('Comfy.BrowseTemplates') + + const dialog = comfyPage.page.getByRole('dialog').filter({ + has: comfyPage.page.getByRole('heading', { name: 'Modèles', exact: true }) + }) + await expect(dialog).toBeVisible() + + // Verify French-localized strings + await expect( + dialog.getByRole('heading', { name: 'Modèles', exact: true }) + ).toBeVisible() +}) +``` + +## Mocking Template Responses + +For testing template display with controlled data: + +```typescript +await comfyPage.page.route('**/templates/index.json', async (route, _) => { + const response = [ + { + moduleName: 'default', + title: 'Test Templates', + type: 'image', + templates: [ + { + name: 'short-description', + title: 'Short Description', + mediaType: 'image', + mediaSubtype: 'webp', + description: 'This is a short description.' + } + ] + } + ] + await route.fulfill({ + status: 200, + body: JSON.stringify(response), + headers: { + 'Content-Type': 'application/json', + 'Cache-Control': 'no-store' + } + }) +}) + +// Mock thumbnail images to avoid 404s +await comfyPage.page.route('**/templates/**.webp', async (route) => { + await route.fulfill({ + status: 200, + path: 'browser_tests/assets/example.webp', + headers: { + 'Content-Type': 'image/webp', + 'Cache-Control': 'no-store' + } + }) +}) +``` + +## Testing First-Time User Experience + +```typescript +test('dialog shown to first-time users', async ({ comfyPage }) => { + // Mark as first-time user + await comfyPage.setSetting('Comfy.TutorialCompleted', false) + + // Reload page + await comfyPage.setup({ clearStorage: true }) + + // Templates dialog should auto-show + expect(await comfyPage.templates.content.isVisible()).toBe(true) +}) +``` + +## Testing 404 Fallback + +```typescript +test('Falls back to English when locale file not found', async ({ + comfyPage +}) => { + await comfyPage.setSetting('Comfy.Locale', 'de') + + // Intercept German file to simulate 404 + await comfyPage.page.route('**/templates/index.de.json', async (route) => { + await route.fulfill({ + status: 404, + headers: { 'Content-Type': 'text/plain' }, + body: 'Not Found' + }) + }) + + // Allow English fallback + await comfyPage.page.route('**/templates/index.json', (route) => + route.continue() + ) + + await comfyPage.executeCommand('Comfy.BrowseTemplates') + await expect(comfyPage.templates.content).toBeVisible() +}) +``` diff --git a/.agents/skills/writing-playwright-tests/features/vue-nodes.md b/.agents/skills/writing-playwright-tests/features/vue-nodes.md new file mode 100644 index 000000000..2a202ece3 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/features/vue-nodes.md @@ -0,0 +1,134 @@ +# Vue Nodes 2.0 Patterns + +Vue Nodes 2.0 is an alternative rendering mode for nodes. Tests need to explicitly enable it. + +## Enabling Vue Nodes + +```typescript +test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.VueNodes.Enabled', true) + await comfyPage.vueNodes.waitForNodes() +}) +``` + +## VueNodeHelpers API + +Access via `comfyPage.vueNodes`: + +```typescript +// Get node by title +const node = comfyPage.vueNodes.getNodeByTitle('Load Checkpoint') + +// Get node count +const count = await comfyPage.vueNodes.getNodeCount() + +// Get selected node count +const selectedCount = await comfyPage.vueNodes.getSelectedNodeCount() + +// Wait for nodes to render +await comfyPage.vueNodes.waitForNodes() +await comfyPage.vueNodes.waitForNodes(5) // Wait for at least 5 nodes + +// Enter a subgraph +await comfyPage.vueNodes.enterSubgraph() +``` + +## Selecting Vue Nodes + +```typescript +// Click node by title (header text) +await comfyPage.page.getByText('Load Checkpoint').click() + +// Multi-select with modifier keys +await comfyPage.page.getByText('KSampler').click({ modifiers: ['Control'] }) + +// Select all with keyboard +await comfyPage.canvas.press('Control+a') +``` + +## Node State Testing + +Vue Nodes use CSS classes to indicate state: + +```typescript +// Bypass state +const BYPASS_CLASS = /before:bg-bypass\/60/ +const node = comfyPage.vueNodes.getNodeByTitle('Load Checkpoint') +await expect(node).toHaveClass(BYPASS_CLASS) + +// Selection state (outline) +await expect(node).toHaveClass(/outline-node-component-outline/) +``` + +## Vue Node Hotkeys + +```typescript +await comfyPage.page.getByText('Load Checkpoint').click() +await comfyPage.page.keyboard.press('Control+b') // Bypass +await comfyPage.page.keyboard.press('p') // Pin +await comfyPage.page.keyboard.press('m') // Mute +``` + +## DOM-based Widget Access + +Vue Nodes use DOM elements instead of canvas rendering: + +```typescript +// Get widget by name +const widget = comfyPage.vueNodes.getWidgetByName('KSampler', 'seed') + +// Get input number controls +const controls = comfyPage.vueNodes.getInputNumberControls(widget) +await controls.incrementButton.click() +await controls.decrementButton.click() +await controls.input.fill('42') + +// Widget reactivity test pattern +const widgetLocator = comfyPage.page.locator( + 'css=[data-testid="node-body-4"] > .lg-node-widgets > div' +) +await expect(widgetLocator).toHaveCount(2) +``` + +## VueNodeFixture Pattern + +For more complex DOM interactions: + +```typescript +const fixture = await comfyPage.vueNodes.getFixtureByTitle('Load Checkpoint') +// Fixture maintains stable reference even if title changes +``` + +## Common Vue Nodes Settings + +```typescript +await comfyPage.setSetting('Comfy.VueNodes.Enabled', true) +await comfyPage.setSetting('Comfy.UseNewMenu', 'Top') +await comfyPage.setSetting('Comfy.Minimap.Visible', false) +await comfyPage.setSetting('Comfy.Graph.CanvasMenu', true) +``` + +## Example: Complete Vue Node Test + +```typescript +import { + comfyExpect as expect, + comfyPageFixture as test +} from '../../../fixtures/ComfyPage' + +test.describe('Vue Node Feature', () => { + test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.VueNodes.Enabled', true) + await comfyPage.setSetting('Comfy.UseNewMenu', 'Top') + await comfyPage.vueNodes.waitForNodes() + }) + + test('should toggle bypass state', async ({ comfyPage }) => { + await comfyPage.page.getByText('Load Checkpoint').click() + await comfyPage.page.keyboard.press('Control+b') + + const node = comfyPage.vueNodes.getNodeByTitle('Load Checkpoint') + await expect(node).toHaveClass(/before:bg-bypass\/60/) + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/reference/cheatsheet.md b/.agents/skills/writing-playwright-tests/reference/cheatsheet.md new file mode 100644 index 000000000..74d86347d --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/cheatsheet.md @@ -0,0 +1,235 @@ +# Playwright Test Cheatsheet + +Quick reference for common operations. See linked files for details. + +## Imports + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' +``` + +## Test Structure + +```typescript +test.describe('Feature', { tag: ['@screenshot', '@canvas'] }, () => { + test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() + }) + + test('does something', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('myWorkflow') + await comfyPage.nextFrame() + // test logic + }) +}) +``` + +## Workflows + +| Operation | Code | +| ---------------- | ------------------------------------------------------------- | +| Load from assets | `await comfyPage.loadWorkflow('widgets/slider')` | +| Load default | `await comfyPage.executeCommand('Comfy.LoadDefaultWorkflow')` | +| Reset view | `await comfyPage.resetView()` | + +## Canvas + +| Operation | Code | +| ------------ | ----------------------------------------------------------- | +| Click | `await comfyPage.canvas.click({ position: { x, y } })` | +| Double-click | `await comfyPage.page.mouse.dblclick(x, y, { delay: 5 })` | +| Pan | `await comfyPage.pan({ x: 100, y: 100 })` | +| Zoom | `await comfyPage.zoom(-100)` / `zoom(100)` | +| Next frame | `await comfyPage.nextFrame()` ← **Always after canvas ops** | +| Focus | `await comfyPage.canvas.click()` ← **Before keyboard** | + +## Keyboard Shortcuts + +```typescript +await comfyPage.canvas.click() // Focus first! +await comfyPage.canvas.press('Control+a') +await comfyPage.canvas.press('Delete') +``` + +| Shortcut | Helper | +| ---------- | ------------------------- | +| Select all | `await comfyPage.ctrlA()` | +| Copy | `await comfyPage.ctrlC()` | +| Paste | `await comfyPage.ctrlV()` | +| Undo | `await comfyPage.ctrlZ()` | +| Redo | `await comfyPage.ctrlY()` | +| Bypass | `await comfyPage.ctrlB()` | + +## Nodes + +| Operation | Code | +| ------------ | ---------------------------------------------------- | +| By type | `(await comfyPage.getNodeRefsByType('KSampler'))[0]` | +| By title | `(await comfyPage.getNodeRefsByTitle('My Node'))[0]` | +| By ID | `await comfyPage.getNodeRefById(3)` | +| First node | `await comfyPage.getFirstNodeRef()` | +| Click title | `await node.click('title')` | +| Right-click | `await node.click('title', { button: 'right' })` | +| Collapse | `await node.click('collapse')` | +| Get position | `await node.getPosition()` | +| Get size | `await node.getSize()` | +| Context menu | `await node.clickContextMenuOption('Colors')` | + +## Widgets + +```typescript +const widget = await node.getWidget(0) // by index +await widget.click() +await widget.dragHorizontal(50) // for sliders +const value = await widget.getValue() +``` + +## Connections + +```typescript +await node1.connectOutput(0, node2, 0) // output slot to input slot +await node1.connectWidget(0, node2, 0) // output to widget +await comfyPage.disconnectEdge() +await comfyPage.connectEdge() +``` + +## Settings + +```typescript +await comfyPage.setSetting('Comfy.UseNewMenu', 'Top') +await comfyPage.setSetting('Comfy.EnableTooltips', false) +const val = await comfyPage.getSetting('Comfy.UseNewMenu') +``` + +## Search Box + +```typescript +await comfyPage.doubleClickCanvas() // opens search box +await comfyPage.searchBox.fillAndSelectFirstNode('KSampler') +await comfyPage.searchBox.addFilter('MODEL', 'Input Type') +``` + +## Sidebar + +```typescript +// Workflows tab +const tab = comfyPage.menu.workflowsTab +await tab.open() +await tab.switchToWorkflow('workflow1') +const names = await tab.getOpenedWorkflowNames() + +// Node library +const lib = comfyPage.menu.nodeLibraryTab +await lib.open() +await lib.nodeLibrarySearchBoxInput.fill('KSampler') + +// Topbar +await comfyPage.menu.topbar.saveWorkflow('file.json') +``` + +## Dialogs + +```typescript +// Confirm dialog +await comfyPage.confirmDialog.confirm() +await comfyPage.confirmDialog.cancel() + +// Settings dialog +await comfyPage.settingDialog.open() +await comfyPage.settingDialog.close() + +// LiteGraph prompt dialog +await comfyPage.page.waitForSelector('.graphdialog input', { state: 'visible' }) +await comfyPage.page.fill('.graphdialog input', 'value') +await comfyPage.page.keyboard.press('Enter') +``` + +## Assertions + +```typescript +// Standard Playwright +await expect(element).toBeVisible() +await expect(page).toHaveURL('/path') + +// Custom ComfyUI matchers +await expect(node).toBeCollapsed() +await expect(node).toBeBypassed() +await expect(node).toBePinned() + +// Screenshots +await expect(comfyPage.canvas).toHaveScreenshot('name.png') +``` + +## Mocking + +```typescript +// API route +await comfyPage.page.route('**/api/queue', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify({ data: 'mocked' }) + }) +) + +// Block resources +await context.route('**/*.{png,jpg}', (route) => route.abort()) +``` + +## File Drag-and-Drop + +```typescript +await comfyPage.dragAndDropFile('browser_tests/assets/images/test.png', { + x: 100, + y: 100 +}) +``` + +## Subgraphs + +```typescript +await subgraphNode.navigateIntoSubgraph() +await comfyPage.page.keyboard.press('Escape') // exit +await comfyPage.connectFromSubgraphInput(node, 0) +await comfyPage.connectToSubgraphOutput(node, 0) +``` + +## Page Evaluation + +```typescript +// Run code in browser context +await comfyPage.page.evaluate(() => { + window['app'].graph.setDirtyCanvas(true, true) +}) + +// Wait for condition +await comfyPage.page.waitForFunction( + () => window['app']?.extensionManager?.workflow?.isBusy === false +) +``` + +## Tags Quick Reference + +| Tag | Purpose | Run with | +| ------------- | -------------------- | ------------------------- | +| `@smoke` | Fast essential tests | `--grep @smoke` | +| `@slow` | Long-running tests | `--grep-invert @slow` | +| `@screenshot` | Visual regression | `--grep @screenshot` | +| `@mobile` | Mobile viewport | `--project=mobile-chrome` | +| `@2x` | HiDPI scale | `--project=chromium-2x` | +| `@canvas` | Canvas tests | `--grep @canvas` | +| `@node` | Node tests | `--grep @node` | +| `@widget` | Widget tests | `--grep @widget` | + +## Run Commands + +```bash +pnpm exec playwright test --ui # UI mode (recommended) +pnpm test:browser:local # Local with traces +pnpm exec playwright test --grep @smoke # By tag +pnpm exec playwright test --grep-invert @screenshot # Skip screenshots +pnpm exec playwright test mytest.spec.ts:25 # Specific line +pnpm exec playwright test --debug # Debug mode +``` diff --git a/.agents/skills/writing-playwright-tests/reference/config.md b/.agents/skills/writing-playwright-tests/reference/config.md new file mode 100644 index 000000000..82f49f32e --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/config.md @@ -0,0 +1,178 @@ +# Configuration Reference + +## Playwright Config + +**File:** `playwright.config.ts` + +### Key Settings + +```typescript +{ + testDir: './browser_tests', + fullyParallel: true, + forbidOnly: !!process.env.CI, // Fail if .only() left in + + globalSetup: './browser_tests/globalSetup.ts', + globalTeardown: './browser_tests/globalTeardown.ts', + + // CI vs Local + retries: process.env.CI ? 3 : 0, + trace: process.env.CI ? 'on-first-retry' : 'on', +} +``` + +### Projects + +| Project | Viewport | Tags | Use Case | +| --------------- | ---------- | ------------------ | -------------- | +| `chromium` | Default | Excludes `@mobile` | Standard tests | +| `chromium-2x` | 2x scale | `@2x` only | HiDPI tests | +| `chromium-0.5x` | 0.5x scale | `@0.5x` only | Low DPI tests | +| `mobile-chrome` | Pixel 5 | `@mobile` only | Mobile tests | + +### Timeouts + +| Setting | Value | +| ---------------------- | ---------- | +| Project timeout | 15 seconds | +| Global timeout (local) | 30 seconds | + +## Environment Variables + +| Variable | Purpose | Default | +| --------------------- | ----------------------- | ----------------------- | +| `CI` | Detected automatically | - | +| `PLAYWRIGHT_LOCAL` | Enable local dev mode | unset | +| `PLAYWRIGHT_TEST_URL` | Target URL | `http://localhost:8188` | +| `TEST_COMFYUI_DIR` | ComfyUI path for backup | unset | + +## Running Tests + +### Local Development + +```bash +# UI mode (recommended) +pnpm exec playwright test --ui + +# With local settings (more traces) +pnpm test:browser:local + +# Headless +pnpm test:browser +``` + +### Specific Tests + +```bash +# By file +pnpm exec playwright test canvas.spec.ts + +# By line +pnpm exec playwright test canvas.spec.ts:25 + +# By grep +pnpm exec playwright test --grep "should pan" +``` + +### By Tag + +```bash +# Run smoke tests +pnpm exec playwright test --grep @smoke + +# Skip screenshots +pnpm exec playwright test --grep-invert @screenshot + +# Combine +pnpm exec playwright test --grep @canvas --grep-invert @slow +``` + +### By Project + +```bash +pnpm exec playwright test --project=chromium +pnpm exec playwright test --project=mobile-chrome +``` + +## CI Configuration + +**File:** `.github/workflows/ci-tests-e2e.yaml` + +### Workflow Triggers + +- Push to `main`/`master`/`core/*`/`desktop/*` +- Pull requests (except `wip/*`/`draft/*`/`temp/*`) + +### Sharding + +Chromium tests split across **8 shards** for parallelism. + +Other projects run unsharded. + +### Server Startup + +Backend runs with `--multi-user` flag: + +```bash +python3 main.py --cpu --multi-user --front-end-root "$GITHUB_WORKSPACE/dist" +``` + +### Artifacts + +- Blob reports merged to HTML +- Deployed to Cloudflare Pages +- Link posted in PR comment + +## Updating Screenshots + +### In CI + +Add PR label: `New Browser Test Expectations` + +Workflow will: + +1. Run tests with `--update-snapshots` +2. Commit updated baselines +3. Push to branch + +### Locally + +**Don't do this** - screenshots are Linux-only. + +If you must: + +```bash +pnpm exec playwright test --update-snapshots +# Don't commit these! +``` + +## Global Setup/Teardown + +### globalSetup.ts + +Backs up user data before tests run (local only). + +### globalTeardown.ts + +Restores user data after tests complete (local only). + +Only runs when `TEST_COMFYUI_DIR` is set. + +## Package Scripts + +```json +{ + "test:browser": "playwright test", + "test:browser:local": "PLAYWRIGHT_LOCAL=1 playwright test" +} +``` + +## Configuration Files + +| File | Purpose | +| --------------------------------------- | --------------------- | +| `playwright.config.ts` | Main config | +| `browser_tests/globalSetup.ts` | Pre-test setup | +| `browser_tests/globalTeardown.ts` | Post-test cleanup | +| `.github/workflows/ci-tests-e2e.yaml` | CI workflow | +| `.github/actions/start-comfyui-server/` | Server startup action | diff --git a/.agents/skills/writing-playwright-tests/reference/debugging.md b/.agents/skills/writing-playwright-tests/reference/debugging.md new file mode 100644 index 000000000..2c8aeaa22 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/debugging.md @@ -0,0 +1,225 @@ +# Debugging Flaky Tests + +## Common Causes & Fixes + +### 1. Missing `nextFrame()` + +**Symptom:** Test passes locally, fails in CI; screenshot mismatches + +**Fix:** Add `nextFrame()` after canvas operations: + +```typescript +await comfyPage.canvas.click(100, 200) +await comfyPage.nextFrame() // ← Add this +``` + +### 2. Race Conditions + +**Symptom:** Intermittent failures, "element not found" + +**Fix:** Use proper waits instead of timeouts: + +```typescript +// ❌ Bad +await page.waitForTimeout(500) + +// ✅ Good +await expect(element).toBeVisible() +``` + +### 3. Missing Focus + +**Symptom:** Keyboard shortcuts don't work + +**Fix:** Focus canvas before keyboard events: + +```typescript +await comfyPage.canvas.focus() // ← Add this +await comfyPage.page.keyboard.press('Delete') +``` + +### 4. Double-Click Timing + +**Symptom:** Double-click doesn't trigger edit mode + +**Fix:** Add delay option: + +```typescript +await element.dblclick({ delay: 5 }) +``` + +### 5. Upload Not Complete + +**Symptom:** Widget value incorrect after file upload + +**Fix:** Wait for upload response: + +```typescript +await comfyPage.dragAndDropFile(path, position, { waitForUpload: true }) +``` + +### 6. Tests Polluting Each Other + +**Symptom:** Test fails when run with others, passes alone + +**Fix:** Reset state in afterEach: + +```typescript +test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() +}) +``` + +### 7. Animation Not Complete + +**Symptom:** Screenshot mismatch, elements in wrong position + +**Fix:** Add steps to drag operations: + +```typescript +await comfyMouse.dragFromTo(start, end, { steps: 10 }) // Not steps: 1 +await comfyPage.nextFrame() +``` + +## Debugging Tools + +### Debug Mode + +```bash +npx playwright test --debug +npx playwright test mytest.spec.ts:25 --debug # Specific line +``` + +### Pause in Test + +```typescript +await page.pause() // Opens inspector +``` + +### Trace Viewer + +```bash +npx playwright test --trace on +npx playwright show-report +``` + +### Verbose Logging + +```bash +DEBUG=pw:api npx playwright test +``` + +### UI Mode (Recommended) + +```bash +pnpm exec playwright test --ui +``` + +## Retry Pattern + +For inherently async operations: + +```typescript +await expect(async () => { + const value = await widget.getValue() + expect(value).toBe(100) +}).toPass({ timeout: 2000 }) +``` + +## Debugging Screenshots + +### Local vs CI Mismatch + +Screenshots are Linux-only. Don't commit local screenshots. + +### Update Baselines + +Use PR label: `New Browser Test Expectations` + +Or locally: + +```bash +pnpm exec playwright test --update-snapshots +``` + +### Mask Dynamic Content + +```typescript +await expect(page).toHaveScreenshot('page.png', { + mask: [page.locator('.timestamp'), page.locator('.random-id')] +}) +``` + +## Debugging Workflow Issues + +### Log Workflow State + +```typescript +const workflow = await comfyPage.getWorkflow() +console.log(JSON.stringify(workflow, null, 2)) +``` + +### Check Node Count + +```typescript +const nodes = await comfyPage.getNodes() +console.log('Node count:', nodes.length) +``` + +## CI Debugging + +### View Traces + +1. Go to failed CI run +2. Download artifacts +3. Extract and open in trace viewer: + ```bash + npx playwright show-trace trace.zip + ``` + +### View Report + +CI deploys report to Cloudflare Pages. Link in PR comment. + +### Reproduce CI Environment + +```bash +# Run with CI settings +CI=true pnpm test:browser + +# Run specific shard +CI=true npx playwright test --shard=1/8 +``` + +## Known Issues & Workarounds + +### LiteGraph Click Handler Delay + +LiteGraph uses 256ms setTimeout for click handlers: + +```typescript +// Acceptable workaround +await page.waitForTimeout(300) // Documented exception +``` + +### Version Mismatch Warning + +Disable in tests via setting: + +```typescript +await comfyPage.setSetting('Comfy.Frontend.DisableVersionWarning', true) +``` + +## When to Use Retry vs Fix + +**Use Retry:** + +- External service timing +- Animation completion +- Network latency + +**Fix Root Cause:** + +- Missing awaits +- Missing nextFrame() +- Race conditions in test logic diff --git a/.agents/skills/writing-playwright-tests/reference/examples.md b/.agents/skills/writing-playwright-tests/reference/examples.md new file mode 100644 index 000000000..915cec9fe --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/examples.md @@ -0,0 +1,115 @@ +# Example Test Files + +Real test files to reference when writing new tests. **Always check these first.** + +## By Feature + +| Feature | File | Key Patterns | +| ----------------------- | ----------------------------------------------- | ------------------------------------- | +| Widget interactions | `browser_tests/tests/widget.spec.ts` | Boolean, slider, image, combo widgets | +| Node selection/dragging | `browser_tests/tests/interaction.spec.ts` | Click, drag, multi-select | +| Copy/paste | `browser_tests/tests/copyPaste.spec.ts` | Ctrl+C/V, clipboard | +| Node search box | `browser_tests/tests/nodeSearchBox.spec.ts` | Filters, search, node creation | +| Settings/themes | `browser_tests/tests/colorPalette.spec.ts` | setSetting, color changes | +| Templates | `browser_tests/tests/templates.spec.ts` | Route mocking, template loading | +| Execution | `browser_tests/tests/execution.spec.ts` | Queue prompt, WebSocket | +| Subgraphs | `browser_tests/tests/subgraph.spec.ts` | Navigation, I/O slots | +| Undo/redo | `browser_tests/tests/changeTracker.spec.ts` | Change transactions | +| Keybindings | `browser_tests/tests/keybindings.spec.ts` | Custom shortcuts | +| Workflow sidebar | `browser_tests/tests/sidebar/workflows.spec.ts` | Save, rename, switch | +| Graph validation | `browser_tests/tests/graph.spec.ts` | Link fixing, validation | + +## By Pattern + +### Loading Workflows + +```bash +grep -r "loadWorkflow" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +### Canvas Interactions + +```bash +grep -r "nextFrame" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +### Node References + +```bash +grep -r "getNodeRefsByType\|getNodeRefById" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +### Settings Changes + +```bash +grep -r "setSetting" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +### Mocking API Routes + +```bash +grep -r "page.route" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +### Screenshot Tests + +```bash +grep -r "toHaveScreenshot" browser_tests/tests/ --include="*.spec.ts" | head -10 +``` + +## Key Fixture Files + +| Purpose | Path | +| --------------- | --------------------------------------------------------- | +| Main fixture | `browser_tests/fixtures/ComfyPage.ts` | +| Mouse helper | `browser_tests/fixtures/ComfyMouse.ts` | +| Node reference | `browser_tests/fixtures/utils/litegraphUtils.ts` | +| Search box | `browser_tests/fixtures/components/ComfyNodeSearchBox.ts` | +| Sidebar tab | `browser_tests/fixtures/components/SidebarTab.ts` | +| Settings dialog | `browser_tests/fixtures/components/SettingDialog.ts` | +| Action bar | `browser_tests/helpers/actionbar.ts` | +| Templates | `browser_tests/helpers/templates.ts` | + +## Finding Similar Tests + +Before writing a new test, search for similar patterns: + +```bash +# Find tests for a specific node type +grep -r "KSampler\|CLIPTextEncode" browser_tests/tests/ + +# Find tests using specific fixture method +grep -r "connectOutput\|connectWidget" browser_tests/ + +# Find tests with specific tag +grep -r '@screenshot\|@mobile' browser_tests/tests/ + +# Find tests that mock API +grep -r "route.*fulfill" browser_tests/tests/ +``` + +## Asset Examples + +``` +browser_tests/assets/ +├── default.json # Basic workflow +├── widgets/ +│ ├── boolean_widget.json # Toggle widgets +│ ├── slider_widget.json # Slider/number widgets +│ └── combo_widget.json # Dropdown widgets +├── nodes/ +│ ├── single_ksampler.json # Single node workflow +│ └── primitive.json # Primitive node tests +├── subgraphs/ +│ └── basic-subgraph.json # Subgraph navigation +└── execution/ + └── simple.json # Execution tests +``` + +## Documentation Files + +| File | Contains | +| ------------------------- | ----------------------- | +| `browser_tests/README.md` | Overview, running tests | +| `browser_tests/AGENTS.md` | Agent-specific guidance | +| `browser_tests/CLAUDE.md` | Claude-specific context | diff --git a/.agents/skills/writing-playwright-tests/reference/fixtures.md b/.agents/skills/writing-playwright-tests/reference/fixtures.md new file mode 100644 index 000000000..48a3f9da2 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/fixtures.md @@ -0,0 +1,185 @@ +# Fixtures Reference + +## Core Fixtures + +### comfyPage + +The main page object with all helpers. Extends Playwright's Page. + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test('example', async ({ comfyPage }) => { + // comfyPage is available +}) +``` + +### comfyMouse + +Mouse interaction helper for canvas operations. + +```typescript +test('drag example', async ({ comfyPage, comfyMouse }) => { + await comfyMouse.dragFromTo(start, end, { steps: 10 }) +}) +``` + +## comfyPage Methods + +### Navigation & Workflow + +| Method | Description | +| -------------------- | -------------------------- | +| `loadWorkflow(name)` | Load workflow from assets | +| `getWorkflow()` | Get current workflow state | +| `resetView()` | Reset canvas view | +| `reload()` | Reload the page | + +### Canvas + +| Method | Description | +| --------------- | -------------------------- | +| `canvas` | The canvas locator | +| `nextFrame()` | Wait for next render frame | +| `pan({ x, y })` | Pan the canvas | +| `zoom(factor)` | Zoom the canvas | + +### Nodes + +| Method | Description | +| -------------------------- | ------------------------- | +| `getNodeRefByTitle(title)` | Get node by display title | +| `getNodeRefById(id)` | Get node by numeric ID | +| `getFirstNode()` | Get first node | +| `getLastNode()` | Get last node | +| `getNodes()` | Get all nodes | + +### Settings + +| Method | Description | +| ------------------------ | --------------------- | +| `setSetting(key, value)` | Set a ComfyUI setting | +| `getSetting(key)` | Get a setting value | + +### Files + +| Method | Description | +| ------------------------------------------- | --------------------- | +| `dragAndDropFile(path, position, options?)` | Drag file onto canvas | + +### Execution + +| Method | Description | +| --------------- | ---------------------- | +| `queuePrompt()` | Queue current workflow | +| `interrupt()` | Interrupt execution | + +## NodeReference Methods + +Returned by `getNodeRefByTitle()`, `getNodeRefById()`. + +| Method | Description | +| --------------------- | --------------------------- | +| `click()` | Click the node | +| `drag({ x, y })` | Drag node by offset | +| `collapse()` | Collapse node | +| `expand()` | Expand node | +| `bypass()` | Bypass node | +| `pin()` | Pin node | +| `getWidget(name)` | Get widget by name | +| `getInputSlot(name)` | Get input slot | +| `getOutputSlot(name)` | Get output slot | +| `locator` | Playwright Locator for node | + +## SlotReference Methods + +Returned by `getInputSlot()`, `getOutputSlot()`. + +| Method | Description | +| --------------- | ---------------------------- | +| `getPosition()` | Get slot position `{ x, y }` | +| `click()` | Click the slot | + +## WidgetReference Methods + +Returned by `getWidget()`. + +| Method | Description | +| ----------------- | ------------------ | +| `setValue(value)` | Set widget value | +| `getValue()` | Get current value | +| `click()` | Click widget | +| `locator` | Playwright Locator | + +## comfyMouse Methods + +| Method | Description | +| ---------------------------------- | ---------------------- | +| `dragFromTo(start, end, options?)` | Drag from start to end | +| `move(position)` | Move mouse to position | +| `click(position)` | Click at position | + +### dragFromTo Options + +```typescript +{ + steps?: number, // Number of intermediate steps (default: 1) + button?: 'left' | 'right' | 'middle' +} +``` + +## Custom Assertions (comfyExpect) + +```typescript +import { comfyExpect as expect } from './fixtures/ComfyPage' + +// Node state assertions +await expect(node).toBeCollapsed() +await expect(node).toBeBypassed() +await expect(node).toBePinned() + +// Negation +await expect(node).not.toBeCollapsed() +``` + +## Component Fixtures + +Located in `fixtures/components/`: + +### Topbar + +```typescript +const topbar = comfyPage.topbar +await topbar.clickMenu('File') +await topbar.clickMenuItem('Save') +``` + +### SidebarTab + +```typescript +const sidebar = comfyPage.sidebar +await sidebar.open('nodes') +await sidebar.search('KSampler') +``` + +### SearchBox + +```typescript +await comfyPage.searchBox.open() +await comfyPage.searchBox.search('Load Checkpoint') +await comfyPage.searchBox.selectFirst() +``` + +## Source Files + +| Fixture | File | +| ------------- | ------------------------------------------------- | +| ComfyPage | `browser_tests/fixtures/ComfyPage.ts` | +| ComfyMouse | `browser_tests/fixtures/ComfyMouse.ts` | +| NodeReference | `browser_tests/fixtures/NodeReference.ts` | +| Topbar | `browser_tests/fixtures/components/Topbar.ts` | +| SidebarTab | `browser_tests/fixtures/components/SidebarTab.ts` | +| utils | `browser_tests/fixtures/utils.ts` | diff --git a/.agents/skills/writing-playwright-tests/reference/playwright-best-practices.md b/.agents/skills/writing-playwright-tests/reference/playwright-best-practices.md new file mode 100644 index 000000000..670f1412e --- /dev/null +++ b/.agents/skills/writing-playwright-tests/reference/playwright-best-practices.md @@ -0,0 +1,219 @@ +# Playwright Best Practices + +Official Playwright patterns. ComfyUI-specific patterns in [../SKILL.md](../SKILL.md). + +## Locator Priority + +Use locators in this order of preference: + +| Priority | Method | Use Case | +| -------- | ------------------------- | ---------------------------------------- | +| 1 | `page.getByRole()` | Buttons, links, headings (accessibility) | +| 2 | `page.getByLabel()` | Form controls with labels | +| 3 | `page.getByPlaceholder()` | Inputs by placeholder | +| 4 | `page.getByText()` | Elements by text content | +| 5 | `page.getByAltText()` | Images by alt text | +| 6 | `page.getByTitle()` | Elements by title attribute | +| 7 | `page.getByTestId()` | `data-testid` fallback | + +```typescript +// ✅ Good - uses role and accessible name +await page.getByRole('button', { name: 'Submit' }).click() + +// ❌ Bad - tied to implementation details +await page.locator('button.btn-primary.submit-btn').click() +``` + +## Web-First Assertions + +**Always use auto-retrying assertions:** + +```typescript +// ✅ Good - auto-waits and retries +await expect(page.getByText('Welcome')).toBeVisible() +await expect(page).toHaveTitle(/Dashboard/) + +// ❌ Bad - doesn't wait, leads to flaky tests +expect(await page.getByText('Welcome').isVisible()).toBe(true) +``` + +### Auto-Retrying Assertions + +| Assertion | Purpose | +| ------------------- | ------------------- | +| `toBeVisible()` | Element is visible | +| `toBeHidden()` | Element is hidden | +| `toBeEnabled()` | Element is enabled | +| `toBeDisabled()` | Element is disabled | +| `toHaveText()` | Exact text match | +| `toContainText()` | Partial text match | +| `toHaveValue()` | Input value | +| `toHaveAttribute()` | Attribute value | +| `toHaveCount()` | Number of elements | +| `toHaveURL()` | Page URL | +| `toHaveTitle()` | Page title | + +### Soft Assertions + +Non-blocking checks: + +```typescript +await expect.soft(page.getByTestId('status')).toHaveText('Success') +await expect.soft(page.getByTestId('count')).toHaveText('5') +// Test continues even if above fail +``` + +## Anti-Patterns + +### ❌ Manual Waits + +```typescript +// Bad +await page.waitForTimeout(2000) + +// Good +await expect(page.getByText('Loaded')).toBeVisible() +``` + +### ❌ Implementation-Tied Selectors + +```typescript +// Bad +await page.locator('div.container > button.btn-primary').click() + +// Good +await page.getByRole('button', { name: 'Submit' }).click() +``` + +### ❌ Using first/last/nth Without Reason + +```typescript +// Bad - fragile +await page.getByRole('button').first().click() + +// Good - uniquely identify +await page.getByRole('button', { name: 'Submit' }).click() +``` + +### ❌ Non-Awaited Assertions + +```typescript +// Bad +expect(await page.getByText('Hello').isVisible()).toBe(true) + +// Good +await expect(page.getByText('Hello')).toBeVisible() +``` + +### ❌ Shared State Between Tests + +```typescript +// Bad +let sharedData +test('first', async () => { + sharedData = 'value' +}) +test('second', async () => { + expect(sharedData).toBe('value') +}) + +// Good - each test independent +test('first', async ({ page }) => { + /* complete setup */ +}) +test('second', async ({ page }) => { + /* complete setup */ +}) +``` + +## Locator Chaining + +```typescript +// Within a container +const dialog = page.getByRole('dialog') +await dialog.getByRole('button', { name: 'Submit' }).click() + +// Filter +const product = page.getByRole('listitem').filter({ hasText: 'Product 2' }) +await product.getByRole('button', { name: 'Add' }).click() +``` + +## Network Mocking + +```typescript +// Mock API +await page.route('**/api/users', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify([{ id: 1, name: 'Test' }]) + }) +) + +// Block resources +await context.route('**/*.{png,jpg}', (route) => route.abort()) +``` + +## Test Annotations + +```typescript +// Skip conditionally +test('firefox only', async ({ browserName }) => { + test.skip(browserName !== 'firefox', 'Firefox only') +}) + +// Slow test (triples timeout) +test('complex', async () => { + test.slow() +}) + +// Tag +test('login', { tag: '@smoke' }, async () => {}) +``` + +## Retry Configuration + +```typescript +// Per-describe +test.describe(() => { + test.describe.configure({ retries: 2 }) + test('flaky', async () => {}) +}) + +// Detect retry +test('cleanup aware', async ({}, testInfo) => { + if (testInfo.retry) await cleanup() +}) +``` + +## Parallel Execution + +```typescript +// Parallel (default) +test.describe.configure({ mode: 'parallel' }) + +// Serial (dependent tests) +test.describe.configure({ mode: 'serial' }) +``` + +## Debugging + +```bash +# Debug mode +npx playwright test --debug + +# Trace +npx playwright test --trace on +npx playwright show-report +``` + +```typescript +// Pause in test +await page.pause() +``` + +## Official Docs + +- [Best Practices](https://playwright.dev/docs/best-practices) +- [Locators](https://playwright.dev/docs/locators) +- [Assertions](https://playwright.dev/docs/test-assertions) +- [Auto-waiting](https://playwright.dev/docs/actionability) diff --git a/.agents/skills/writing-playwright-tests/testing/advanced.md b/.agents/skills/writing-playwright-tests/testing/advanced.md new file mode 100644 index 000000000..04822e3a5 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/testing/advanced.md @@ -0,0 +1,225 @@ +# Advanced Patterns + +## Change Transactions (Grouped Undo) + +Group multiple changes for single undo: + +```typescript +await comfyPage.page.evaluate(() => { + window['app'].canvas.emitBeforeChange() +}) + +// ... make multiple changes ... + +await comfyPage.page.evaluate(() => { + window['app'].canvas.emitAfterChange() +}) +``` + +## Mark Canvas Dirty + +After programmatic changes to the graph: + +```typescript +await comfyPage.page.evaluate(() => { + window['app'].graph.nodes[0].addWidget('number', 'new_widget', 10) + window['app'].graph.setDirtyCanvas(true, true) // Required! +}) +await comfyPage.nextFrame() +``` + +## Wait for Workflow Service + +After file operations, wait for workflow service to settle: + +```typescript +await comfyPage.page.waitForFunction( + () => window['app']?.extensionManager?.workflow?.isBusy === false, + undefined, + { timeout: 3000 } +) +``` + +## Dialog Mask Handling + +Wait for dialog masks to disappear: + +```typescript +const mask = comfyPage.page.locator('.p-dialog-mask') +if ((await mask.count()) > 0) { + await mask.first().waitFor({ state: 'hidden', timeout: 3000 }) +} +``` + +## LiteGraph Context Menu + +Click items in LiteGraph context menus: + +```typescript +await node.click('title', { button: 'right' }) +await comfyPage.clickLitegraphContextMenuItem('Remove Slot') +``` + +## LiteGraph Prompt Dialog + +Handle LiteGraph's prompt dialogs: + +```typescript +await comfyPage.page.waitForSelector('.graphdialog input', { state: 'visible' }) +await comfyPage.page.fill('.graphdialog input', 'new_name') +await comfyPage.page.keyboard.press('Enter') +``` + +## Move Mouse Away for Clean Screenshots + +Avoid hover effects in screenshots: + +```typescript +await node.click('title') +await comfyPage.moveMouseToEmptyArea() +await expect(comfyPage.canvas).toHaveScreenshot('clean.png') +``` + +## Setup Workflows Directory + +Set up/clear workflows for a test: + +```typescript +test.afterEach(async ({ comfyPage }) => { + await comfyPage.setupWorkflowsDirectory({}) // Clear all +}) + +// Or set up specific workflows +await comfyPage.setupWorkflowsDirectory({ + 'workflow1.json': workflowData1, + 'workflow2.json': workflowData2 +}) +``` + +## Execute Commands + +Run ComfyUI commands: + +```typescript +await comfyPage.executeCommand('Comfy.LoadDefaultWorkflow') +await comfyPage.executeCommand('Comfy.NewBlankWorkflow') +await comfyPage.executeCommand('Comfy.ClearWorkflow') +``` + +## Access App Instance + +Direct access to the app instance: + +```typescript +const result = await comfyPage.page.evaluate(() => { + return window['app'].graph.nodes.length +}) +``` + +## WebSocket Mocking + +Mock WebSocket status updates: + +```typescript +// Get WebSocket fixture +const { websocket } = await comfyPage.getWebSocket() + +// Send mock status +await websocket.send( + JSON.stringify({ + type: 'status', + data: { exec_info: { queue_remaining: 0 } } + }) +) +``` + +## Vue Node Testing + +Test Vue-rendered nodes: + +```typescript +const vueNode = comfyPage.vueNodes.getByTitle('My Vue Node') +await vueNode.waitForRender() +const content = await vueNode.getContent() +``` + +## Test with Ctrl Modifier + +Click with modifier keys: + +```typescript +await node.click('title', { modifiers: ['Control'] }) +await node.click('title', { modifiers: ['Shift'] }) +await node.click('title', { modifiers: ['Control', 'Shift'] }) +``` + +## Parallel Test Isolation + +Each test gets unique user ID for isolation: + +```typescript +test('isolated test', async ({ comfyPage }) => { + // comfyPage.id is unique per test instance + const uniqueWorkflow = `workflow-${comfyPage.id}.json` +}) +``` + +## Platform-Specific Tests + +Skip tests based on platform: + +```typescript +test('linux only', async ({ comfyPage }) => { + test.skip(process.platform !== 'linux', 'Linux only test') + // ... +}) +``` + +## Wait for Animation + +Wait for CSS animations to complete: + +```typescript +await comfyPage.page.waitForFunction(() => { + const el = document.querySelector('.animated-element') + return ( + getComputedStyle(el).animationPlayState === 'paused' || + getComputedStyle(el).animationName === 'none' + ) +}) +``` + +## Capture Network Response + +Wait for and capture specific network responses: + +```typescript +const responsePromise = comfyPage.page.waitForResponse('**/api/history') +await comfyPage.page.click('[data-testid="history-btn"]') +const response = await responsePromise +const data = await response.json() +``` + +## Retry Flaky Operations + +Use `toPass` for inherently flaky assertions: + +```typescript +await expect(async () => { + const value = await widget.getValue() + expect(value).toBe(100) +}).toPass({ timeout: 2000, intervals: [100, 200, 500] }) +``` + +## Screenshot with Masking + +Mask dynamic content in screenshots: + +```typescript +await expect(comfyPage.canvas).toHaveScreenshot('page.png', { + mask: [ + comfyPage.page.locator('.timestamp'), + comfyPage.page.locator('.random-id') + ] +}) +``` diff --git a/.agents/skills/writing-playwright-tests/testing/assets.md b/.agents/skills/writing-playwright-tests/testing/assets.md new file mode 100644 index 000000000..ce4eb2698 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/testing/assets.md @@ -0,0 +1,184 @@ +# Asset Patterns + +## Why Use Assets + +**Always prefer pre-made workflow assets over programmatic workflow creation.** + +Benefits: + +- Stable, reproducible test state +- Faster test execution +- Easier to debug (open asset in ComfyUI) +- Less fragile than runtime creation + +## Asset Directory Structure + +``` +browser_tests/assets/ +├── default.json # Basic empty workflow +├── canvas/ # Canvas state tests +│ ├── pan_zoom.json +│ └── selection.json +├── groups/ # Group-related +│ ├── basic_group.json +│ └── nested_groups.json +├── nodes/ # Node-specific +│ ├── reroute.json +│ ├── primitive.json +│ └── bypass.json +├── widgets/ # Widget tests +│ ├── combo_widget.json +│ ├── slider_widget.json +│ └── text_widget.json +├── workflows/ # Complex scenarios +│ ├── img2img.json +│ ├── inpainting.json +│ └── controlnet.json +└── images/ # Image files for drag-drop + ├── test_image.png + └── mask.png +``` + +## Creating New Assets + +### 1. Build in ComfyUI + +1. Open ComfyUI +2. Create the workflow state you need +3. Set up nodes, connections, widget values + +### 2. Export as JSON + +1. Click workflow menu → Save (API Format) +2. Or use developer console: `app.graphToPrompt()` + +### 3. Save to Assets + +```bash +# Save to appropriate subdirectory +mv workflow.json browser_tests/assets/feature/my_workflow.json +``` + +### 4. Use in Test + +```typescript +await comfyPage.loadWorkflow('feature/my_workflow') +``` + +## Loading Assets + +### Basic Load + +```typescript +// Loads browser_tests/assets/myWorkflow.json +await comfyPage.loadWorkflow('myWorkflow') +await comfyPage.nextFrame() +``` + +### With Path + +```typescript +// From subdirectory +await comfyPage.loadWorkflow('widgets/combo_widget') +``` + +### Default Workflow + +```typescript +// Load the basic starting workflow +await comfyPage.loadWorkflow('default') +``` + +## Image Assets + +For drag-and-drop tests: + +```typescript +await comfyPage.dragAndDropFile('browser_tests/assets/images/test_image.png', { + x: 100, + y: 100 +}) +await comfyPage.nextFrame() +``` + +## Asset Best Practices + +### 1. Keep Assets Minimal + +Only include nodes needed for the test. Avoid complex workflows unless testing complexity. + +### 2. Use Descriptive Names + +``` +❌ test1.json +✅ slider_widget_with_min_max.json +``` + +### 3. Organize by Feature + +``` +assets/widgets/ # All widget tests +assets/nodes/ # Node-specific tests +assets/canvas/ # Canvas interaction tests +``` + +### 4. Document Non-Obvious Assets + +Add a comment at the top of the JSON or a README in the subdirectory: + +```json +{ + "_comment": "Workflow with 3 nodes connected in series for connection tests", + "nodes": [...] +} +``` + +### 5. Version Control Assets + +Assets are committed to git. Changes should be reviewed like code. + +## Finding Existing Assets + +```bash +# List all assets +find browser_tests/assets -name "*.json" | head -20 + +# Search by content +grep -r "KSampler" browser_tests/assets/ + +# Find assets used by a test +grep -r "loadWorkflow" browser_tests/*.spec.ts +``` + +## Example: Asset-Based Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('Combo Widget', { tag: ['@widget'] }, () => { + test.beforeEach(async ({ comfyPage }) => { + // Load premade asset instead of building workflow + await comfyPage.loadWorkflow('widgets/combo_widget') + await comfyPage.nextFrame() + }) + + test.afterEach(async ({ comfyPage }) => { + await comfyPage.resetView() + }) + + test('changes combo selection', async ({ comfyPage }) => { + const node = comfyPage.getNodeRefByTitle('KSampler') + const widget = node.getWidget('sampler_name') + + await widget.click() + await comfyPage.page.getByRole('option', { name: 'euler' }).click() + await comfyPage.nextFrame() + + const value = await widget.getValue() + expect(value).toBe('euler') + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/testing/file-upload.md b/.agents/skills/writing-playwright-tests/testing/file-upload.md new file mode 100644 index 000000000..e84c1c06d --- /dev/null +++ b/.agents/skills/writing-playwright-tests/testing/file-upload.md @@ -0,0 +1,141 @@ +# File Upload & Drag-Drop Patterns + +## Drag and Drop Files + +Use for testing file drops onto canvas or nodes: + +```typescript +// Drop file on canvas (default position) +await comfyPage.dragAndDropFile('workflowInMedia/workflow.webp') + +// Drop file at specific position (on a node) +const node = await comfyPage.getNodeRefsByType('LoadImage')[0] +const { x, y } = await node.getPosition() +await comfyPage.dragAndDropFile('image32x32.webp', { + dropPosition: { x, y } +}) + +// Wait for upload to complete +await comfyPage.dragAndDropFile('animated_webp.webp', { + dropPosition: { x, y }, + waitForUpload: true +}) +``` + +## Drag and Drop URLs + +For testing URL drops (simulating drag from browser): + +```typescript +await comfyPage.dragAndDropURL('https://example.com/workflow.png') +await expect(comfyPage.canvas).toHaveScreenshot('dropped_url.png') +``` + +## File Chooser Pattern + +For testing file upload dialogs: + +```typescript +// Set up file chooser promise before clicking +const fileChooserPromise = comfyPage.page.waitForEvent('filechooser') +await uploadButton.click() +const fileChooser = await fileChooserPromise + +// Upload the test file +await fileChooser.setFiles(comfyPage.assetPath('image32x32.webp')) +``` + +## Workflow Upload Input + +For loading workflow files: + +```typescript +// This is what loadWorkflow() uses internally +await comfyPage.workflowUploadInput.setInputFiles( + comfyPage.assetPath('workflow.json') +) +await comfyPage.nextFrame() +``` + +## Available Test Assets + +Located in `browser_tests/assets/`: + +### Images + +- `image32x32.webp` - Small test image +- `image64x64.webp` - Larger test image +- `example.webp` - Example thumbnail +- `animated_webp.webp` - Animated WebP for animation tests + +### Workflows in Media + +- `workflowInMedia/workflow.webp` - Workflow embedded in WebP +- `workflowInMedia/workflow.png` - Workflow embedded in PNG +- `workflowInMedia/workflow.mp4` - Workflow embedded in MP4 +- `workflowInMedia/workflow.svg` - Workflow embedded in SVG + +## Example: Image Upload Test + +```typescript +test('Can drag and drop image', async ({ comfyPage }) => { + await comfyPage.loadWorkflow('widgets/load_image_widget') + + // Get position of the LoadImage node + const nodes = await comfyPage.getNodeRefsByType('LoadImage') + const loadImageNode = nodes[0] + const { x, y } = await loadImageNode.getPosition() + + // Drag and drop image file onto the node + await comfyPage.dragAndDropFile('image32x32.webp', { + dropPosition: { x, y } + }) + + // Verify the image preview changed + await expect(comfyPage.canvas).toHaveScreenshot( + 'image_preview_drag_and_dropped.png' + ) + + // Verify the filename combo was updated + const fileComboWidget = await loadImageNode.getWidget(0) + const filename = await fileComboWidget.getValue() + expect(filename).toBe('image32x32.webp') +}) +``` + +## Example: File Chooser Test + +```typescript +test('should upload image file', async ({ comfyPage }) => { + await comfyPage.page.keyboard.press('Control+,') + const uploadButton = comfyPage.page.locator('button:has(.pi-upload)') + + // Set up file upload handler before clicking + const fileChooserPromise = comfyPage.page.waitForEvent('filechooser') + await uploadButton.click() + const fileChooser = await fileChooserPromise + + // Upload the test image + await fileChooser.setFiles(comfyPage.assetPath('image32x32.webp')) + + // Verify upload succeeded + const urlInput = comfyPage.page.locator('input[type="text"]') + await expect(urlInput).toHaveValue(/^\/api\/view\?/) +}) +``` + +## Organizing Test Assets + +Assets should be organized by feature: + +``` +browser_tests/assets/ +├── widgets/ # Widget-specific workflows +│ ├── load_image_widget.json +│ └── boolean_widget.json +├── workflowInMedia/ # Files with embedded workflows +├── nodes/ # Node-specific workflows +└── image32x32.webp # Shared image assets +``` + +See [patterns/assets.md](assets.md) for full asset organization guide. diff --git a/.agents/skills/writing-playwright-tests/testing/mocking.md b/.agents/skills/writing-playwright-tests/testing/mocking.md new file mode 100644 index 000000000..dfba112c1 --- /dev/null +++ b/.agents/skills/writing-playwright-tests/testing/mocking.md @@ -0,0 +1,162 @@ +# Mocking Patterns + +## API Route Mocking + +```typescript +test('mocks API response', async ({ comfyPage }) => { + await comfyPage.page.route('**/api/queue', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify({ queue_running: [], queue_pending: [] }) + }) + ) + + // Test continues with mocked response +}) +``` + +## Mocking External Resources + +### Block Image Loading (Performance) + +```typescript +test.beforeEach(async ({ comfyPage }) => { + await comfyPage.page + .context() + .route('**/*.{png,jpg,jpeg}', (route) => route.abort()) +}) +``` + +### Mock Model List + +```typescript +await comfyPage.page.route('**/object_info', (route) => + route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + CheckpointLoaderSimple: { + input: { + required: { + ckpt_name: [['model1.safetensors', 'model2.safetensors']] + } + } + } + }) + }) +) +``` + +## Mocking Release Info (Default) + +By default, tests mock the release check to prevent popup: + +```typescript +// This is already done in fixtures, but if you need custom: +await comfyPage.page.route('**/api/releases', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify({ releases: [] }) + }) +) +``` + +## WebSocket Mocking + +For status updates and execution progress: + +```typescript +// The fixture provides a websocket helper +const { websocket } = await comfyPage.getWebSocket() + +// Send mock status +await websocket.send( + JSON.stringify({ + type: 'status', + data: { exec_info: { queue_remaining: 0 } } + }) +) +``` + +## Mock File Upload Response + +```typescript +await comfyPage.page.route('**/upload/image', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify({ name: 'uploaded.png', subfolder: '', type: 'input' }) + }) +) +``` + +## Intercepting Requests + +```typescript +test('captures API call', async ({ comfyPage }) => { + const requestPromise = comfyPage.page.waitForRequest('**/api/prompt') + + // Trigger action that makes request + await comfyPage.page.keyboard.press('Control+Enter') + + const request = await requestPromise + const body = request.postDataJSON() + expect(body.prompt).toBeDefined() +}) +``` + +## Intercepting Responses + +```typescript +test('validates response', async ({ comfyPage }) => { + const responsePromise = comfyPage.page.waitForResponse('**/api/history') + + // Trigger action + await comfyPage.page.click('[data-testid="history-btn"]') + + const response = await responsePromise + expect(response.status()).toBe(200) +}) +``` + +## Example: Complete Mocking Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('API Mocking', { tag: ['@ui'] }, () => { + test('shows empty queue message', async ({ comfyPage }) => { + // Mock empty queue + await comfyPage.page.route('**/api/queue', (route) => + route.fulfill({ + status: 200, + body: JSON.stringify({ + queue_running: [], + queue_pending: [] + }) + }) + ) + + await comfyPage.loadWorkflow('default') + await comfyPage.page.click('[data-testid="queue-btn"]') + + await expect(comfyPage.page.getByText('Queue is empty')).toBeVisible() + }) + + test('handles API error', async ({ comfyPage }) => { + await comfyPage.page.route('**/api/prompt', (route) => + route.fulfill({ + status: 500, + body: JSON.stringify({ error: 'Server error' }) + }) + ) + + await comfyPage.loadWorkflow('default') + await comfyPage.page.keyboard.press('Control+Enter') + + await expect(comfyPage.page.getByText('Error')).toBeVisible() + }) +}) +``` diff --git a/.agents/skills/writing-playwright-tests/testing/widgets.md b/.agents/skills/writing-playwright-tests/testing/widgets.md new file mode 100644 index 000000000..ea4f0a2dd --- /dev/null +++ b/.agents/skills/writing-playwright-tests/testing/widgets.md @@ -0,0 +1,160 @@ +# Widget Patterns + +## Getting Widgets + +```typescript +const node = comfyPage.getNodeRefByTitle('KSampler') +const widget = node.getWidget('seed') +``` + +## Widget Types + +### Number Widgets (INT, FLOAT) + +```typescript +const seedWidget = node.getWidget('seed') +await seedWidget.setValue(12345) +const value = await seedWidget.getValue() +``` + +### Combo/Dropdown Widgets + +```typescript +const samplerWidget = node.getWidget('sampler_name') +await samplerWidget.setValue('euler') + +// Or click to open dropdown +await samplerWidget.click() +await comfyPage.page.getByText('euler_ancestral').click() +await comfyPage.nextFrame() +``` + +### String/Text Widgets + +```typescript +const promptWidget = node.getWidget('text') +await promptWidget.setValue('a beautiful landscape') +``` + +### Toggle/Boolean Widgets + +```typescript +const toggleWidget = node.getWidget('enable') +await toggleWidget.setValue(true) +// or +await toggleWidget.click() +await comfyPage.nextFrame() +``` + +### Slider Widgets + +```typescript +const sliderWidget = node.getWidget('denoise') +await sliderWidget.setValue(0.75) +``` + +## Widget Value Assertions + +```typescript +const widget = node.getWidget('steps') +const value = await widget.getValue() +expect(value).toBe(20) +``` + +## Widget Visibility + +```typescript +// Check widget is visible +await expect(widget.locator).toBeVisible() + +// Widget might be hidden when node is collapsed +await node.expand() +await comfyPage.nextFrame() +await expect(widget.locator).toBeVisible() +``` + +## Common Widget Gotchas + +### 1. Wait for Value Change + +Widget values may not update instantly: + +```typescript +await widget.setValue(100) +await comfyPage.nextFrame() + +// Retry assertion +await expect(async () => { + const value = await widget.getValue() + expect(value).toBe(100) +}).toPass({ timeout: 2000 }) +``` + +### 2. Combo Widget Selection + +Click-based selection is more reliable than setValue for combos: + +```typescript +await samplerWidget.click() +await comfyPage.page.getByRole('option', { name: 'euler' }).click() +await comfyPage.nextFrame() +``` + +### 3. Widget Focus + +Some widgets need focus before input: + +```typescript +await widget.locator.click() +await widget.locator.fill('new value') +await comfyPage.nextFrame() +``` + +## Example: Complete Widget Test + +```typescript +import { + comfyPageFixture as test, + comfyExpect as expect +} from './fixtures/ComfyPage' + +test.describe('Widget Operations', { tag: ['@widget'] }, () => { + test.beforeEach(async ({ comfyPage }) => { + await comfyPage.setSetting('Comfy.UseNewMenu', 'Disabled') + await comfyPage.loadWorkflow('widgets/all_types') + await comfyPage.nextFrame() + }) + + test('sets number widget value', async ({ comfyPage }) => { + const node = comfyPage.getNodeRefByTitle('KSampler') + const seedWidget = node.getWidget('seed') + + await seedWidget.setValue(42) + await comfyPage.nextFrame() + + const value = await seedWidget.getValue() + expect(value).toBe(42) + }) + + test('selects combo option', async ({ comfyPage }) => { + const node = comfyPage.getNodeRefByTitle('KSampler') + const samplerWidget = node.getWidget('sampler_name') + + await samplerWidget.click() + await comfyPage.page.getByRole('option', { name: 'dpmpp_2m' }).click() + await comfyPage.nextFrame() + + const value = await samplerWidget.getValue() + expect(value).toBe('dpmpp_2m') + }) +}) +``` + +## Asset Workflows for Widget Testing + +Pre-made workflows for widget tests: + +- `assets/widgets/combo_widget.json` +- `assets/widgets/slider_widget.json` +- `assets/widgets/text_widget.json` +- `assets/widgets/number_widget.json`