Compare commits

..

30 Commits

Author SHA1 Message Date
dante01yoon
59fceb3098 test: type stale autogrow regression guard 2026-04-13 17:25:49 +09:00
dante01yoon
3ae9a83c4c fix: harden cloud frontend runtime guards 2026-04-13 16:05:03 +09:00
Kelly Yang
bd82c855e0 test: add minimap E2E tests for graph content and click-to-navigate (#10738)
## Summary

Adds Playwright E2E tests verifying that 
1. the minimap canvas renders node content
2. clears when the graph is empty
3. correctly navigates the main canvas on click

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10738-test-add-minimap-E2E-tests-for-graph-content-and-click-to-navigate-3336d73d365081eb955ce711b3efc57f)
by [Unito](https://www.unito.io)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: changes are limited to adding `data-testid` attributes to
the minimap UI and expanding Playwright E2E assertions, with no
production behavior changes expected.
> 
> **Overview**
> Strengthens minimap E2E coverage by switching existing assertions from
CSS selectors to new `data-testid`-based selectors and adding helper
utilities for canvas/overlay interactions.
> 
> Adds new Playwright tests that verify the minimap canvas renders
content when nodes exist, clears when the graph is emptied, and that
clicking the minimap pans the main canvas (including a
post-`fitViewToSelectionAnimated` tolerance check).
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
06e7542af1. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: Alexander Brown <DrJKL0424@gmail.com>
Co-authored-by: GitHub Action <action@github.com>
2026-04-13 04:28:04 +00:00
Kelly Yang
5b7ef3fe21 test: Painter Widget E2E Test Plan (#10846)
### Summary of Improvements

* **Custom Test Coverage Extension**: Enhanced the Painter widget E2E
test suite by refactoring logic for better maintainability and
robustness.
* **Stable Component Targeting**: Introduced
`data-testid="painter-dimension-text"` to `WidgetPainter.vue`, providing
a reliable, non-CSS-dependent locator for canvas size verification.
* **Improved Test Organization**: Reorganized existing test scenarios
into logical categories using `test.describe` blocks (Drawing, Brush
Settings, Canvas Size Controls, etc.).
* **Asynchronous Helper Integration**: Converted `hasCanvasContent` to
an asynchronous helper and unified its usage across multiple test cases
to eliminate redundant pixel-checking logic.
* **Locator Resilience**: Updated Reka UI slider interaction logic to
use more precise targeting (`:not([data-slot])`), preventing ambiguity
and improving test stability.
* **Scenario Refinement**: Updated the `pointerup` test logic to
accurately reflect pointer capture behavior when interactions occur
outside the canvas boundaries.
* **Enhanced Verification Feedback**: Added descriptive error messages
to `expect.poll` assertions to provide clearer context on potential
failure points.
* **Standardized Tagging**: Restored the original tagging strategy
(including `@smoke` and `@screenshot` tags) to ensure tests are
categorized correctly for CI environments.

### Red-Green Verification

| Commit | CI Status | Purpose |
| :--- | :--- | :--- |
| `test: refactor painter widget e2e tests and address review findings`
| 🟢 Green | Addresses all E2E test quality and stability issues from
review findings. |

### Test Plan

- [x] **Quality Checks**: `pnpm format`, `pnpm lint`, and `pnpm
typecheck` verified as passing.
- [x] **Component Integration**: `WidgetPainter.vue` `data-testid`
correctly applied and used in tests.
- [x] **Helper Reliability**: `hasCanvasContent` correctly identifies
colored pixels and returns a promise for `expect.poll`.
- [x] **Locator Robustness**: Verified Reka slider locators correctly
exclude internal thumb spans.
- [x] **Boundary Interaction**: Verified `pointerup` correctly ends
strokes when triggered outside the viewport.
- [x] **Tagging Consistency**: Verified `@smoke` and `@screenshot` tags
are present in the final test suite.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10846-test-Painter-Widget-E2E-Test-Plan-3386d73d365081deb70fe4afbd417efb)
by [Unito](https://www.unito.io)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Primarily adds/refactors Playwright E2E tests and stable `data-testid`
hooks, with no changes to Painter drawing logic. Risk is limited to
potential test brittleness or minor UI attribute changes.
> 
> **Overview**
> Expands the Painter widget Playwright suite with new grouped scenarios
covering drawing/erasing behavior, tool switching, brush inputs, canvas
resizing (including preserving drawings), clear behavior, and
serialization/upload flows (including failure toast).
> 
> Refactors the tests to use a shared `@e2e/helpers/painter` module
(`drawStroke`, `hasCanvasContent`, `triggerSerialization`), improves
stability via role/testid-based locators and clearer `expect.poll`
messaging, and adds `data-testid` attributes (e.g.,
`painter-clear-button`, `painter-*-row`, `painter-dimension-text`) to
`WidgetPainter.vue` to avoid CSS-dependent selectors.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
053a8e9ed2. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: GitHub Action <action@github.com>
2026-04-13 00:13:04 -04:00
Kelly Yang
85de833776 test: add E2E tests for ImageCompare widget (#10767)
## Summary
Add E2E tests for ImageCompare widget
Covers slider interaction, batch navigation, single-image modes, visual
regression screenshots, and edge cases for the ImageCompare Vue node
widget.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10767-test-add-E2E-tests-for-ImageCompare-widget-3346d73d365081c6bfc6fbd97fa04e4d)
by [Unito](https://www.unito.io)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Adds Playwright E2E coverage and screenshot assertions only; main risk
is increased CI runtime/flakiness due to additional image-loading and
hover/position polling.
> 
> **Overview**
> Adds a new Playwright E2E suite for the ImageCompare Vue widget
(tagged `@widget`) that programmatically sets widget values and asserts
rendering for empty, single-image, and dual-image states.
> 
> Expands coverage to **slider behavior** (default 50%, hover movement,
clamping, persistence) using polling on inline `clip-path`/handle
position, and adds **batch navigation** tests for multi-image
before/after sets.
> 
> Introduces **visual regression screenshots** at default and specific
slider positions, plus edge-case tests for broken URLs, rapid updates
resetting batch index, legacy string values, and custom alt text.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
2c65440384. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: GitHub Action <action@github.com>
2026-04-13 00:09:50 -04:00
Kelly Yang
cab46567c0 test: add E2E tests for ImageCropV2 widget (#10737)
## Summary
Adds Playwright E2E tests for the ImageCropV2 widget covering 
1. the empty state (no source image)
2. default control rendering
3. source image display with crop overlay
4. drag-to-reposition behavior.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10737-test-add-E2E-tests-for-ImageCropV2-widget-3336d73d365081b28ed9db63e5df383e)
by [Unito](https://www.unito.io)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Low risk: primarily adds Playwright E2E coverage and introduces
`data-testid` attributes for more stable selectors, with no changes to
core crop behavior.
> 
> **Overview**
> Adds new Playwright E2E coverage for the `ImageCropV2` Vue-node
widget, including workflows/fixtures for a disconnected input and a
`LoadImage -> ImageCropV2 -> PreviewImage` pipeline.
> 
> Tests validate the empty state and default controls, verify the crop
overlay renders after execution with screenshot assertions, and exercise
drag-to-reposition by dispatching pointer events and asserting the
widget’s crop value updates.
> 
> Updates `WidgetImageCrop.vue` to add `data-testid` hooks (empty
state/icon and crop overlay) to make the E2E selectors stable.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
9f29272742. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-04-12 23:58:01 -04:00
Comfy Org PR Bot
63435bdb34 1.44.3 (#11170)
Patch version increment to 1.44.3

**Base branch:** `main`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11170-1-44-3-3406d73d365081799aa4e189009d123b)
by [Unito](https://www.unito.io)

Co-authored-by: christian-byrne <72887196+christian-byrne@users.noreply.github.com>
2026-04-12 23:11:20 +00:00
Kelly Yang
20255da61f feat(load3d): add optional HDRI environment lighting to 3D preview nodes (#10818)
## Summary

Adds `HDRIManager` to load `.hdr/.exr` files as equirectangular
environment maps via **three.js** `RGBELoader/EXRLoader`
- Uploads HDRI files to the server via `/upload/image` API so they
persist across page reloads
- Restores HDRI state (enabled, **intensity**, **background**) from node
properties on reload
- Auto-enables "**Show as Background**" on successful upload for
immediate visual feedback
- Hides standard directional lights when HDRI is active; restores them
when disabled
- Hides the Light Intensity control while HDRI is active (lights have no
effect when HDRI overrides scene lighting)
- Limits HDRI availability to PBR-capable formats (.gltf, .glb, .fbx,
.obj); automatically disables when switching to an incompatible model
- Adds intensity slider and "**Show as Background**" toggle to the HDRI
panel

## How to Use HDRI Environment Lighting
1. Load a 3D model using a Load3D or Load3DViewer node (supported
formats: .gltf, .glb, .fbx, .obj)
2. Open the control panel → go to the Light tab
3. Click the globe icon to open the **HDRI panel**
4. Click Upload HDRI and select a` .hdr` or `.exr` file
5. The environment lighting applies automatically — the scene background
also updates to preview the panorama
6. Use the intensity slider to adjust the strength of the environment
lighting
7. Toggle Show as Background to show or hide the HDRI panorama behind
the model

## Screenshots



https://github.com/user-attachments/assets/1ec56ef0-853e-452f-ae2b-2474c9d0d781



┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10818-feat-load3d-add-optional-HDRI-environment-lighting-to-3D-preview-nodes-3366d73d365081ea8c7ad9226b8b1e2f)
by [Unito](https://www.unito.io)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Adds new HDRI loading/rendering path and persists new
`LightConfig.hdri` state, touching Three.js rendering, file uploads, and
node property restoration. Risk is moderate due to new async flows and
potential compatibility/performance issues with model switching and
renderer settings.
> 
> **Overview**
> Adds optional **HDRI environment lighting** to Load3D previews,
including a new `HDRIManager` that loads `.hdr`/`.exr` files into
Three.js environment/background and exposes controls for enable/disable,
background display, and intensity.
> 
> Extends `LightConfig` with an `hdri` block that is persisted on nodes
and restored on reload; `useLoad3d` now uploads HDRI files, loads them
into `Load3d`, maps scene light intensity to HDRI intensity, and
auto-disables HDRI when the current model format doesn’t support it.
> 
> Updates the UI to include embedded HDRI controls under the Light panel
(with dismissable overlays and icon updates), adjusts light intensity
behavior when HDRI is active, and adds tests/strings/utilities
(`getFilenameExtension`, `mapSceneLightIntensityToHdri`, new constants)
to support the feature.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
b12c9722dc. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Terry Jia <terryjia88@gmail.com>
2026-04-12 05:55:48 -04:00
Christian Byrne
c2dba8f4ee chore(#11080): consolidate duplicate rgbToHSL — use shared colorUtil (#11134)
## Summary

Consolidate duplicate `rgbToHSL` implementation — mask editor now uses
the shared `colorUtil.ts` version instead of its own copy.

## Changes

- Export `rgbToHsl` from `src/utils/colorUtil.ts` (was private)
- Replace 30-line local `rgbToHSL` in `useCanvasTools.ts` with a 2-line
wrapper that imports from `colorUtil.ts` and scales the return values
from 0-1 to degree/percentage

## Testing

### Automated

- All 176 existing tests pass (`colorUtil.test.ts` + `maskeditor/`
suite)
- No new tests needed — behavior is identical

### E2E Verification Steps

1. Open any image in the mask editor
2. Select the magic wand / color picker tool
3. Use HSL-based color matching — results should be identical to before

## Review Focus

The canonical `rgbToHsl` returns normalized 0-1 values while the mask
editor needs degree/percentage scale (h: 0-360, s: 0-100, l: 0-100). The
local wrapper handles this conversion.

Fixes #11080

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11134-chore-11080-consolidate-duplicate-rgbToHSL-use-shared-colorUtil-33e6d73d36508120bbd8f444f5cc94b6)
by [Unito](https://www.unito.io)

Co-authored-by: Terry Jia <terryjia88@gmail.com>
2026-04-12 01:40:55 +00:00
Alexander Brown
6f579c5992 fix: enable playwright/no-force-option lint rule (#11164)
## Summary

Enable the previously disabled `playwright/no-force-option` lint rule at
error level and resolve all 29 violations across 10 files.

## Changes

### Lint rule
- `.oxlintrc.json`: `playwright/no-force-option` changed from `off` to
`error`

### Shared utility
- `CanvasHelper.ts`: Add `mouseClickAt()` and `mouseDblclickAt()`
methods that convert canvas-element-relative positions to absolute page
coordinates and use `page.mouse` APIs, avoiding Playwright's locator
actionability checks that fail when Vue DOM overlays sit above the
`<canvas>` element

### Force removal (20 violations)
- `selectionToolboxActions.spec.ts`: Remove `force: true` from 8 toolbox
button clicks (the `pointer-events: none` splitter overlay does not
intercept `elementFromPoint()`)
- `selectionToolboxSubmenus.spec.ts`: Remove `force: true` from 2
popover menu item clicks
- `BuilderSelectHelper.ts`: Remove `force: true` from 2 widget/node
clicks (builder mode does not disable pointer events)
- `linkInteraction.spec.ts`: Remove `force: true` from 3 slot `dragTo()`
calls (`::after` pseudo-elements do not intercept `elementFromPoint()`)
- `SidebarTab.ts`: Remove `force: true` from toast dismissal (`.catch()`
already handles failures)
- `nodeHelp.spec.ts`: Remove `force: true` from info button click
(preceding `toBeVisible()` assertion is sufficient)

### Rewrites (3 violations)
- `integerWidget.spec.ts`: Replace force-clicking disabled buttons with
`toBeDisabled()` assertions
- `Topbar.ts`: Replace force-click with `waitFor({ state: 'visible' })`
after hover

### Canvas coordinate clicks (9 violations)
- `litegraphUtils.ts`: Convert `NodeReference.click()` and
`navigateIntoSubgraph()` to use
`canvasOps.mouseClickAt()`/`mouseDblclickAt()`
- `subgraphPromotion.spec.ts`: Convert 3 right-click canvas calls to
`canvasOps.mouseClickAt()`
- `selectionToolboxSubmenus.spec.ts`: Convert 1 canvas dismiss-click to
`canvasOps.mouseClickAt()`

## Rationale

The original `force: true` usages were added defensively based on
incorrect assumptions about the `z-999 pointer-events: none` splitter
overlay intercepting Playwright's actionability checks. In reality,
`elementFromPoint()` skips elements with `pointer-events: none`, so the
overlay is transparent to Playwright's hit-test.

For canvas coordinate clicks, `force: true` on a locator does not tunnel
through DOM overlays — it only skips Playwright's preflight checks.
`page.mouse.click()` is the correct API for coordinate-based canvas
interactions.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11164-fix-enable-playwright-no-force-option-lint-rule-33f6d73d365081e78601c6114121d272)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
2026-04-11 19:59:34 +00:00
Dante
e729e5edb8 fix: place cloned node above original in Vue renderer (#10361)
## Summary

Cloned/pasted nodes in Node 2.0 (Vue renderer) mode now appear above the
original node instead of behind it.

## Root Cause

The legacy LiteGraph canvas renderer uses array ordering for z-ordering:
nodes are stored in `graph._nodes` and drawn sequentially, so newly
added nodes (appended to the end) are automatically drawn on top. There
is no explicit z-index.

The Vue renderer (Node 2.0) uses explicit CSS `z-index` for node
ordering. New nodes default to `zIndex: 0` in `layoutMutations.ts`. When
a node has been interacted with, `bringNodeToFront` raises its z-index.
A cloned node at z-index 0 therefore appears behind any previously
interacted node.

The alt-click clone path in `LGraphNode.vue` already handles this
correctly by calling `bringNodeToFront()` after cloning. However, the
menu clone and keyboard paste paths go through `_deserializeItems` in
`LGraphCanvas.ts`, which does not set z-index for new nodes.

| Clone method | Legacy renderer | Vue renderer (before fix) | Vue
renderer (after fix) |
|---|---|---|---|
| Alt-click drag | On top (array order) | On top (`bringNodeToFront`
called) | On top |
| Right-click menu Clone | On top (array order) | Behind original
(z-index 0) | On top |
| Ctrl+C / Ctrl+V | On top (array order) | Behind original (z-index 0) |
On top |

## Steps to Reproduce

1. Enable Node 2.0 mode (Vue renderer) in settings
2. Add any node to the canvas
3. Click or drag the node (raises its z-index via `bringNodeToFront`)
4. Right-click the node and select "Clone"
5. **Expected**: Cloned node appears above the original, immediately
draggable
6. **Actual**: Cloned node appears behind the original; user must move
the original to access the clone

## Changes

After `batchUpdateNodeBounds` in `_deserializeItems`, calls
`bringNodeToFront` for each newly created node so they receive a z-index
above all existing nodes.

## Side Effect Analysis

Checked all call sites of `_deserializeItems`:

1. **Initial graph load / workflow open**: `loadGraphData` in `app.ts`
does NOT call `_deserializeItems`. Workflow loading goes through
`LGraph.configure()` which directly adds nodes and links. The layout
store is initialized separately via `initializeFromLiteGraph`. No side
effect.

2. **Paste from clipboard (Ctrl+V)**: Both `usePaste.ts` (line 52) and
`pasteFromClipboard` (line 4080) call `_deserializeItems`. Pasted nodes
appearing on top is the correct and desired behavior. No issue.

3. **Undo/Redo**: `ChangeTracker.updateState()` calls
`app.loadGraphData()`, which does a full graph reconfigure -- it does
NOT go through `_deserializeItems`. No side effect.

4. **Subgraph blueprint addition**: `litegraphService.ts` (line 906)
calls `_deserializeItems` when adding subgraph blueprints from the node
library. These are freshly placed nodes that should appear on top.
Desired behavior.

5. **Alt-click clone in LGraphNode.vue**: This path calls
`LGraphCanvas.cloneNodes()` -> `_deserializeItems()`, then separately
calls `bringNodeToFront()` again on line 433 of `LGraphNode.vue`. The
second call is now redundant (the node is already at max z-index), but
harmless -- `bringNodeToFront` finds the current max, adds 1, and sets.
The z-index will increment from N to N+1 on the second call. This is a
minor redundancy, not a bug.

6. **Performance**: `bringNodeToFront` iterates all nodes in the layout
store once per call (O(m)) to find max z-index. For n new nodes, the
total cost is O(n*m). In practice, clone/paste operations involve a
small number of nodes (typically 1-10), so this is negligible. For
extremely large pastes (100+ nodes), each call also increments the max
by 1, so z-indices will be sequential (which is actually a reasonable
stacking order).

7. **layoutStore availability**: `layoutStore` is a module-level
singleton (`new LayoutStoreImpl()`) -- not a Pinia store -- so it is
always available. The `useLayoutMutations()` composable is a plain
function returning an object of closures over `layoutStore`. It does not
require Vue component context. No risk of runtime errors.

8. **Legacy renderer (non-Vue mode)**: When Node 2.0 mode is disabled,
the layout store still exists but is not used for rendering. Calling
`bringNodeToFront` will update z-index values in the Yjs document that
are never read. This is harmless.

## Red-Green Verification

| Commit | Result | Description |
|---|---|---|
| `6894b99` `test:` | RED | Test asserts cloned node z-index > original.
Fails with `expected 0 to be greater than 5`. |
| `3567469` `fix:` | GREEN | Calls `bringNodeToFront` for each new node
in `_deserializeItems`. Test passes. |

Fixes #10307

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-11 12:12:37 +00:00
Alexander Brown
3043b181d7 refactor: extract composables from VTU holdout components, complete VTL migration (#10966)
## Summary

Extract internal logic from the 2 remaining VTU holdout components into
composables, enabling full VTL migration.

## Changes

- **What**: Extract `useProcessedWidgets` from `NodeWidgets.vue`
(486→135 LOC) and `useWidgetSelectItems`/`useWidgetSelectActions` from
`WidgetSelectDropdown.vue` (563→170 LOC). Rewrite both component test
files as composable unit tests + slim behavioral VTL tests. Remove
`@vue/test-utils` devDependency.
- **Dependencies**: Removes `@vue/test-utils`

## Review Focus

- Composable extraction is mechanical — no logic changes, just moving
code into testable units
- `useProcessedWidgets` handles widget deduplication, promotion border
styling, error detection, and identity resolution (~290 LOC)
- `useWidgetSelectItems` handles the full computed chain from widget
values → dropdown items including cloud asset mode and multi-output job
resolution (~350 LOC)
- `useWidgetSelectActions` handles selection resolution and file upload
(~120 LOC)
- 40 new composable-level unit tests replace 13 `wrapper.vm.*` accesses
across the 2 holdout files

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10966-refactor-extract-composables-from-VTU-holdout-components-complete-VTL-migration-33c6d73d36508148a3a4ccf346722d6d)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
2026-04-10 19:04:16 -07:00
Alexander Brown
8c9328c1b2 feat: add eslint-plugin-playwright via oxlint JS plugins (#11136)
## Summary

Add eslint-plugin-playwright as an oxlint JS plugin scoped to
browser_tests/, enforcing Playwright best practices at lint time.

## Changes

- **What**: Configure eslint-plugin-playwright@2.10.1 via oxlint's alpha
`jsPlugins` field (`.oxlintrc.json` override scoped to
`browser_tests/**/*.ts`). 18 recommended rules +
`prefer-native-locators` + `require-to-pass-timeout` at error severity.
All 173 initial violations resolved (config, auto-fix, manual fixes).
`no-force-option` set to off — 28 violations need triage (canvas overlay
workarounds vs unnecessary force) in a dedicated PR.
- **Dependencies**: `eslint-plugin-playwright@^2.10.1` (devDependency,
required by oxlint jsPlugins at runtime)

## Review Focus

- `.oxlintrc.json` override structure — this is the first use of
oxlint's JS plugins alpha feature in this repo
- Manual fixes in spec files: `waitForSelector` → `locator.waitFor`,
deprecated page methods → locator equivalents, `toPass()` timeout
additions
- Compound CSS selectors replaced with `.and()` (Playwright native
locator composition) to avoid `prefer-native-locators` suppressions
- Lint script changes in `package.json` to include `browser_tests/` in
oxlint targets

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
2026-04-11 01:25:14 +00:00
Christian Byrne
577f373cde fix: auto fit-to-view on first subgraph entry (#10995)
## Summary

Auto-fit viewport to subgraph content on first entry so interior nodes
are immediately visible.

## Changes

- **What**: On cache miss in `restoreViewport()`, call `fitView()` via
`requestAnimationFrame` instead of silently returning. Existing
cache-hit path (revisiting a subgraph) is unchanged.

## Review Focus

The `anyItemOverlapsRect` guard in `app.ts` (workflow load path) is
intentionally **not** touched — it serves a different purpose
(respecting `extra.ds` on workflow load). This fix only affects subgraph
navigation transitions where there is no saved viewport to respect.

Fixes #8173

## Screenshots (if applicable)

N/A — viewport positioning fix; before: empty canvas on subgraph entry,
after: nodes visible.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10995-fix-auto-fit-to-view-on-first-subgraph-entry-33d6d73d365081f3a9b3cc2124979624)
by [Unito](https://www.unito.io)
2026-04-11 00:45:38 +00:00
Christian Byrne
44f88027b6 fix: debounce reconnecting toast to prevent false-positive banner (#10997)
## Summary

Debounce the reconnecting toast with a 1.5s grace period so brief
WebSocket blips don't flash a persistent error banner.

## Problem

After #9543 made all error toasts sticky (no auto-dismiss), brief
WebSocket disconnections (<1s) would show a persistent "Reconnecting..."
error banner that stays until the `reconnected` event fires. On staging,
users see this banner even though jobs are actively executing.

## Changes

- Extract reconnection toast logic from `GraphView.vue` into
`useReconnectingNotification` composable
- Add 1.5s delay (via `useTimeoutFn` from VueUse) before showing the
reconnecting toast
- If `reconnected` fires within the delay window, suppress both the
error and success toasts entirely
- Clean up unused `useToast`/`useI18n` imports from `GraphView.vue`

## Testing

- Sub-1.5s disconnections: no toast shown
- Longer disconnections (>1.5s): error toast shown, cleared with success
toast on reconnect
- Setting `Comfy.Toast.DisableReconnectingToast`: no toasts shown at all
- Multiple rapid `reconnecting` events: only one toast shown

6 unit tests covering all scenarios.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10997-fix-debounce-reconnecting-toast-to-prevent-false-positive-banner-33d6d73d3650810289e8f57c046972f1)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-11 00:43:42 +00:00
Comfy Org PR Bot
5d07de1913 1.44.2 (#11151)
Patch version increment to 1.44.2

**Base branch:** `main`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11151-1-44-2-33f6d73d3650815c9767fa5668e67a47)
by [Unito](https://www.unito.io)

Co-authored-by: christian-byrne <72887196+christian-byrne@users.noreply.github.com>
2026-04-11 00:41:25 +00:00
Johnpaul Chiwetelu
f0ae91de43 test: add e2e coverage for alt+drag duplicate and meta multi-select drag (#10994)
## Summary

Add Playwright coverage for two previously-untested canvas gestures:
Alt+drag to duplicate a regular node, and holding Meta across
click-click-click-drag to move multiple selected Vue nodes together.

## Changes

- **What**:
- `browser_tests/tests/interaction.spec.ts` — new `Node Duplication`
sub-describe with `Can duplicate a regular node via Alt+drag`. Asserts
CLIPTextEncode count goes 2 → 3 via poll, original node still exists.
Exercises the legacy canvas path at
`src/lib/litegraph/src/LGraphCanvas.ts:2434-2458`, which was only tested
for subgraph nodes before.
- `browser_tests/tests/vueNodes/interactions/node/move.spec.ts` — new
`should move all selected nodes together when dragging one with Meta
held`. Holds Meta for the entire sequence (3× `click({ modifiers:
['Meta'] })` + drag), asserts selection stays at 3 and all three nodes
move by the same delta. Exercises
`src/renderer/extensions/vueNodes/layout/useNodeDrag.ts:54-191`.
- Small refactor: `getLoadCheckpointHeaderPos` now delegates to a
reusable `getHeaderPos(comfyPage, title)` helper. Added `deltaBetween`
and `expectSameDelta` utilities (stricter than `expectPosChanged`, which
only checks `> 0`).


┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10994-test-add-e2e-coverage-for-alt-drag-duplicate-and-meta-multi-select-drag-33d6d73d3650812dbf15c7053f44f0fd)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-11 00:18:16 +00:00
Alexander Brown
fb8025c49f chore: disable vitest/require-mock-type-parameters oxlint rule (#11146)
## Summary

Disables the `vitest/require-mock-type-parameters` oxlint rule,
eliminating all 2,813 lint warnings in the codebase.

## Details

Every warning came from this single rule requiring explicit type
parameters on `vi.fn()` calls. Investigation showed:

- **85% are bare `vi.fn()`** — no type info available without manual
cross-referencing
- The rule's auto-fixer is **declared but not implemented** — `lint:fix`
doesn't help
- No existing codemods exist for this
- A full manual sweep would take 3–5 days across ~210 test files

## Test Plan

- `pnpm lint` passes with 0 warnings, 0 errors

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11146-fix-disable-vitest-require-mock-type-parameters-oxlint-rule-33e6d73d36508186bf1cdc2ce6d2ba57)
by [Unito](https://www.unito.io)

Co-authored-by: Amp <amp@ampcode.com>
2026-04-11 00:18:09 +00:00
Alexander Brown
5c14badc42 fix(vite): hide git rev-parse window on Windows (#11144)
## Summary

Add `windowsHide: true` to the `execSync('git rev-parse HEAD')` call in
`vite.config.mts` to prevent a console window from flashing on Windows
during builds.

## Changes

- **What**: Pass `windowsHide: true` option to `execSync` when fetching
the git commit hash at build time. This suppresses the transient cmd.exe
popup that appears on Windows.

## Review Focus

Minimal, single-option change. `windowsHide` is a Node.js built-in
option for `child_process` methods — no-op on non-Windows platforms.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11144-fix-vite-hide-git-rev-parse-window-on-Windows-33e6d73d365081ed9a14da5f47ccac4d)
by [Unito](https://www.unito.io)

Co-authored-by: Amp <amp@ampcode.com>
2026-04-10 23:42:26 +00:00
pythongosssss
82fb8ce658 test: App mode - Execution tests (#10801)
## Summary

Adds tests that simulate the execution flow and output feed

## Changes

- **What**: 
- Add ExecutionHelper for mocking network activity
- Refactor ws fixture to use Playwright websocket helper instead of
patching window
- 

## Review Focus

<!-- Critical design decisions or edge cases that need attention -->

<!-- If this PR fixes an issue, uncomment and update the line below -->
<!-- Fixes #ISSUE_NUMBER -->

## Screenshots (if applicable)

<!-- Add screenshots or video recording to help explain your changes -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10801-test-App-mode-Execution-tests-3356d73d365081e4acf0c34378600031)
by [Unito](https://www.unito.io)
2026-04-10 23:31:56 +00:00
Christian Byrne
c3e823e55b fix: use standard size-4 for blueprint action icons (#10992)
## Summary

Fix undersized delete and edit icons on user blueprint items in the node
library sidebar.

## Changes

- **What**: Changed blueprint action icons (trash, edit) from `size-3.5`
(14px) to `size-4` (16px), matching the standard icon size used across
the codebase.

## Review Focus

Trivial sizing fix — `size-4` is the codebase-wide convention for
iconify icons in buttons, and what the button base styles default SVGs
to.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10992-fix-use-standard-size-4-for-blueprint-action-icons-33d6d73d365081be8c65f9e2a7b1d6ec)
by [Unito](https://www.unito.io)
2026-04-10 23:04:17 +00:00
pythongosssss
ebc9025de5 fix/feat: App mode - Persist user resized widget heights (#10993)
## Summary

Saves the user sized textarea/image dropzone elements to the linearData
in the workflow.

## Changes

- **What**: 
- Adds a 3rd element to the linearData input tuple for configuration
data
- Add appmode widget resize composable for persisting resizes
- Tests

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10993-fix-feat-App-mode-Persist-user-resized-widget-heights-33d6d73d36508144b700c6bfcbfa5b3c)
by [Unito](https://www.unito.io)
2026-04-10 22:24:46 +00:00
Alexander Brown
0353524e6f refactor: standardize Page Object locators as public readonly instead of getters (#11135)
*PR Created by the Glary-Bot Agent*

---

## Summary

- Convert ~120 getter-based locators across 18 browser test fixture
files to `public readonly` constructor-assigned properties
- Removes unnecessary indirection, makes object shape explicit, and
improves IDE auto-complete / type inference
- Keeps lazy-init getters (`??=`), computed properties, and `private get
page()` convenience accessors as getters

## Changes

**`browser_tests/fixtures/components/`** (6 files):
`ComfyNodeSearchBox`, `ContextMenu`, `SettingDialog`, `SignInDialog`,
`SidebarTab` (all 6 classes), `Topbar`

**`browser_tests/fixtures/`** (4 files): `ComfyPage` (ComfyMenu.buttons,
ComfyPage.visibleToasts), `UserSelectPage`, `ComfyMouse`,
`VueNodeHelpers`

**`browser_tests/fixtures/helpers/`** (7 files): `AppModeHelper`,
`BuilderFooterHelper`, `BuilderSaveAsHelper`, `BuilderSelectHelper`,
`BuilderStepsHelper`, `ToastHelper`, `NodeOperationsHelper`

**`browser_tests/fixtures/utils/`** (1 file): `vueNodeFixtures`

## Validation

- `pnpm typecheck` 
- `pnpm typecheck:browser` 
- `pnpm exec eslint browser_tests/fixtures/` 
- All pre-commit hooks pass (oxfmt, oxlint, eslint, typecheck,
typecheck:browser) 
- No visual/manual verification needed — changes are test fixture
locator declarations only (no UI or runtime behavior change)

Fixes #11131

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11135-refactor-standardize-Page-Object-locators-as-public-readonly-instead-of-getters-33e6d73d3650819690cbc639f3d30daf)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Glary-Bot <glary-bot@users.noreply.github.com>
2026-04-10 21:10:17 +00:00
Alexander Brown
f1bb756929 fix: remove redundant and counterproductive e2e timeout overrides (#11110)
## Summary

Alright, alright, alright. These e2e tests have been runnin' around like
they're late for somethin', settin' tight little timeouts like the
world's gonna end in 250 milliseconds. Man, you gotta *breathe*. Let the
framework do its thing. Go slow to go fast, that's what I always say.

## Changes

- **What**: Removed ~120 redundant timeout overrides from auto-retrying
Playwright assertions (`toBeVisible`, `toBeHidden`, `toHaveCount`,
`toBeEnabled`, `toHaveAttribute`, `toContainText`, `expect.poll`) where
5000ms is already the default. Also removed sub-5s timeouts (1s, 2s, 3s)
that were just *begging* for flaky failures — like wearin' a belt and
suspenders and also holdin' your pants up with both hands. Raised the
absurdly short timeouts in `customMatchers.ts` (250ms `toPass` → 5000ms,
256ms poll → default). Kept `timeout: 5000` on `.toPass()` calls
(defaults to 0), `.waitFor()`, `waitForRequest`, `waitForFunction`,
intentionally-short timeouts inside retry loops, and conditional
`.isVisible()/.catch()` checks — those fellas actually need the help.

## Review Focus

Every remaining timeout in the diff is there for a *reason*. The ones on
`.toPass()` stay because that API defaults to zero — it won't retry at
all without one. The ones on `.waitFor()` and `waitForRequest` stay
because those are locator actions, not auto-retrying assertions. The
intentionally-short ones inside `toPass` retry loops
(`interaction.spec.ts`) and the negative assertions (`actionbar.spec.ts`
confirming no response arrives) — those are *supposed* to be tight.

The short timeouts on regular assertions were actively *encouragin'*
flaky failures. That's like settin' your alarm for 4 AM and then gettin'
mad you're tired. Just... don't do that, man. Let things take the time
they need.

38 files, net -115 lines. Less code, more chill. That's livin'.

---------

Co-authored-by: Amp <amp@ampcode.com>
2026-04-10 19:47:20 +00:00
Christian Byrne
d70c3cbfc2 fix: remove ticket-intake skill from repo (#11127)
Removes the `.claude/skills/ticket-intake/` directory — personal
pipeline automation that doesn't belong in the shared repository.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11127-fix-remove-ticket-intake-skill-from-repo-33e6d73d365081849f00d7f988ac17e5)
by [Unito](https://www.unito.io)

Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-10 11:42:47 -07:00
pythongosssss
e38dd1efae test: App mode - Add test for cross-workflow input corruption (#10944)
## Summary

Adds test for the issue where inputs from one workflow would overwrite
those on another after sorting

## Changes

- **What**: 
- add test

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10944-test-App-mode-Add-test-for-cross-workflow-input-corruption-33b6d73d365081deab26d8f23d2318ae)
by [Unito](https://www.unito.io)
2026-04-10 17:55:33 +00:00
Comfy Org PR Bot
62779d3c51 1.44.1 (#11007)
Patch version increment to 1.44.1

**Base branch:** `main`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11007-1-44-1-33e6d73d365081e3bbd3c02a3a7ca5d0)
by [Unito](https://www.unito.io)

Co-authored-by: christian-byrne <72887196+christian-byrne@users.noreply.github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-10 17:35:22 +00:00
Christian Byrne
4cf160d66e fix: disable pointer events on non-visible DOM widget overlays (#11063)
## Problem

When a node with DOM widget overlays (e.g. CLIPTextEncode) is collapsed,
the overlay elements can intercept pointer events intended for the
canvas collapse toggler, making click-to-expand unreliable.

## Root Cause

`updateWidgets()` runs during `onDrawForeground` (canvas render cycle)
and sets `widgetState.visible = false` for collapsed nodes. `v-show`
then hides the element with `display: none`. However, there is a timing
gap between the canvas state change and Vue's DOM update — during this
gap the widget overlay still intercepts pointer events.

## Fix

Add `!widgetState.visible` to the `pointerEvents` condition in
`composeStyle()`. This immediately sets `pointer-events: none` when the
widget becomes invisible, preventing event interception before `v-show`
applies `display: none`.

Also restores click-to-expand in the E2E test, removing the programmatic
`node.collapse()` workaround from PR #10967.

- Fixes #11006

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11063-fix-disable-pointer-events-on-non-visible-DOM-widget-overlays-33e6d73d36508179a83cd47121cf933f)
by [Unito](https://www.unito.io)
2026-04-10 14:28:25 +00:00
pythongosssss
c0871ba219 ci: Update actions for queue merge validation (#11114)
## Summary

PR merge queue was enabled but actions were not updated to trigger on
`merge_group`:

https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue#triggering-merge-group-checks-with-github-actions

## Changes

- **What**: 
- `.github/actions/lint-format-verify/action.yml` extrated shared Verify
lint and format steps
- `.github/workflows/ci-lint-format.yaml` updated to use shared
composite action
- `.github/workflows/ci-lint-format-queue.yaml` new action that triggers
on merge_group to validate format
- `.github/workflows/ci-tests-e2e.yaml` triggers on merge_group
- `.github/workflows/ci-tests-unit.yaml` triggers on merge_group

## Review Focus

<!-- Critical design decisions or edge cases that need attention -->

<!-- If this PR fixes an issue, uncomment and update the line below -->
<!-- Fixes #ISSUE_NUMBER -->

## Screenshots (if applicable)

<!-- Add screenshots or video recording to help explain your changes -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11114-ci-Update-actions-for-queue-merge-validation-33e6d73d3650815f8c88f40736b513ec)
by [Unito](https://www.unito.io)
2026-04-10 17:09:48 +00:00
Terry Jia
6d4fc1bb10 fix: resolve incorrect GLSL live preview for non-primitive widget types (#11010)
## Summary

Three issues caused GLSL preview to diverge from backend results:

1. Uniform source resolution always read widgets[0] instead of using
link.origin_slot to select the correct widget. Added directValue
fallback for widgets not registered in widgetValueStore.

2. Hex color strings (e.g. "#45edf5") were coerced to 0 by Number().
Added hexToInt to colorUtil and used it in toNumber coercion.

3. Custom size_mode was ignored — preview always used upstream image
dimensions. Now checks size_mode widget first and respects "custom".

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11010-fix-resolve-incorrect-GLSL-live-preview-for-non-primitive-widget-types-33e6d73d36508101a76bfe8383c0c6ab)
by [Unito](https://www.unito.io)
2026-04-10 10:21:27 -04:00
265 changed files with 8743 additions and 12632 deletions

View File

@@ -1,278 +0,0 @@
---
name: reproduce-issue
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
---
# Issue Reproduction Skill
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
## Architecture
Two videos are produced:
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
```
Phase 1: Research → Read issue, understand prerequisites
Phase 2: Environment → Install custom nodes, load workflows, configure settings
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
Phase 5: Report → Generate a structured reproduction report
```
## Prerequisites
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
- `gh` CLI (authenticated, for reading issues)
- ComfyUI backend with Python environment (for installing custom nodes)
## Phase 1: Research the Issue
1. Fetch the issue details:
```bash
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
```
2. Extract from the issue body:
- **Reproduction steps** (the exact sequence)
- **Prerequisites**: specific workflows, custom nodes, settings, models
- **Environment**: OS, browser, ComfyUI version
- **Media**: screenshots or videos showing the bug
3. Search the codebase for related code:
- Find the feature/component mentioned in the issue
- Understand how it works currently
- Identify what state the UI needs to be in
## Phase 2: Environment Setup
Set up everything the issue requires BEFORE attempting reproduction.
### Custom Nodes
If the issue mentions custom nodes:
```bash
# Find the custom node repo
# Clone into ComfyUI's custom_nodes directory
cd <comfyui_path>/custom_nodes
git clone <custom_node_repo_url>
# Install dependencies if needed
cd <custom_node_name>
pip install -r requirements.txt 2>/dev/null || true
# Restart ComfyUI server to load the new nodes
```
### Workflows
If the issue references a specific workflow:
```bash
# Download workflow JSON if a URL is provided
curl -L "<workflow_url>" -o /tmp/test-workflow.json
# Load it via the API
curl -X POST http://127.0.0.1:8188/api/workflow \
-H "Content-Type: application/json" \
-d @/tmp/test-workflow.json
```
Or load via playwright-cli:
```bash
playwright-cli goto "http://127.0.0.1:8188"
# Drag-and-drop or use File > Open to load the workflow
```
### Settings
If the issue requires specific settings:
```bash
# Use playwright-cli to open settings and change them
playwright-cli press "Control+,"
playwright-cli snapshot
# Find and modify the relevant setting
```
## Phase 3: Interactive Exploration — Research Video
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
```bash
# Open browser and start video recording
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
# Take a snapshot to see current state
playwright-cli snapshot
# Interact based on what you see
playwright-cli click <ref>
playwright-cli fill <ref> "text"
playwright-cli press "Control+s"
# Check results
playwright-cli snapshot
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
```
### Key Principles
- **Observe before acting**: Always `snapshot` before interacting
- **Retry and adapt**: If a step fails, try a different approach
- **Document what works**: Keep notes on which steps trigger the bug
- **Don't give up**: Try multiple approaches if the first doesn't work
- **Establish prerequisites**: Many bugs require specific UI state:
- Save a workflow first (File > Save)
- Make changes to dirty the workflow
- Open multiple tabs
- Add specific node types
- Change settings
- Resize the window
### Common ComfyUI Interactions via playwright-cli
| Action | Command |
| ------------------- | -------------------------------------------------------------- |
| Open hamburger menu | `playwright-cli click` on the C logo button |
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
| Add node | Double-click canvas → type node name → select from results |
| Connect nodes | Drag from output slot to input slot |
| Save workflow | `playwright-cli press "Control+s"` |
| Save As | Menu > File > Save As |
| Select node | Click on the node |
| Delete node | Select → `playwright-cli press "Delete"` |
| Right-click menu | `playwright-cli click <ref> --button right` |
| Keyboard shortcut | `playwright-cli press "Control+z"` |
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
Once the bug is confirmed, **stop the research video** and **close the research browser**:
```bash
playwright-cli video-stop
playwright-cli close
```
Now start a **fresh browser session** for the clean reproduce video (Video 2).
**IMPORTANT constraints:**
- **Max 5 minutes** — the reproduce video must be short and focused
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
1. **Open browser and start recording**:
```bash
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
```
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
3. **Take key screenshots** at critical moments:
```bash
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
# ... trigger the bug ...
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
```
4. **Stop recording and close** immediately after the bug is demonstrated:
```bash
playwright-cli video-stop
playwright-cli close
```
## Phase 5: Generate Report
Create a reproduction report at `tmp/qa/reproduce-report.md`:
```markdown
# Issue Reproduction Report
- **Issue**: <issue_url>
- **Title**: <issue_title>
- **Date**: <today>
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
## Environment
- ComfyUI Server: <url>
- OS: <os>
- Custom Nodes Installed: <list or "none">
- Settings Changed: <list or "none">
## Prerequisites
List everything that had to be set up before the bug could be triggered:
1. ...
2. ...
## Reproduction Steps
Minimal steps to reproduce (the clean sequence):
1. ...
2. ...
3. ...
## Expected Behavior
<from the issue>
## Actual Behavior
<what actually happened>
## Evidence
- Research video: `research-video/video.webm` (full exploration session)
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
- Screenshots: `before-bug.png`, `bug-visible.png`
## Root Cause Analysis (if identified)
<code pointers, hypothesis about what's going wrong>
## Notes
<any additional observations, workarounds discovered, related issues>
```
## Handling Failures
If the bug **cannot be reproduced**:
1. Document what you tried and why it didn't work
2. Check if the issue was already fixed (search git log for related commits)
3. Check if it's environment-specific (OS, browser, specific version)
4. Set report status to "Not Reproduced" with detailed notes
5. The report is still valuable — it saves others from repeating the same investigation
## CI Integration
In CI, this skill runs as a Claude Code agent with:
- `ANTHROPIC_API_KEY` for Claude
- `GEMINI_API_KEY` for initial issue analysis (optional)
- ComfyUI server pre-started in the container
- `playwright-cli` pre-installed
The CI workflow:
1. Gemini generates a reproduce guide (markdown) from the issue
2. Claude agent receives the guide and runs this skill
3. Claude explores interactively, installs dependencies, retries
4. Claude records a clean demo once reproduced
5. Video and report are uploaded as artifacts

View File

@@ -1,283 +0,0 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Automated quality assurance for the ComfyUI frontend. The pipeline reproduces reported bugs using Playwright E2E tests, records video evidence, and deploys reports to Cloudflare Pages.
## Architecture Overview
The QA pipeline uses a **three-phase approach**:
1. **RESEARCH** — Claude writes Playwright E2E tests to reproduce bugs (assertion-backed, no hallucination)
2. **REPRODUCE** — Deterministic replay of the research test with video recording
3. **REPORT** — Deploy results to Cloudflare Pages with badge, video, and verdict
### Key Design Decision
Earlier iterations used AI vision (Gemini) to drive a browser and judge results from video. This was abandoned after discovering **AI reviewers hallucinate** — Gemini reported "REPRODUCED" when videos showed idle screens. The current approach uses **Playwright assertions** as the source of truth: if the test passes, the bug is proven.
## Prerequisites
- Node.js 22+
- `pnpm` package manager
- `gh` CLI (authenticated)
- Playwright browsers: `npx playwright install chromium`
- Environment variables:
- `GEMINI_API_KEY` — for PR analysis and video review
- `ANTHROPIC_API_KEY` — for Claude Agent SDK (research phase)
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — for report deployment
## Pipeline Scripts
| Script | Role | Model |
| --------------------------------- | ------------------------------------------------------- | ----------------------------- |
| `scripts/qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide | gemini-3.1-pro-preview |
| `scripts/qa-agent.ts` | Research phase: Claude writes E2E tests | claude-sonnet-4-6 (Agent SDK) |
| `scripts/qa-record.ts` | Before/after video recording with Gemini-driven actions | gemini-3.1-pro-preview |
| `scripts/qa-reproduce.ts` | Deterministic replay with narration | gemini-3-flash-preview |
| `scripts/qa-video-review.ts` | Video comparison review | gemini-3-flash-preview |
| `scripts/qa-generate-test.ts` | Regression test generation from QA report | gemini-3-flash-preview |
| `scripts/qa-deploy-pages.sh` | Deploy to Cloudflare Pages + badge | — |
| `scripts/qa-batch.sh` | Batch-trigger QA for multiple issues | — |
| `scripts/qa-report-template.html` | Report site (light/dark, seekbar, copy badge) | — |
## Triggering QA
### Via GitHub Labels
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after comparison)
- **`qa-full`** — Full QA (3-OS matrix, after-only)
- **`qa-issue`** — Reproduce a bug from an issue
### Via Batch Script
```bash
# Trigger QA for specific issue numbers
./scripts/qa-batch.sh 10394 10238 9996
# From a triage file (top 5 Tier 1 issues)
./scripts/qa-batch.sh --from tmp/issues.md --top 5
# Preview without pushing
./scripts/qa-batch.sh --dry-run 10394
# Clean up old trigger branches
./scripts/qa-batch.sh --cleanup
```
### Via Workflow Dispatch
Go to Actions → "PR: QA" → Run workflow → choose mode (focused/full).
## CI Workflow (`.github/workflows/pr-qa.yaml`)
```
resolve-matrix → analyze-pr ──┐
├→ qa-before (main branch, worktree build)
├→ qa-after (PR branch)
└→ report (video review, deploy, comment)
```
Before/after jobs run **in parallel** on separate runners for clean isolation.
### Issue Reproduce Mode
For issues (not PRs), the pipeline:
1. Fetches the issue body and comments
2. Runs `qa-analyze-pr.ts --type issue` to generate a QA guide
3. Runs the research phase (Claude writes E2E test to reproduce)
4. Records video of the test execution
5. Posts results as a comment on the issue
## Running Locally
### Step 1: Environment Setup
```bash
# Ensure ComfyUI server is running
# Default: http://127.0.0.1:8188
# Install Playwright browsers
npx playwright install chromium
```
### Step 2: Analyze the Issue/PR
```bash
# For a PR
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides
# For an issue
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides \
--type issue
```
### Step 3: Record Before/After
```bash
# Before (main branch)
pnpm exec tsx scripts/qa-record.ts \
--mode before \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-before \
--qa-guide qa-guides/qa-guide-1.json
# After (PR branch)
pnpm exec tsx scripts/qa-record.ts \
--mode after \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-after \
--qa-guide qa-guides/qa-guide-1.json
```
### Step 4: Review Videos
```bash
pnpm exec tsx scripts/qa-video-review.ts \
--artifacts-dir /tmp/qa-artifacts \
--video-file qa-session.mp4 \
--before-video qa-before-session.mp4 \
--output-dir /tmp/video-reviews \
--pr-context /tmp/pr-context.txt
```
## Research Phase Details (`qa-agent.ts`)
Claude receives:
- The issue description and comments
- A QA guide from `qa-analyze-pr.ts`
- An accessibility tree snapshot of the current UI
Claude's tools:
- **`inspect(selector?)`** — Read a11y tree to discover element selectors
- **`writeTest(code)`** — Write a Playwright `.spec.ts` file
- **`runTest()`** — Execute the test and get pass/fail + errors
- **`done(verdict, summary, evidence, testCode)`** — Finish with verdict
The test uses the project's Playwright fixtures (`comfyPageFixture`), giving access to `comfyPage.page`, `comfyPage.menu`, `comfyPage.settings`, etc.
### Verdict Logic
- **REPRODUCED** — Test passes (asserting the bug exists) → bug is proven
- **NOT_REPRODUCIBLE** — Claude exhausted attempts, test cannot pass
- **INCONCLUSIVE** — Agent timed out or encountered infrastructure issues
Auto-completion: if a test passed but `done()` was never called, the pipeline auto-completes with REPRODUCED.
## Manual QA (Fallback)
When the automated pipeline isn't suitable (e.g., visual-only bugs, complex multi-step interactions), use **playwright-cli** for manual browser interaction:
```bash
# Install
npm install -g @playwright/cli@latest
# Open browser and navigate
playwright-cli open http://127.0.0.1:8188
# Get element references
playwright-cli snapshot
# Interact
playwright-cli click e1
playwright-cli fill e2 "test text"
playwright-cli press Escape
playwright-cli screenshot --filename=f.png
```
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation to refresh refs.
## Manual QA Test Plan
When performing manual QA (either via playwright-cli or the automated pipeline), systematically test each area below.
### Application Load & Routes
| Test | Steps |
| ----------------- | ------------------------------------------------------------ |
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### Canvas & Graph View
| Test | Steps |
| ------------------------- | -------------------------------------------------------------- |
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Context menus | Right-click node vs empty canvas — different menus |
### Sidebar Tabs
| Test | Steps |
| ----------------- | ------------------------------------- |
| Workflows tab | Press W — workflows sidebar opens |
| Node Library tab | Press N — node library opens |
| Model Library tab | Press M — model library opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
### Settings Dialog
| Test | Steps |
| ---------------- | ---------------------------------------------------- |
| Open settings | Press Ctrl+, or click settings button |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Close settings | Press Escape or click close button |
### Execution & Queue
| Test | Steps |
| -------------- | ----------------------------------------------------- |
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
## Report Site
Deployed to Cloudflare Pages at `https://comfy-qa.pages.dev/<branch>/`.
Features:
- Light/dark theme
- Seekable video player with preload
- Copy badge button (markdown)
- Date-stamped badges (e.g., `QA0327`)
- Vertical box badge for issues and PRs
## Known Issues & Troubleshooting
See `docs/qa/TROUBLESHOOTING.md` for common failures:
- `set -euo pipefail` + grep with no match → append `|| true`
- `__name is not defined` in `page.evaluate` → use `addScriptTag`
- Cursor not visible in videos → monkey-patch `page.mouse` methods
- Agent not calling `done()` → auto-complete from passing test
## Backlog
See `docs/qa/backlog.md` for planned improvements:
- **Type B comparison**: Different commits for regression detection
- **Type C comparison**: Cross-browser testing
- **Pre-seed assets**: Upload test images before recording
- **Lazy a11y tree**: Reduce token usage with `inspect(selector)` vs full dump

View File

@@ -1,668 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
*
* Instead of driving a browser interactively, Claude:
* 1. Reads the issue + a11y snapshot of the UI
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
* 3. Runs the test → reads errors → rewrites → repeats until it works
* 4. Outputs the passing test + verdict
*
* Tools:
* - inspect(selector) — read a11y tree to understand UI state
* - writeTest(code) — write a Playwright test file
* - runTest() — execute the test and get results
* - done(verdict, summary, testCode) — finish with the working test
*/
import type { Page } from '@playwright/test'
/* eslint-disable import-x/no-unresolved */
// @ts-expect-error — claude-agent-sdk has no type declarations for vue-tsc
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
/* eslint-enable import-x/no-unresolved */
import { z } from 'zod'
import { mkdirSync, readFileSync, writeFileSync } from 'fs'
import { execSync } from 'child_process'
// ── Types ──
interface ResearchOptions {
page: Page
issueContext: string
qaGuide: string
outputDir: string
serverUrl: string
anthropicApiKey?: string
maxTurns?: number
timeBudgetMs?: number
}
export type ReproMethod = 'e2e_test' | 'video' | 'both' | 'none'
export interface ResearchResult {
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
reproducedBy: ReproMethod
summary: string
evidence: string
testCode: string
videoScript?: string
log: Array<{
turn: number
timestampMs: number
toolName: string
toolInput: unknown
toolResult: string
}>
}
// ── Main research function ──
export async function runResearchPhase(
opts: ResearchOptions
): Promise<ResearchResult> {
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
opts
const maxTurns = opts.maxTurns ?? 50
let agentDone = false
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
let finalReproducedBy: ReproMethod = 'none'
let finalSummary = 'Agent did not complete'
let finalEvidence = ''
let finalTestCode = ''
let finalVideoScript = ''
let turnCount = 0
let lastPassedTurn = -1
const startTime = Date.now()
const researchLog: ResearchResult['log'] = []
const testDir = `${outputDir}/research`
mkdirSync(testDir, { recursive: true })
const testPath = `${testDir}/reproduce.spec.ts`
// Get initial a11y snapshot for context
let initialA11y = ''
try {
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
initialA11y = initialA11y.slice(0, 3000)
} catch {
initialA11y = '(could not capture initial a11y snapshot)'
}
// ── Tool: inspect ──
const inspectTool = tool(
'inspect',
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
{
selector: z
.string()
.optional()
.describe(
'Optional filter — only show elements matching this name/role. Omit for full tree.'
)
},
async (args: { selector?: string }) => {
let resultText: string
try {
const ariaText = await page
.locator('body')
.ariaSnapshot({ timeout: 5000 })
if (args.selector) {
const lines = ariaText.split('\n')
const matches = lines.filter((l: string) =>
l.toLowerCase().includes(args.selector!.toLowerCase())
)
resultText =
matches.length > 0
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
} else {
resultText = ariaText.slice(0, 3000)
}
} catch (e) {
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'inspect',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readFixture ──
const readFixtureTool = tool(
'readFixture',
'Read a fixture or helper file from browser_tests/fixtures/ to understand the API. Use this to discover available methods on comfyPage helpers before writing your test.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/fixtures/, e.g. "helpers/CanvasHelper.ts" or "components/Topbar.ts" or "ComfyPage.ts"'
)
},
async (args: { path: string }) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/fixtures/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
resultText = `Could not read fixture: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readFixture',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readTest ──
const readTestTool = tool(
'readTest',
'Read an existing E2E test file from browser_tests/tests/ to learn patterns and conventions used in this project.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/tests/, e.g. "workflow.spec.ts" or "subgraph.spec.ts"'
)
},
async (args: { path: string }) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/tests/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
// List available test files if the path doesn't exist
try {
const { readdirSync } = await import('fs')
const files = readdirSync(`${projectRoot}/browser_tests/tests/`)
.filter((f: string) => f.endsWith('.spec.ts'))
.slice(0, 30)
resultText = `File not found: ${args.path}\n\nAvailable test files:\n${files.join('\n')}`
} catch {
resultText = `Could not read test: ${e instanceof Error ? e.message : e}`
}
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readTest',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: writeTest ──
const writeTestTool = tool(
'writeTest',
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
{
code: z
.string()
.describe('Complete Playwright test file content (.spec.ts)')
},
async (args: { code: string }) => {
writeFileSync(testPath, args.code)
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'writeTest',
toolInput: { path: testPath, codeLength: args.code.length },
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
})
return {
content: [
{
type: 'text' as const,
text: `Test written to ${testPath}. Use runTest() to execute it.`
}
]
}
}
)
// ── Tool: runTest ──
// Place test in browser_tests/ so Playwright config finds fixtures
const projectRoot = process.cwd()
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
const runTestTool = tool(
'runTest',
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
{},
async () => {
turnCount++
// Copy the test to browser_tests/tests/ where Playwright expects it
const { copyFileSync } = await import('fs')
try {
copyFileSync(testPath, browserTestPath)
} catch {
// directory may not exist
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
copyFileSync(testPath, browserTestPath)
}
let resultText: string
try {
const output = execSync(
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
{
timeout: 90000,
encoding: 'utf-8',
env: {
...process.env,
COMFYUI_BASE_URL: serverUrl
}
}
)
resultText = `TEST PASSED:\n${output.slice(-1500)}`
} catch (e) {
const err = e as { stdout?: string; stderr?: string; message?: string }
const output = (err.stdout || '') + '\n' + (err.stderr || '')
resultText = `TEST FAILED:\n${output.slice(-2000)}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'runTest',
toolInput: { testPath },
toolResult: resultText.slice(0, 1000)
})
// Auto-save passing test code for fallback completion — but only if
// the test contains a bug-specific assertion (not just a discovery/debug test)
if (resultText.startsWith('TEST PASSED')) {
try {
const code = readFileSync(browserTestPath, 'utf-8')
const hasBugAssertion =
/expect\s*\(/.test(code) &&
!/^\s*expect\([^)]+\)\.toBeDefined\(\)/m.test(code) &&
!/^\s*expect\([^)]+\)\.toBeGreaterThan\(0\)/m.test(code) &&
!/Inspect|Find|Debug|discover/i.test(
code.match(/test\(['"`]([^'"`]+)/)?.[1] ?? ''
)
if (hasBugAssertion) {
finalTestCode = code
lastPassedTurn = turnCount
}
} catch {
// ignore
}
resultText +=
'\n\n⚠ Test PASSED — call done() now with verdict REPRODUCED and the test code. Do NOT write more tests.'
}
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: done ──
const doneTool = tool(
'done',
'Finish research with verdict and the final test code.',
{
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
reproducedBy: z
.enum(['e2e_test', 'video', 'both', 'none'])
.describe(
'How the bug was proven: e2e_test = Playwright assertion passed, video = visual evidence only, both = both methods, none = not reproduced'
),
summary: z.string().describe('What you found and why'),
evidence: z.string().describe('Test output that proves the verdict'),
testCode: z
.string()
.describe(
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
),
videoScript: z
.string()
.optional()
.describe(
'Demowright video script for Phase 2 demo recording. REQUIRED when verdict is REPRODUCED. A separate test file using createVideoScript with title, segments, and outro. Do NOT include demowright imports in testCode.'
)
},
async (args: {
verdict: ResearchResult['verdict']
reproducedBy: ReproMethod
summary: string
evidence: string
testCode: string
videoScript?: string
}) => {
agentDone = true
finalVerdict = args.verdict
finalReproducedBy = args.reproducedBy
finalSummary = args.summary
finalEvidence = args.evidence
finalTestCode = args.testCode
finalVideoScript = args.videoScript ?? ''
writeFileSync(testPath, args.testCode)
if (args.videoScript) {
writeFileSync(`${outputDir}/video-script.spec.ts`, args.videoScript)
}
return {
content: [
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
]
}
}
)
// ── MCP Server ──
const server = createSdkMcpServer({
name: 'qa-research',
version: '1.0.0',
tools: [
inspectTool,
readFixtureTool,
readTestTool,
writeTestTool,
runTestTool,
doneTool
]
})
// ── System prompt ──
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
## Your tools
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
- readFixture(path) — Read fixture source code from browser_tests/fixtures/. Use to discover available methods. E.g. "helpers/CanvasHelper.ts", "components/Topbar.ts", "ComfyPage.ts"
- readTest(path) — Read an existing test from browser_tests/tests/ to learn patterns. E.g. "workflow.spec.ts". Pass any name to list available files.
- writeTest(code) — Write a Playwright test file (.spec.ts)
- runTest() — Execute the test and get results (pass/fail + errors)
- done(verdict, summary, evidence, testCode) — Finish with the final test
## Workflow
1. Read the issue description carefully
2. Use inspect() to understand the current UI state and discover element selectors
3. If unsure about the fixture API, use readFixture() to read the relevant helper source code
4. If unsure about test patterns, use readTest() to read an existing test for reference
5. Write a Playwright test that:
- Performs the exact reproduction steps from the issue
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
6. Run the test with runTest()
7. If it fails: read the error, fix the test, run again (max 5 attempts)
8. Call done() with the final verdict and test code
## Test writing guidelines
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
- Import expect: \`import { expect } from '@playwright/test'\`
- The fixture provides \`comfyPage\` which has all the helpers listed below
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
- Keep tests focused and minimal — test ONLY the reported bug
- Write ONE test, not multiple. Focus on the single clearest reproduction.
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
- Use \`comfyPage.nextFrame()\` after interactions that trigger UI updates
- NEVER use \`page.waitForTimeout()\` — use Locator actions and retrying assertions instead
- ALWAYS call done() when finished, even if the test passed — do not keep iterating after a passing test
- Use \`expect.poll()\` for async assertions: \`await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(8)\`
- CRITICAL: Your assertions must be SPECIFIC TO THE BUG. A test that asserts \`expect(count).toBeGreaterThan(0)\` proves nothing — it would pass even without the bug. Instead assert the exact broken state, e.g. \`expect(clonedWidgets).toHaveLength(0)\` (missing widgets) or \`expect(zIndex).toBeLessThan(parentZIndex)\` (wrong z-order). If a test passes trivially, it's a false positive.
- NEVER write "debug", "discovery", or "inspect node types" tests. These waste turns and produce false REPRODUCED verdicts. If you need to discover node type names, use inspect() or readFixture() — not a passing test.
- If you cannot write a bug-specific assertion, call done() with verdict NOT_REPRODUCIBLE and explain why.
## ComfyPage Fixture API Reference
### Core properties
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.canvas\` — Locator for #graph-canvas
- \`comfyPage.queueButton\` — "Queue Prompt" button
- \`comfyPage.runButton\` — "Run" button (new UI)
- \`comfyPage.confirmDialog\` — ConfirmDialog (has .confirm, .delete, .overwrite, .reject locators + .click(name) method)
- \`comfyPage.nextFrame()\` — wait for next requestAnimationFrame
- \`comfyPage.setup()\` — navigate + wait for app ready (called automatically by fixture)
### Menu (comfyPage.menu)
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.triggerTopbarCommand(['File', 'Save As'])\` — navigate menu hierarchy
- \`.openTopbarMenu()\` / \`.closeTopbarMenu()\` — open/close hamburger
- \`.openSubmenu('File')\` — hover to open submenu, returns submenu Locator
- \`.getTabNames()\` — get all open workflow tab names
- \`.getActiveTabName()\` — get active tab name
- \`.getWorkflowTab(name)\` — get tab Locator
- \`.closeWorkflowTab(name)\` — close a tab
- \`.saveWorkflow(name)\` / \`.saveWorkflowAs(name)\` / \`.exportWorkflow(name)\`
- \`.switchTheme('dark' | 'light')\`
- \`comfyPage.menu.workflowsTab\` — WorkflowsSidebarTab:
- \`.open()\` / \`.close()\` — toggle workflows sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflow names
- \`comfyPage.menu.nodeLibraryTab\` — NodeLibrarySidebarTab
- \`comfyPage.menu.assetsTab\` — AssetsSidebarTab
### Canvas (comfyPage.canvasOps)
- \`.click({x, y})\` — click at position on canvas
- \`.rightClick(x, y)\` — right-click (opens context menu)
- \`.doubleClick()\` — double-click canvas (opens node search)
- \`.clickEmptySpace()\` — click known empty area
- \`.dragAndDrop(source, target)\` — drag from source to target position
- \`.pan(offset, safeSpot?)\` — pan canvas by offset
- \`.zoom(deltaY, steps?)\` — zoom via scroll wheel
- \`.resetView()\` — reset zoom/pan to default
- \`.getScale()\` / \`.setScale(n)\` — get/set canvas zoom
- \`.getNodeCenterByTitle(title)\` — get screen coords of node center
- \`.disconnectEdge()\` / \`.connectEdge()\` — default graph edge operations
### Node Operations (comfyPage.nodeOps)
- \`.getGraphNodesCount()\` — count all nodes
- \`.getSelectedGraphNodesCount()\` — count selected nodes
- \`.getNodes()\` — get all nodes
- \`.getFirstNodeRef()\` — get NodeReference for first node
- \`.getNodeRefById(id)\` — get NodeReference by ID
- \`.getNodeRefsByType(type)\` — get all nodes of a type
- \`.waitForGraphNodes(count)\` — wait until node count matches
### Settings (comfyPage.settings)
- \`.setSetting(id, value)\` — change a ComfyUI setting
- \`.getSetting(id)\` — read current setting value
### Keyboard (comfyPage.keyboard)
- \`.undo()\` / \`.redo()\` — Ctrl+Z / Ctrl+Y
- \`.bypass()\` — Ctrl+B
- \`.selectAll()\` — Ctrl+A
- \`.ctrlSend(key)\` — send Ctrl+key
### Workflow (comfyPage.workflow)
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\`
- \`.isCurrentWorkflowModified()\` — check dirty state
### Context Menu (comfyPage.contextMenu)
- \`.openFor(locator)\` — right-click locator and wait for menu
- \`.clickMenuItem(name)\` — click a menu item by name
- \`.isVisible()\` — check if context menu is showing
- \`.assertHasItems(items)\` — assert menu contains items
### Other helpers
- \`comfyPage.settingDialog\` — SettingDialog component
- \`comfyPage.searchBox\` / \`comfyPage.searchBoxV2\` — node search
- \`comfyPage.toast\` — ToastHelper (\`.visibleToasts\`)
- \`comfyPage.subgraph\` — SubgraphHelper
- \`comfyPage.vueNodes\` — VueNodeHelpers
- \`comfyPage.bottomPanel\` — BottomPanel
- \`comfyPage.clipboard\` — ClipboardHelper
- \`comfyPage.dragDrop\` — DragDropHelper
### Available fixture files (use readFixture to explore)
- ComfyPage.ts — main fixture with all helpers
- helpers/CanvasHelper.ts, NodeOperationsHelper.ts, WorkflowHelper.ts
- helpers/KeyboardHelper.ts, SettingsHelper.ts, SubgraphHelper.ts
- components/Topbar.ts, ContextMenu.ts, SettingDialog.ts, SidebarTab.ts
## Video Script (IMPORTANT — provide via done() tool)
When calling done(), provide a \`videoScript\` parameter with a SEPARATE test file that uses demowright's createVideoScript.
Do NOT put demowright imports in testCode — they won't resolve in Phase 1.
The videoScript is a complete, standalone Playwright test file for Phase 2 demo recording:
\`\`\`typescript
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
import { createVideoScript } from 'demowright/video-script'
test('Demo: Bug Title', async ({ comfyPage }) => {
// IMPORTANT: ALL setup code MUST go here BEFORE createVideoScript()
// so the title card is the FIRST thing viewers see in the video
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.workflow.setupWorkflowsDirectory({})
const script = createVideoScript()
.title('Bug Title Here', { subtitle: 'Issue #NNNN', durationMs: 4000 })
.segment('Step 1: description of what we do', async (pace) => {
await pace() // narration finishes FIRST
await comfyPage.menu.topbar.saveWorkflow('name') // THEN action
await comfyPage.page.waitForTimeout(2000) // pause for viewer
})
.segment('Bug evidence: what we see proves the bug', async (pace) => {
await pace()
await comfyPage.page.waitForTimeout(5000) // hold on evidence
})
.outro({ text: 'Bug Reproduced', subtitle: 'Summary' })
await script.render(comfyPage.page)
})
\`\`\`
Key API:
- \`.title(text, {subtitle?, durationMs?})\` — title card (4s default)
- \`.segment(narrationText, async (pace) => { await pace(); ...actions... })\` — TTS narrated step
- \`.outro({text?, subtitle?, durationMs?})\` — ending card
- \`pace()\` — wait for narration audio to finish
CRITICAL TIMING: Call \`await pace()\` FIRST in each segment, BEFORE the Playwright actions.
This makes the narration play and finish, THEN the actions execute — so viewers hear what's about
to happen before it happens. Pattern:
\`\`\`typescript
.segment('Now we save the workflow as a new name', async (pace) => {
await pace() // narration finishes first
await comfyPage.menu.topbar.saveWorkflowAs('new-name') // then action happens
await comfyPage.page.waitForTimeout(2000) // pause so viewer sees the result
})
\`\`\`
IMPORTANT RULES for videoScript:
1. You MUST provide videoScript when verdict is REPRODUCED — every reproduced bug needs a narrated demo
2. ALL setup code (setSetting, setupWorkflowsDirectory) goes BEFORE createVideoScript() — title card must be first thing in video
3. Call \`await pace()\` FIRST in each segment callback, BEFORE actions
4. Add \`waitForTimeout(2000)\` after each action so viewers can see the result
5. Final evidence segment: hold for 5+ seconds
6. Reproduce the same steps as testCode but slower with clear narration
## Current UI state (accessibility tree)
${initialA11y}
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
## Issue to Reproduce
${issueContext}`
// ── Run the agent ──
console.warn('Starting research phase (Claude writes E2E tests)...')
try {
for await (const message of query({
prompt:
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, readFixture() or readTest() if you need to understand the fixture API or see existing test patterns, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
options: {
model: 'claude-sonnet-4-6',
systemPrompt,
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
maxTurns,
mcpServers: { 'qa-research': server },
allowedTools: [
'mcp__qa-research__inspect',
'mcp__qa-research__readFixture',
'mcp__qa-research__readTest',
'mcp__qa-research__writeTest',
'mcp__qa-research__runTest',
'mcp__qa-research__done'
]
}
})) {
if (message.type === 'assistant' && message.message?.content) {
for (const block of message.message.content) {
if ('text' in block && block.text) {
console.warn(` Claude: ${block.text.slice(0, 200)}`)
}
if ('name' in block) {
console.warn(
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
)
}
}
}
if (agentDone) break
}
} catch (e) {
const errMsg = e instanceof Error ? e.message : String(e)
console.warn(`Research error: ${errMsg}`)
// Detect billing/auth errors and surface them clearly
if (
errMsg.includes('Credit balance is too low') ||
errMsg.includes('insufficient_quota') ||
errMsg.includes('rate_limit')
) {
finalSummary = `API error: ${errMsg.slice(0, 200)}`
finalEvidence = 'Agent could not start due to API billing/auth issue'
console.warn(
'::error::Anthropic API credits exhausted — cannot run research phase'
)
}
}
// Auto-complete: if a test passed but done() was never called, use the passing test
if (!agentDone && lastPassedTurn >= 0 && finalTestCode) {
console.warn(
`Auto-completing: test passed at turn ${lastPassedTurn} but done() was not called`
)
finalVerdict = 'REPRODUCED'
finalReproducedBy = 'e2e_test'
finalSummary = `Test passed at turn ${lastPassedTurn} (auto-completed — agent did not call done())`
finalEvidence = `Test passed with exit code 0`
}
const result: ResearchResult = {
verdict: finalVerdict,
reproducedBy: finalReproducedBy,
summary: finalSummary,
evidence: finalEvidence,
testCode: finalTestCode,
videoScript: finalVideoScript || undefined,
log: researchLog
}
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
console.warn(
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
)
return result
}

View File

@@ -1,84 +0,0 @@
import { describe, expect, it } from 'vitest'
import { extractMediaUrls } from './qa-analyze-pr'
describe('extractMediaUrls', () => {
it('extracts markdown image URLs', () => {
const text = '![screenshot](https://example.com/image.png)'
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
})
it('extracts multiple markdown images', () => {
const text = [
'![before](https://example.com/before.png)',
'Some text',
'![after](https://example.com/after.jpg)'
].join('\n')
expect(extractMediaUrls(text)).toEqual([
'https://example.com/before.png',
'https://example.com/after.jpg'
])
})
it('extracts raw URLs with media extensions', () => {
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
})
it('extracts GitHub user-attachments URLs', () => {
const text =
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
expect(extractMediaUrls(text)).toEqual([
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
])
})
it('extracts private-user-images URLs', () => {
const text =
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
expect(extractMediaUrls(text)).toEqual([
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
])
})
it('extracts URLs with query parameters', () => {
const text = 'https://example.com/image.png?w=800&h=600'
expect(extractMediaUrls(text)).toEqual([
'https://example.com/image.png?w=800&h=600'
])
})
it('deduplicates URLs', () => {
const text = [
'![img](https://example.com/same.png)',
'![img2](https://example.com/same.png)',
'Also https://example.com/same.png'
].join('\n')
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
})
it('returns empty array for empty input', () => {
expect(extractMediaUrls('')).toEqual([])
})
it('returns empty array for text with no media URLs', () => {
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
})
it('handles mixed media types', () => {
const text = [
'![screen](https://example.com/screenshot.png)',
'Video: https://example.com/demo.webm',
'![gif](https://example.com/animation.gif)'
].join('\n')
const urls = extractMediaUrls(text)
expect(urls).toContain('https://example.com/screenshot.png')
expect(urls).toContain('https://example.com/demo.webm')
expect(urls).toContain('https://example.com/animation.gif')
})
it('ignores non-http URLs in markdown', () => {
const text = '![local](./local-image.png)'
expect(extractMediaUrls(text)).toEqual([])
})
})

View File

@@ -1,799 +0,0 @@
#!/usr/bin/env tsx
/**
* QA PR Analysis Script
*
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
* for before/after recording sessions. Fetches PR thread, extracts media,
* and produces structured test plans.
*
* Usage:
* pnpm exec tsx scripts/qa-analyze-pr.ts \
* --pr-number 10270 \
* --repo owner/repo \
* --output-dir qa-guides/ \
* [--model gemini-3.1-pro-preview]
*
* Env: GEMINI_API_KEY (required)
*/
import { execSync } from 'node:child_process'
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
// ── Types ──
interface QaGuideStep {
action: string
description: string
expected_before?: string
expected_after?: string
}
interface QaGuide {
summary: string
test_focus: string
prerequisites: string[]
steps: QaGuideStep[]
visual_checks: string[]
}
interface PrThread {
title: string
body: string
labels: string[]
issueComments: string[]
reviewComments: string[]
reviews: string[]
diff: string
}
type TargetType = 'pr' | 'issue'
interface Options {
prNumber: string
repo: string
outputDir: string
model: string
apiKey: string
mediaBudgetBytes: number
maxVideoBytes: number
type: TargetType
}
// ── CLI parsing ──
function parseArgs(): Options {
const args = process.argv.slice(2)
const opts: Partial<Options> = {
model: 'gemini-3.1-pro-preview',
apiKey: process.env.GEMINI_API_KEY || '',
mediaBudgetBytes: 20 * 1024 * 1024,
maxVideoBytes: 10 * 1024 * 1024,
type: 'pr'
}
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--pr-number':
opts.prNumber = args[++i]
break
case '--repo':
opts.repo = args[++i]
break
case '--output-dir':
opts.outputDir = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--type':
opts.type = args[++i] as TargetType
break
case '--help':
console.warn(
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
)
process.exit(0)
}
}
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
console.error(
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
)
process.exit(1)
}
if (!opts.apiKey) {
console.error('GEMINI_API_KEY environment variable is required')
process.exit(1)
}
return opts as Options
}
// ── PR thread fetching ──
function ghExec(cmd: string): string {
try {
return execSync(cmd, {
encoding: 'utf-8',
timeout: 30_000,
stdio: ['pipe', 'pipe', 'pipe']
}).trim()
} catch (err) {
console.warn(`gh command failed: ${cmd}`)
console.warn((err as Error).message)
return ''
}
}
function fetchPrThread(prNumber: string, repo: string): PrThread {
console.warn('Fetching PR thread...')
const prView = ghExec(
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
)
const prData = prView
? JSON.parse(prView)
: { title: '', body: '', labels: [] }
const issueCommentsRaw = ghExec(
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
)
const issueComments: string[] = issueCommentsRaw
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewCommentsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
)
const reviewComments: string[] = reviewCommentsRaw
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
)
const reviews: string[] = reviewsRaw
? JSON.parse(reviewsRaw)
.filter((r: { body: string }) => r.body)
.map((r: { body: string }) => r.body)
: []
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
console.warn(
`PR #${prNumber}: "${prData.title}" | ` +
`${issueComments.length} issue comments, ` +
`${reviewComments.length} review comments, ` +
`${reviews.length} reviews, ` +
`diff: ${diff.length} chars`
)
return {
title: prData.title || '',
body: prData.body || '',
labels: (prData.labels || []).map((l: { name: string }) => l.name),
issueComments,
reviewComments,
reviews,
diff
}
}
interface IssueThread {
title: string
body: string
labels: string[]
comments: string[]
}
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
console.warn('Fetching issue thread...')
const issueView = ghExec(
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
)
const issueData = issueView
? JSON.parse(issueView)
: { title: '', body: '', labels: [] }
const commentsRaw = ghExec(
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
)
const comments: string[] = commentsRaw
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
: []
console.warn(
`Issue #${issueNumber}: "${issueData.title}" | ` +
`${comments.length} comments`
)
return {
title: issueData.title || '',
body: issueData.body || '',
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
comments
}
}
// ── Media extraction ──
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
const MEDIA_URL_PATTERNS = [
// Markdown images: ![alt](url)
/!\[[^\]]*\]\(([^)]+)\)/g,
// GitHub user-attachments
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
// Private user images
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
// Raw URLs with media extensions (standalone or in text)
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
]
export function extractMediaUrls(text: string): string[] {
if (!text) return []
const urls = new Set<string>()
for (const pattern of MEDIA_URL_PATTERNS) {
// Reset lastIndex for global patterns
pattern.lastIndex = 0
let match: RegExpExecArray | null
while ((match = pattern.exec(text)) !== null) {
// For markdown images, the URL is in capture group 1
const url = match[1] || match[0]
// Clean trailing markdown/html artifacts
const cleaned = url.replace(/[)>"'\s]+$/, '')
if (cleaned.startsWith('http')) {
urls.add(cleaned)
}
}
}
return [...urls]
}
// ── Media downloading ──
const ALLOWED_MEDIA_DOMAINS = [
'github.com',
'raw.githubusercontent.com',
'user-images.githubusercontent.com',
'private-user-images.githubusercontent.com',
'objects.githubusercontent.com',
'github.githubassets.com'
]
function isAllowedMediaDomain(url: string): boolean {
try {
const hostname = new URL(url).hostname
return ALLOWED_MEDIA_DOMAINS.some(
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
)
} catch {
return false
}
}
async function downloadMedia(
urls: string[],
outputDir: string,
budgetBytes: number,
maxVideoBytes: number
): Promise<Array<{ path: string; mimeType: string }>> {
const downloaded: Array<{ path: string; mimeType: string }> = []
let totalBytes = 0
const mediaDir = resolve(outputDir, 'media')
mkdirSync(mediaDir, { recursive: true })
for (const url of urls) {
if (totalBytes >= budgetBytes) {
console.warn(
`Media budget exhausted (${totalBytes} bytes), skipping rest`
)
break
}
if (!isAllowedMediaDomain(url)) {
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
continue
}
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(15_000),
headers: { Accept: 'image/*,video/*' },
redirect: 'follow'
})
if (!response.ok) {
console.warn(`Failed to download ${url}: ${response.status}`)
continue
}
const contentLength = response.headers.get('content-length')
if (contentLength) {
const declaredSize = Number.parseInt(contentLength, 10)
if (declaredSize > budgetBytes - totalBytes) {
console.warn(
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
)
continue
}
}
const contentType = response.headers.get('content-type') || ''
const buffer = Buffer.from(await response.arrayBuffer())
// Skip oversized videos
const isVideo =
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
if (isVideo && buffer.length > maxVideoBytes) {
console.warn(
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
)
continue
}
if (totalBytes + buffer.length > budgetBytes) {
console.warn(`Would exceed budget, skipping ${url}`)
continue
}
const ext = guessExtension(url, contentType)
const filename = `media-${downloaded.length}${ext}`
const filepath = resolve(mediaDir, filename)
writeFileSync(filepath, buffer)
totalBytes += buffer.length
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
downloaded.push({ path: filepath, mimeType })
console.warn(
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
)
} catch (err) {
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
}
}
console.warn(
`Downloaded ${downloaded.length}/${urls.length} media files ` +
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
)
return downloaded
}
function guessExtension(url: string, contentType: string): string {
const urlMatch = url.match(MEDIA_EXTENSIONS)
if (urlMatch) return urlMatch[0].toLowerCase()
const typeMap: Record<string, string> = {
'image/png': '.png',
'image/jpeg': '.jpg',
'image/gif': '.gif',
'image/webp': '.webp',
'video/mp4': '.mp4',
'video/webm': '.webm'
}
return typeMap[contentType.split(';')[0]] || '.bin'
}
function guessMimeType(ext: string): string {
const map: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime'
}
return map[ext] || 'application/octet-stream'
}
// ── Gemini analysis ──
function buildIssueAnalysisPrompt(issue: IssueThread): string {
const allText = [
`# Issue: ${issue.title}`,
'',
'## Description',
issue.body,
'',
issue.comments.length > 0
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
: ''
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
The UI has:
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
- Nodes are boxes with input/output slots connected by wires
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
- Sidebars (Workflows, Node Library, Models)
- A topbar with workflow tabs and Queue button
- The default workflow loads with these nodes (approximate center coordinates):
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
${allText}
## Available test actions
Each step must use one of these actions:
### Menu actions
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
### Element actions (by visible text)
- "click" — clicks an element by visible text (text required)
- "rightClick" — right-clicks an element to open context menu (text required)
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
### Canvas actions (by coordinates — viewport is 1280x720)
- "clickCanvas" — click at coordinates (x, y required)
- "rightClickCanvas" — right-click at coordinates (x, y required)
- "doubleClick" — double-click at coordinates to open node search (x, y)
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
### Utility
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Common ComfyUI interactions
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
- Double-click empty canvas → opens node search dialog
- Ctrl+C / Ctrl+V → copy/paste selected nodes
- Delete key → remove selected node
- Ctrl+G → group selected nodes
- Drag from output slot to input slot → create connection
- Click a node to select it, Shift+click for multi-select
## Output format
Return a JSON object with exactly one key: "reproduce", containing:
{
"summary": "One sentence: what bug this issue reports",
"test_focus": "Specific behavior to reproduce",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "clickCanvas",
"description": "Click on first node to select it",
"expected_before": "What should happen if the bug is present"
}
],
"visual_checks": ["Specific visual evidence of the bug to look for"]
}
## Rules
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
- Do NOT include login steps.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
function buildAnalysisPrompt(thread: PrThread): string {
const allText = [
`# PR: ${thread.title}`,
'',
'## Description',
thread.body,
'',
thread.issueComments.length > 0
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
: '',
thread.reviewComments.length > 0
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
: '',
thread.reviews.length > 0
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
: '',
'',
'## Diff (truncated)',
'```',
thread.diff.slice(0, 8000),
'```'
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
${allText}
## Available test actions
Each step must use one of these actions:
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required)
- "click" — clicks an element by visible text (text required)
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Output format
Return a JSON object with exactly two keys: "before" and "after", each containing:
{
"summary": "One sentence: what this PR changes",
"test_focus": "Specific behaviors to verify in this recording",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "openMenu",
"description": "Open the main menu to check file options",
"expected_before": "Old behavior description (before key only)",
"expected_after": "New behavior description (after key only)"
}
],
"visual_checks": ["Specific visual elements to look for"]
}
## Rules
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
- Use information from PR description, screenshots, and comments to understand intended behavior.
- Include at least one screenshot step in each guide.
- Do NOT include login steps.
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
async function analyzeWithGemini(
thread: PrThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<{ before: QaGuide; after: QaGuide }> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildAnalysisPrompt(thread)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
// Add media as inline data
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
// Strip markdown fences if present
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
// Handle different response shapes from Gemini
let before: QaGuide
let after: QaGuide
if (Array.isArray(parsed) && parsed.length >= 2) {
// Array format: [before, after]
before = parsed[0]
after = parsed[1]
} else if (parsed.before && parsed.after) {
// Object format: { before, after }
before = parsed.before
after = parsed.after
} else {
// Try nested wrapper keys
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
if (inner.before && inner.after) {
before = inner.before
after = inner.after
} else {
console.warn(
'Full response:',
JSON.stringify(parsed, null, 2).slice(0, 2000)
)
throw new Error(
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
)
}
}
return { before, after }
}
async function analyzeIssueWithGemini(
issue: IssueThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<QaGuide> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildIssueAnalysisPrompt(issue)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
const guide: QaGuide =
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
return guide
}
// ── Main ──
async function main() {
const opts = parseArgs()
mkdirSync(opts.outputDir, { recursive: true })
if (opts.type === 'issue') {
await analyzeIssue(opts)
} else {
await analyzePr(opts)
}
}
async function analyzeIssue(opts: Options) {
const issue = fetchIssueThread(opts.prNumber, opts.repo)
const allText = [issue.body, ...issue.comments].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guide = await analyzeIssueWithGemini(
issue,
media,
opts.model,
opts.apiKey
)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
console.warn(`Wrote QA guide:`)
console.warn(` Reproduce: ${beforePath}`)
}
async function analyzePr(opts: Options) {
const thread = fetchPrThread(opts.prNumber, opts.repo)
const allText = [
thread.body,
...thread.issueComments,
...thread.reviewComments,
...thread.reviews
].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
console.warn(`Wrote QA guides:`)
console.warn(` Before: ${beforePath}`)
console.warn(` After: ${afterPath}`)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
if (isExecutedAsScript(import.meta.url)) {
main().catch((err) => {
console.error('PR analysis failed:', err)
process.exit(1)
})
}

View File

@@ -1,413 +0,0 @@
#!/usr/bin/env bash
# Deploy QA report to Cloudflare Pages.
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
# Writes outputs to GITHUB_OUTPUT: badge_status, url
set -euo pipefail
npm install -g wrangler@4.74.0 >/dev/null 2>&1
DEPLOY_DIR=$(mktemp -d)
mkdir -p "$DEPLOY_DIR"
for os in Linux macOS Windows; do
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
for prefix in qa qa-before; do
VID="${DIR}/${prefix}-session.mp4"
if [ -f "$VID" ]; then
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
cp "$VID" "$DEST"
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
fi
done
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
for numbered in "$DIR"/qa-session-[0-9].mp4; do
[ -f "$numbered" ] || continue
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
cp "$numbered" "$DEST"
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
done
# Generate GIF thumbnail from after video (or first pass)
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
if [ -f "$THUMB_SRC" ]; then
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| echo "GIF generation failed for ${os} (non-fatal)"
fi
done
# Build video cards and report sections
CARDS=""
# shellcheck disable=SC2034 # accessed via eval
ICONS_Linux="&#x1F427;" ICONS_macOS="&#x1F34E;" ICONS_Windows="&#x1FA9F;"
CARD_COUNT=0
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
for os in Linux macOS Windows; do
eval "ICON=\$ICONS_${os}"
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
[ "$HAS_AFTER" = "0" ] && continue
# Collect all reports for this platform (single + multi-pass)
REPORT_FILES=""
REPORT_LINK=""
REPORT_HTML=""
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
done
if [ -n "$REPORT_FILES" ]; then
# Concatenate all reports into one combined report file
COMBINED_MD=""
for rpt in $REPORT_FILES; do
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
RPT_MD=$(sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' "$rpt")
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD}&#10;&#10;---&#10;&#10;"
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
done
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
FIRST_BASENAME=$(basename "$FIRST_REPORT")
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
fi
if [ "$HAS_BEFORE" = "1" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
else
PASS_VIDEOS=""
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
[ -f "$pass_vid" ] || continue
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
done
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
fi
CARD_COUNT=$((CARD_COUNT + 1))
done
# Build commit info and target link for the report header
COMMIT_HTML=""
REPO_URL="https://github.com/${REPO}"
if [ -n "${TARGET_NUM:-}" ]; then
if [ "$TARGET_TYPE" = "issue" ]; then
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
else
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
fi
fi
if [ -n "${BEFORE_SHA:-}" ]; then
SHORT_BEFORE="${BEFORE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
fi
if [ -n "${AFTER_SHA:-}" ]; then
SHORT_AFTER="${AFTER_SHA:0:7}"
AFTER_LABEL="PR"
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
fi
if [ -n "${PIPELINE_SHA:-}" ]; then
SHORT_PIPE="${PIPELINE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
fi
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" &middot; ${COMMIT_HTML}"
RUN_LINK=""
if [ -n "${RUN_URL:-}" ]; then
RUN_LINK=" &middot; <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
fi
# Timing info
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
TIMING_HTML=""
if [ -n "${RUN_START_TIME:-}" ]; then
TIMING_HTML=" &middot; <span class=sha title='Pipeline timing'>${RUN_START_TIME} &rarr; ${DEPLOY_TIME}</span>"
fi
# Generate index.html from template
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
# Write dynamic content to temp files for safe substitution
# Cloudflare Pages _headers file — enable range requests for video seeking
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
/*.mp4
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
HEADERSEOF
# Build purpose description from pr-context.txt
PURPOSE_HTML=""
if [ -f pr-context.txt ]; then
# Extract title line and first paragraph of description
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt 2>/dev/null | sed 's/^Title: //' || true)
if [ "$TARGET_TYPE" = "issue" ]; then
PURPOSE_LABEL="Issue #${TARGET_NUM}"
PURPOSE_VERB="reports"
else
PURPOSE_LABEL="PR #${TARGET_NUM}"
PURPOSE_VERB="aims to"
fi
# Get first ~300 chars of description body (after "Description:" line)
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt 2>/dev/null | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt 2>/dev/null | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
# Build requirements from QA guide JSON
REQS_HTML=""
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1 || true)
if [ -f "$QA_GUIDE" ]; then
PREREQS=$(python3 -c "
import json, sys, html
try:
g = json.load(open(sys.argv[1]))
prereqs = g.get('prerequisites', [])
steps = g.get('steps', [])
focus = g.get('test_focus', '')
parts = []
if focus:
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
if prereqs:
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
if steps:
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
if len(steps) > 6:
parts[-1] += ' → ...'
print('<br>'.join(parts))
except: pass
" "$QA_GUIDE" 2>/dev/null)
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
fi
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
fi
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
# Badge HTML with copy button (placeholder URL filled after deploy)
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
python3 -c "
import sys, pathlib
d = pathlib.Path(sys.argv[1])
t = pathlib.Path(sys.argv[2]).read_text()
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
sys.stdout.write(t)
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
ERROREOF
# Copy research log to deploy dir if it exists
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
if [ -f "$rlog" ]; then
cp "$rlog" "$DEPLOY_DIR/research-log.json"
echo "Found research log: $rlog"
break
fi
done
# Copy generated test code to deploy dir
for tfile in qa-artifacts/*/research/reproduce.spec.ts qa-artifacts/*/*/research/reproduce.spec.ts qa-artifacts/before/*/research/reproduce.spec.ts; do
if [ -f "$tfile" ]; then
cp "$tfile" "$DEPLOY_DIR/reproduce.spec.ts"
echo "Found test code: $tfile"
break
fi
done
# Copy video script if available
for vsfile in qa-artifacts/*/video-script.spec.ts qa-artifacts/*/*/video-script.spec.ts qa-artifacts/before/*/video-script.spec.ts; do
if [ -f "$vsfile" ]; then
cp "$vsfile" "$DEPLOY_DIR/video-script.spec.ts"
echo "Found video script: $vsfile"
break
fi
done
# Generate badge SVGs into deploy dir
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
# Try research log first (ground truth from a11y assertions)
RESEARCH_VERDICT=""
REPRO_METHOD=""
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
REPRO_METHOD=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('reproducedBy','none'))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none} (by: ${REPRO_METHOD:-none})"
if [ -n "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=1
case "$RESEARCH_VERDICT" in
REPRODUCED) REPRO_COUNT=1 ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
INCONCLUSIVE) INCONC_COUNT=1 ;;
esac
fi
fi
# Check video review verdicts (always, not just as fallback)
VIDEO_REPRODUCED=false
if [ -d video-reviews ]; then
for rpt in video-reviews/*-qa-video-report.md; do
[ -f "$rpt" ] || continue
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
if [ -n "$VERDICT_JSON" ]; then
echo "Video review verdict: $VERDICT_JSON ($(basename "$rpt"))"
[ "$VERDICT_JSON" = "REPRODUCED" ] && VIDEO_REPRODUCED=true
# Only count video as separate report if no research log
if [ -z "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
case "$VERDICT_JSON" in
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
esac
fi
elif [ -z "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
# Fallback: grep Summary section (for older reports without ## Verdict)
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
INCONC_COUNT=$((INCONC_COUNT + 1))
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
REPRO_COUNT=$((REPRO_COUNT + 1))
VIDEO_REPRODUCED=true
fi
fi
done
fi
# Upgrade reproduction method to "both" when E2E and video agree
if [ "$REPRO_METHOD" = "e2e_test" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
REPRO_METHOD="both"
echo "Upgraded reproducedBy to 'both' (E2E + video review agree)"
elif [ -z "$RESEARCH_VERDICT" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
REPRO_METHOD="video"
fi
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
# Warn on verdict mismatch between E2E and video review
if [ -n "$RESEARCH_VERDICT" ]; then
VIDEO_VERDICT=$(grep -oP '"verdict":\s*"[A-Z_]+' video-reviews/*-qa-video-report.md 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
if [ -n "$VIDEO_VERDICT" ] && [ "$RESEARCH_VERDICT" != "$VIDEO_VERDICT" ]; then
echo "⚠ Verdict mismatch: E2E=$RESEARCH_VERDICT vs Video=$VIDEO_VERDICT (E2E takes priority)"
fi
fi
echo "Verdict: ${REPRO_COUNT}${NOT_REPRO_COUNT}${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
# Badge text:
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
if [ "$TOTAL_REPORTS" -le 1 ]; then
# Single report — simple label
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
elif [ "$FAIL_COUNT" -gt 0 ]; then
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
fi
else
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
PARTS=""
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}"
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}"
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}"
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
# Color based on best outcome
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#9f9f9f"
fi
fi
# Badge label: #NUM QA0327 (with today's date)
QA_DATE=$(date -u '+%m%d')
BADGE_LABEL="QA${QA_DATE}"
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
# For PRs, also extract fix quality from Overall Risk section
FIX_RESULT="" FIX_COLOR="#4c1"
if [ "$TARGET_TYPE" != "issue" ]; then
# Try structured JSON risk first
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
if [ -n "$ALL_RISKS" ]; then
# Use worst risk across all reports
if echo "$ALL_RISKS" | grep -q 'high'; then
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
elif echo "$ALL_RISKS" | grep -q 'medium'; then
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
elif echo "$ALL_RISKS" | grep -q 'low'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
else
# Fallback: grep Overall Risk section
RISK_TEXT=""
if [ -d video-reviews ]; then
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20 || true)
fi
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]' || true)
if [ -n "$RISK_FIRST" ]; then
case "$RISK_FIRST" in
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
esac
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
fi
fi
# Always use vertical box badge
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
"$FIX_RESULT" "$FIX_COLOR" "$REPRO_METHOD"
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
# Remove files exceeding Cloudflare Pages 25MB limit to prevent silent deploy failures
MAX_SIZE=$((25 * 1024 * 1024))
find "$DEPLOY_DIR" -type f -size +${MAX_SIZE}c | while read -r big_file; do
SIZE_MB=$(( $(stat -c%s "$big_file") / 1024 / 1024 ))
echo "Removing oversized file: $(basename "$big_file") (${SIZE_MB}MB > 25MB limit)"
rm "$big_file"
done
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
DEPLOY_OUTPUT=$(wrangler pages deploy "$DEPLOY_DIR" \
--project-name="comfy-qa" \
--branch="$BRANCH" 2>&1) || true
echo "$DEPLOY_OUTPUT" | tail -5
URL=$(echo "$DEPLOY_OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1 || true)
FALLBACK_URL="https://${BRANCH}.comfy-qa.pages.dev"
echo "url=${URL:-$FALLBACK_URL}" >> "$GITHUB_OUTPUT"
echo "Deployed to: ${URL:-$FALLBACK_URL}"

View File

@@ -1,208 +0,0 @@
#!/usr/bin/env tsx
/**
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
*
* Usage:
* pnpm exec tsx scripts/qa-generate-test.ts \
* --qa-report <path> QA video review report (markdown)
* --pr-diff <path> PR diff file
* --output <path> Output .spec.ts file path
* --model <name> Gemini model (default: gemini-3-flash-preview)
*/
import { readFile, writeFile } from 'node:fs/promises'
import { basename, resolve } from 'node:path'
import { GoogleGenerativeAI } from '@google/generative-ai'
interface CliOptions {
qaReport: string
prDiff: string
output: string
model: string
}
const DEFAULTS: CliOptions = {
qaReport: '',
prDiff: '',
output: '',
model: 'gemini-3-flash-preview'
}
// ── Fixture API reference for the prompt ────────────────────────────
const FIXTURE_API = `
## ComfyUI Playwright Test Fixture API
Import pattern:
\`\`\`typescript
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
\`\`\`
### Available helpers on \`comfyPage\`:
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
- \`.getActiveTabName(): Promise<string>\` — get active tab name
- \`.saveWorkflow(name)\` — Save via File > Save dialog
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
- \`.exportWorkflow(name)\` — Export via File > Export dialog
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
- \`.getWorkflowTab(name)\` — get a tab locator by name
- \`.closeWorkflowTab(name)\` — close a tab
- \`.openTopbarMenu()\` — open the hamburger menu
- \`.openSubmenu(label)\` — hover to open a submenu
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
- \`.open()\` / \`.close()\` — toggle sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
- \`.getPersistedItem(name)\` — get a workflow item locator
- \`comfyPage.workflow\` — WorkflowHelper:
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\` — delete a workflow
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
- \`comfyPage.settings.setSetting(key, value)\` — change settings
- \`comfyPage.keyboard\` — KeyboardHelper:
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
- \`comfyPage.nodeOps\` — NodeOperationsHelper
- \`comfyPage.canvas\` — CanvasHelper
- \`comfyPage.contextMenu\` — ContextMenu
- \`comfyPage.toast\` — ToastHelper
- \`comfyPage.confirmDialog\` — confirmation dialog
- \`comfyPage.nextFrame()\` — wait for Vue re-render
### Test patterns:
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
`
// ── Prompt builder ──────────────────────────────────────────────────
function buildPrompt(qaReport: string, prDiff: string): string {
return `You are a Playwright test generator for the ComfyUI frontend.
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
described in the QA report below. The test must:
1. Use the ComfyUI Playwright fixture API (documented below)
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
3. NOT test code implementation details
4. Be concise — only test the behavior that the PR changed
5. Follow existing test conventions (see API reference)
${FIXTURE_API}
## QA Video Review Report
${qaReport}
## PR Diff (for context on what changed)
${prDiff.slice(0, 8000)}
## Output Requirements
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
- Start with imports, end with closing brace
- Use descriptive test names that explain the expected behavior
- Add screenshot assertions where visual verification matters
- Keep it focused: 2-5 test cases covering the core behavioral change
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
`
}
// ── Gemini call ─────────────────────────────────────────────────────
async function generateTest(
qaReport: string,
prDiff: string,
model: string
): Promise<string> {
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
const genAI = new GoogleGenerativeAI(apiKey)
const genModel = genAI.getGenerativeModel({ model })
const prompt = buildPrompt(qaReport, prDiff)
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
const result = await genModel.generateContent({
contents: [{ role: 'user', parts: [{ text: prompt }] }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192
}
})
const text = result.response.text()
// Strip markdown fences if model wraps output
return text
.replace(/^```(?:typescript|ts)?\n?/, '')
.replace(/\n?```$/, '')
.trim()
}
// ── CLI ─────────────────────────────────────────────────────────────
function parseArgs(): CliOptions {
const args = process.argv.slice(2)
const opts = { ...DEFAULTS }
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--qa-report':
opts.qaReport = args[++i]
break
case '--pr-diff':
opts.prDiff = args[++i]
break
case '--output':
opts.output = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--help':
console.warn(`Usage:
pnpm exec tsx scripts/qa-generate-test.ts [options]
Options:
--qa-report <path> QA video review report (markdown) [required]
--pr-diff <path> PR diff file [required]
--output <path> Output .spec.ts path [required]
--model <name> Gemini model (default: gemini-3-flash-preview)`)
process.exit(0)
}
}
if (!opts.qaReport || !opts.prDiff || !opts.output) {
console.error('Missing required args. Run with --help for usage.')
process.exit(1)
}
return opts
}
async function main() {
const opts = parseArgs()
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
console.warn(
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
)
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
const testCode = await generateTest(qaReport, prDiff, opts.model)
const outputPath = resolve(opts.output)
await writeFile(outputPath, testCode + '\n')
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})

File diff suppressed because it is too large Load Diff

View File

@@ -1,164 +0,0 @@
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<style>
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
*{margin:0;padding:0;box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
.container{max-width:1200px;margin:0 auto}
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
.header-icon svg{color:var(--primary)}
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
.icon{font-size:1.125rem}
.links{display:flex;gap:.5rem}
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
.comp-panel{border-right:1px solid var(--border-faint)}
.comp-panel:last-child{border-right:none}
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
.report summary:hover{color:var(--fg)}
.report summary svg{flex-shrink:0;opacity:.5}
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
.report-body p{margin:.4rem 0}
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
.report-body li{margin:.25rem 0}
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
.report-body strong{color:var(--fg)}
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
.badge-img{height:20px;display:block}
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
.purpose strong{color:var(--fg);font-weight:600}
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
</style></head><body><div class=container>
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend &middot; Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
<div id=research-section style="margin-top:2rem"></div>
</div><script>
// Load research-log.json and reproduce.spec.ts if available
(async()=>{
const sec=document.getElementById('research-section');
try{
const [logRes,testRes,vsRes]=await Promise.allSettled([fetch('research-log.json'),fetch('reproduce.spec.ts'),fetch('video-script.spec.ts')]);
let html='';
if(logRes.status==='fulfilled'&&logRes.value.ok){
const log=await logRes.value.json();
// Show verdict banner for non-reproduced results
if(log.verdict&&log.verdict!=='REPRODUCED'){
const colors={NOT_REPRODUCIBLE:{bg:'oklch(25% 0.08 25)',border:'oklch(40% 0.15 25)',icon:'✗'},INCONCLUSIVE:{bg:'oklch(25% 0.06 80)',border:'oklch(40% 0.12 80)',icon:'⚠'}};
const c=colors[log.verdict]||colors.INCONCLUSIVE;
html+=`<div style="margin-bottom:1.5rem;padding:1.25rem;background:${c.bg};border:1px solid ${c.border};border-radius:var(--r-lg)"><div style="font-size:1.25rem;font-weight:700;margin-bottom:.5rem">${c.icon} ${log.verdict.replace(/_/g,' ')}</div><div style="font-size:.9rem;line-height:1.6;opacity:.9">${(log.summary||'No details available.').replace(/</g,'&lt;')}</div>${log.evidence?`<div style="margin-top:.75rem;padding:.75rem;background:oklch(0% 0 0/.2);border-radius:var(--r);font-family:var(--font-mono);font-size:.8rem;white-space:pre-wrap;max-height:200px;overflow:auto">${log.evidence.replace(/</g,'&lt;')}</div>`:''}</div>`;
}
html+=`<details style="margin-bottom:1.5rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Research Log &mdash; ${log.verdict||'?'} (${(log.log||[]).length||'?'} tool calls, ${((log.elapsedMs||0)/1000).toFixed(1)}s)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${JSON.stringify(log,null,2)}</pre></div></details>`;
}
if(testRes.status==='fulfilled'&&testRes.value.ok){
const code=await testRes.value.text();
html+=`<details><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">E2E Test Code (reproduce.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${code.replace(/</g,'&lt;').replace(/>/g,'&gt;')}</pre></div></details>`;
}
if(vsRes.status==='fulfilled'&&vsRes.value.ok){
const vsCode=await vsRes.value.text();
html+=`<details style="margin-top:1rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Video Script (video-script.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${vsCode.replace(/</g,'&lt;').replace(/>/g,'&gt;')}</pre></div></details>`;
}
if(html)sec.innerHTML=html;
}catch(e){console.warn('research load failed',e)}
})();
</script><script>
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[![QA Badge]('+b+')]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
document.querySelectorAll('.video-wrap video').forEach(v=>{
v.playbackRate=1;
const c=document.createElement('div');c.className='vctrl';
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
btn('\u23EE',()=>{v.currentTime=0});
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
sep();
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===1)b.classList.add('active');return b});
sep();c.appendChild(time);
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
// Custom seekbar — works even without server range request support
const seekWrap=document.createElement('div');seekWrap.className='vseek';
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
let seeking=false;
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
seekBar.onchange=()=>{seeking=false};
v.closest('.video-wrap').after(seekWrap);
seekWrap.after(c);
v.ontimeupdate=()=>{
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
};
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
v.parentElement.addEventListener('keydown',e=>{
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
});
v.parentElement.setAttribute('tabindex','0');
});
</script></body></html>

View File

@@ -1,253 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Reproduce Phase — Deterministic replay of research plan with narration
*
* Takes a reproduction plan from the research phase and replays it:
* 1. Execute each action deterministically (no AI decisions)
* 2. Capture a11y snapshot before/after each action
* 3. Gemini describes what visually changed (narration for humans)
* 4. Output: narration-log.json with full evidence chain
*/
import type { Page } from '@playwright/test'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { mkdirSync, writeFileSync } from 'fs'
import type { ActionResult } from './qa-record.js'
// ── Types ──
interface ReproductionStep {
action: Record<string, unknown> & { action: string }
expectedAssertion: string
}
interface NarrationEntry {
step: number
action: string
params: Record<string, unknown>
result: ActionResult
a11yBefore: unknown
a11yAfter: unknown
assertionExpected: string
assertionPassed: boolean
assertionActual: string
geminiNarration: string
timestampMs: number
}
export interface NarrationLog {
entries: NarrationEntry[]
allAssertionsPassed: boolean
}
interface ReproduceOptions {
page: Page
plan: ReproductionStep[]
geminiApiKey: string
outputDir: string
}
// ── A11y helpers ──
interface A11yNode {
role: string
name: string
value?: string
checked?: boolean
disabled?: boolean
expanded?: boolean
children?: A11yNode[]
}
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
if (!node) return null
const sel = selector.toLowerCase()
if (
node.name?.toLowerCase().includes(sel) ||
node.role?.toLowerCase().includes(sel)
) {
return node
}
if (node.children) {
for (const child of node.children) {
const found = searchA11y(child, selector)
if (found) return found
}
}
return null
}
function summarizeA11y(node: A11yNode | null): string {
if (!node) return 'null'
const parts = [`role=${node.role}`, `name="${node.name}"`]
if (node.value !== undefined) parts.push(`value="${node.value}"`)
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
if (node.disabled) parts.push('disabled')
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
return `{${parts.join(', ')}}`
}
// ── Subtitle overlay ──
async function showSubtitle(page: Page, text: string, step: number) {
const encoded = encodeURIComponent(
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
)
await page.addScriptTag({
content: `(function(){
var id='qa-subtitle';
var el=document.getElementById(id);
if(!el){
el=document.createElement('div');
el.id=id;
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
document.body.appendChild(el);
}
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
})()`
})
}
// ── Gemini visual narration ──
async function geminiDescribe(
page: Page,
geminiApiKey: string,
focus: string
): Promise<string> {
try {
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
const genAI = new GoogleGenerativeAI(geminiApiKey)
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
const result = await model.generateContent([
{
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
},
{
inlineData: {
mimeType: 'image/jpeg',
data: screenshot.toString('base64')
}
}
])
return result.response.text().trim()
} catch (e) {
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
}
}
// ── Main reproduce function ──
export async function runReproducePhase(
opts: ReproduceOptions
): Promise<NarrationLog> {
const { page, plan, geminiApiKey, outputDir } = opts
const { executeAction } = await import('./qa-record.js')
const narrationDir = `${outputDir}/narration`
mkdirSync(narrationDir, { recursive: true })
const entries: NarrationEntry[] = []
const startMs = Date.now()
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
for (let i = 0; i < plan.length; i++) {
const step = plan[i]
const actionObj = step.action
const elapsed = Date.now() - startMs
// Show subtitle
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
// Capture a11y BEFORE
const a11yBefore = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Execute action
const result = await executeAction(
page,
actionObj as Parameters<typeof executeAction>[1],
outputDir
)
await new Promise((r) => setTimeout(r, 500))
// Capture a11y AFTER
const a11yAfter = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Check assertion
let assertionPassed = false
let assertionActual = ''
if (step.expectedAssertion) {
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
const selectorName = parts[0]
const expectedState = parts.slice(1).join(':').trim()
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
if (expectedState === 'visible' || expectedState === 'exists') {
assertionPassed = found !== null
} else if (expectedState === 'hidden' || expectedState === 'gone') {
assertionPassed = found === null
} else {
// Generic: check if the actual state contains the expected text
assertionPassed = assertionActual
.toLowerCase()
.includes(expectedState.toLowerCase())
}
console.warn(
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
)
}
// Gemini narration (visual description for humans)
const geminiNarration = await geminiDescribe(
page,
geminiApiKey,
`What changed after ${actionObj.action}?`
)
entries.push({
step: i + 1,
action: actionObj.action,
params: actionObj,
result,
a11yBefore,
a11yAfter,
assertionExpected: step.expectedAssertion,
assertionPassed,
assertionActual,
geminiNarration,
timestampMs: elapsed
})
}
// Final screenshot
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
const log: NarrationLog = {
entries,
allAssertionsPassed: entries.every((e) => e.assertionPassed)
}
writeFileSync(
`${narrationDir}/narration-log.json`,
JSON.stringify(log, null, 2)
)
console.warn(
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
)
return log
}

View File

@@ -1,150 +0,0 @@
import { describe, expect, it } from 'vitest'
import {
extractPlatformFromArtifactDirName,
pickLatestVideosByPlatform,
selectVideoCandidateByFile
} from './qa-video-review'
describe('extractPlatformFromArtifactDirName', () => {
it('extracts and normalizes known qa artifact directory names', () => {
expect(
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
).toBe('windows')
expect(
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
).toBe('macos')
expect(
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
).toBe('linux')
})
it('falls back to slugifying unknown directory names', () => {
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
'custom-platform-run'
)
})
})
describe('pickLatestVideosByPlatform', () => {
it('keeps only the latest candidate per platform', () => {
const selected = pickLatestVideosByPlatform([
{
platformName: 'windows',
videoPath: '/tmp/windows-old.mp4',
mtimeMs: 100
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
},
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
}
])
expect(selected).toEqual([
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
}
])
})
})
describe('selectVideoCandidateByFile', () => {
it('selects a single candidate by artifacts-relative path', () => {
const selected = selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-Linux-1/qa-session.mp4'
}
)
expect(selected).toEqual({
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
})
})
it('throws when basename matches multiple videos', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-session.mp4'
}
)
).toThrow('matched 2 videos')
})
it('throws when there is no matching video', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-macOS-1/qa-session.mp4'
}
)
).toThrow('No video matched')
})
it('throws when video file is missing', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: ' '
}
)
).toThrow('--video-file is required')
})
})

View File

@@ -1,771 +0,0 @@
#!/usr/bin/env tsx
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
import { basename, dirname, extname, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { globSync } from 'glob'
interface CliOptions {
artifactsDir: string
videoFile: string
beforeVideo: string
outputDir: string
model: string
requestTimeoutMs: number
dryRun: boolean
prContext: string
targetUrl: string
passLabel: string
}
interface VideoCandidate {
platformName: string
videoPath: string
mtimeMs: number
}
const DEFAULT_OPTIONS: CliOptions = {
artifactsDir: './tmp/qa-artifacts',
videoFile: '',
beforeVideo: '',
outputDir: './tmp',
model: 'gemini-3-flash-preview',
requestTimeoutMs: 300_000,
dryRun: false,
prContext: '',
targetUrl: '',
passLabel: ''
}
const USAGE = `Usage:
pnpm exec tsx scripts/qa-video-review.ts [options]
Options:
--artifacts-dir <path> Artifacts root directory
(default: ./tmp/qa-artifacts)
--video-file <name-or-path> Video file to analyze (required)
(supports basename or relative/absolute path)
--before-video <path> Before video (main branch) for comparison
When provided, sends both videos to Gemini
for comparative before/after analysis
--output-dir <path> Output directory for markdown reports
(default: ./tmp)
--model <name> Gemini model
(default: gemini-3-flash-preview)
--request-timeout-ms <n> Request timeout in milliseconds
(default: 300000)
--pr-context <file> File with PR context (title, body, diff)
for PR-aware review
--target-url <url> Issue or PR URL to include in the report
--pass-label <label> Label for multi-pass reports (e.g. pass1)
Output becomes {platform}-{label}-qa-video-report.md
--dry-run Discover videos and output targets only
--help Show this help text
Environment:
GEMINI_API_KEY Required unless --dry-run
`
function parsePositiveInteger(rawValue: string, flagName: string): number {
const parsedValue = Number.parseInt(rawValue, 10)
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
}
return parsedValue
}
function parseCliOptions(args: string[]): CliOptions {
const options: CliOptions = { ...DEFAULT_OPTIONS }
for (let index = 0; index < args.length; index += 1) {
const argument = args[index]
const nextValue = args[index + 1]
const requireValue = (flagName: string): string => {
if (!nextValue || nextValue.startsWith('--')) {
throw new Error(`Missing value for ${flagName}`)
}
index += 1
return nextValue
}
if (argument === '--help') {
process.stdout.write(USAGE)
process.exit(0)
}
if (argument === '--artifacts-dir') {
options.artifactsDir = requireValue(argument)
continue
}
if (argument === '--video-file') {
options.videoFile = requireValue(argument)
continue
}
if (argument === '--output-dir') {
options.outputDir = requireValue(argument)
continue
}
if (argument === '--model') {
options.model = requireValue(argument)
continue
}
if (argument === '--request-timeout-ms') {
options.requestTimeoutMs = parsePositiveInteger(
requireValue(argument),
argument
)
continue
}
if (argument === '--before-video') {
options.beforeVideo = requireValue(argument)
continue
}
if (argument === '--pr-context') {
options.prContext = requireValue(argument)
continue
}
if (argument === '--target-url') {
options.targetUrl = requireValue(argument)
continue
}
if (argument === '--pass-label') {
options.passLabel = requireValue(argument)
continue
}
if (argument === '--dry-run') {
options.dryRun = true
continue
}
throw new Error(`Unknown argument: ${argument}`)
}
return options
}
function normalizePlatformName(value: string): string {
const slug = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
return slug.length > 0 ? slug : 'unknown-platform'
}
export function extractPlatformFromArtifactDirName(dirName: string): string {
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
return normalizePlatformName(matchedValue ?? dirName)
}
function extractPlatformFromVideoPath(videoPath: string): string {
const artifactDirName = basename(dirname(videoPath))
return extractPlatformFromArtifactDirName(artifactDirName)
}
export function pickLatestVideosByPlatform(
candidates: VideoCandidate[]
): VideoCandidate[] {
const latestByPlatform = new Map<string, VideoCandidate>()
for (const candidate of candidates) {
const current = latestByPlatform.get(candidate.platformName)
if (!current || candidate.mtimeMs > current.mtimeMs) {
latestByPlatform.set(candidate.platformName, candidate)
}
}
return [...latestByPlatform.values()].sort((a, b) =>
a.platformName.localeCompare(b.platformName)
)
}
function toProjectRelativePath(targetPath: string): string {
const relativePath = relative(process.cwd(), targetPath)
if (relativePath.startsWith('.')) {
return relativePath
}
return `./${relativePath}`
}
function errorToString(error: unknown): string {
return error instanceof Error ? error.message : String(error)
}
function normalizePathForMatch(value: string): string {
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
}
export function selectVideoCandidateByFile(
candidates: VideoCandidate[],
options: { artifactsDir: string; videoFile: string }
): VideoCandidate {
const requestedValue = options.videoFile.trim()
if (requestedValue.length === 0) {
throw new Error('--video-file is required')
}
const artifactsRoot = resolve(options.artifactsDir)
const requestedAbsolutePath = resolve(requestedValue)
const requestedPathKey = normalizePathForMatch(requestedValue)
const matches = candidates.filter((candidate) => {
const candidateAbsolutePath = resolve(candidate.videoPath)
if (candidateAbsolutePath === requestedAbsolutePath) {
return true
}
const candidateBaseName = basename(candidate.videoPath)
if (candidateBaseName === requestedValue) {
return true
}
const relativeToCwd = normalizePathForMatch(
relative(process.cwd(), candidateAbsolutePath)
)
if (relativeToCwd === requestedPathKey) {
return true
}
const relativeToArtifacts = normalizePathForMatch(
relative(artifactsRoot, candidateAbsolutePath)
)
return relativeToArtifacts === requestedPathKey
})
if (matches.length === 1) {
return matches[0]
}
if (matches.length === 0) {
const availableVideos = candidates.map((candidate) =>
toProjectRelativePath(candidate.videoPath)
)
throw new Error(
[
`No video matched --video-file "${options.videoFile}".`,
'Available videos:',
...availableVideos.map((videoPath) => `- ${videoPath}`)
].join('\n')
)
}
throw new Error(
[
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
'Please pass a more specific path.',
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
].join('\n')
)
}
async function collectVideoCandidates(
artifactsDir: string
): Promise<VideoCandidate[]> {
const absoluteArtifactsDir = resolve(artifactsDir)
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
cwd: absoluteArtifactsDir,
absolute: true,
nodir: true
}).sort()
const candidates = await Promise.all(
videoPaths.map(async (videoPath) => {
const videoStat = await stat(videoPath)
return {
platformName: extractPlatformFromVideoPath(videoPath),
videoPath,
mtimeMs: videoStat.mtimeMs
}
})
)
return candidates
}
function getMimeType(filePath: string): string {
const ext = extname(filePath).toLowerCase()
const mimeMap: Record<string, string> = {
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime',
'.avi': 'video/x-msvideo',
'.mkv': 'video/x-matroska',
'.m4v': 'video/mp4'
}
return mimeMap[ext] || 'video/mp4'
}
function buildReviewPrompt(options: {
platformName: string
videoPath: string
prContext: string
isComparative: boolean
}): string {
const { platformName, videoPath, prContext, isComparative } = options
if (isComparative) {
return buildComparativePrompt(platformName, videoPath, prContext)
}
return buildSingleVideoPrompt(platformName, videoPath, prContext)
}
function buildComparativePrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
'',
'You are given TWO videos:',
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
'',
'Both videos show the same test steps executed on different code versions.',
''
]
if (prContext) {
lines.push('## PR Context', prContext, '')
}
lines.push(
'## Your Task',
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
'',
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
' Describe what you observe — this establishes the baseline.',
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
'3. **Comparison**: What specifically changed between before and after?',
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
' that were NOT present in the BEFORE video?',
'',
'Note: Brief black frames during page transitions are NORMAL.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible differences. Avoid speculation.',
'',
'Return markdown with these sections exactly:',
'## Summary',
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
'',
'## Behavior Changes',
'Summarize ALL behavioral differences as a markdown TABLE:',
'| Behavior | Before (main) | After (PR) | Verdict |',
'',
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
'- **Before (main)**: how it works/looks in the BEFORE video',
'- **After (PR)**: how it works/looks in the AFTER video',
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
'',
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
'that were tested, so reviewers can confirm nothing was missed.',
'',
'## Timeline Comparison',
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
'| Time | Type | Severity | Before (main) | After (PR) |',
'',
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
'- **Before (main)**: what is observed in the BEFORE video at that time',
'- **After (PR)**: what is observed in the AFTER video at that time',
'',
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
'omit that row. Focus on meaningful differences, not narrating every frame.',
'',
'## Confirmed Issues',
'For each issue, use this exact format:',
'',
'### [Short issue title]',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'',
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
'',
'**Evidence:** [What you observed at the given timestamp in which video]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'(Assess whether the PR achieves its goal based on the before/after comparison)',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
)
return lines.filter(Boolean).join('\n')
}
function buildSingleVideoPrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer reviewing a UI test session recording.',
'',
'## ANTI-HALLUCINATION RULES (READ FIRST)',
'- Describe ONLY what you can directly observe in the video frames',
'- NEVER infer or assume what "must have happened" between frames',
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
'- Your job is to be a CAMERA — report facts, not interpretations',
''
]
const isIssueContext =
prContext &&
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
prContext
)
if (prContext) {
lines.push(
'## Phase 1: Blind Observation (describe what you SEE and HEAR)',
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
'- What elements does the user click/hover/type?',
'- What dialogs/menus open and close?',
'- What keyboard indicators appear? (look for subtitle overlays)',
'- What is the BEFORE state and AFTER state of each action?',
'- **Audio**: Does the video have a TTS narration audio track? If yes, transcribe what the voice says. This narration describes the bug being reproduced.',
'',
'## Phase 2: Compare against expected behavior',
'Now compare your observations against the context below.',
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
''
)
if (isIssueContext) {
lines.push(
'## Issue Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video perform the reproduction steps described in the issue?',
'2. Did your Phase 1 observations show the reported bug behavior?',
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
''
)
} else {
lines.push(
'## PR Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video test the specific behavior the PR changes?',
'2. Did your Phase 1 observations show the expected before/after difference?',
'3. If the test was incomplete or inconclusive, say so honestly.',
''
)
}
}
lines.push(
`Review this QA session video for platform "${platformName}".`,
`Source video: ${toProjectRelativePath(videoPath)}.`,
'The video shows the full test session — analyze it chronologically.',
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible problems and avoid speculation.',
'If confidence is low, mark it explicitly.',
'',
'Return markdown with these sections exactly:',
'## Summary',
isIssueContext
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
: prContext
? '(Explain what the PR intended and whether the video confirms it works)'
: '',
'## Confirmed Issues',
'For each confirmed issue, use this exact format (one block per issue):',
'',
'### [Short issue title]',
'`HIGH` `01:03` `Confidence: High`',
'',
'[Description of the issue — what went wrong and what was expected]',
'',
'**Evidence:** [What you observed in the video at the given timestamp]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'Do NOT use a table for issues — use the block format above.',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'',
'## Narration',
'If the video contains a TTS audio narration track, transcribe it here.',
'If there is no audio or the video is silent, write "No narration detected."',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low", "narrationDetected": true | false}',
'- REPRODUCED: the bug/behavior is clearly visible in the video',
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient',
'- narrationDetected: true if you heard TTS voice narration in the video, false if silent'
)
return lines.filter(Boolean).join('\n')
}
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
async function readVideoFile(videoPath: string): Promise<Buffer> {
const fileStat = await stat(videoPath)
if (fileStat.size > MAX_VIDEO_BYTES) {
throw new Error(
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
)
}
return readFile(videoPath)
}
async function requestGeminiReview(options: {
apiKey: string
model: string
platformName: string
videoPath: string
beforeVideoPath: string
timeoutMs: number
prContext: string
}): Promise<string> {
const genAI = new GoogleGenerativeAI(options.apiKey)
const model = genAI.getGenerativeModel({ model: options.model })
const isComparative = options.beforeVideoPath.length > 0
const prompt = buildReviewPrompt({
platformName: options.platformName,
videoPath: options.videoPath,
prContext: options.prContext,
isComparative
})
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
if (isComparative) {
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
parts.push(
{ text: 'Video 1 — BEFORE (main branch):' },
{
inlineData: {
mimeType: getMimeType(options.beforeVideoPath),
data: beforeBuffer.toString('base64')
}
}
)
}
const afterBuffer = await readVideoFile(options.videoPath)
if (isComparative) {
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
}
parts.push({
inlineData: {
mimeType: getMimeType(options.videoPath),
data: afterBuffer.toString('base64')
}
})
const result = await model.generateContent(parts, {
timeout: options.timeoutMs
})
const response = result.response
const text = response.text()
if (!text || text.trim().length === 0) {
throw new Error('Gemini API returned no output text')
}
return text.trim()
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
function buildReportMarkdown(input: {
platformName: string
model: string
videoPath: string
videoSizeBytes: number
beforeVideoPath?: string
beforeVideoSizeBytes?: number
reviewText: string
targetUrl?: string
}): string {
const headerLines = [
`# ${input.platformName} QA Video Report`,
'',
`- Generated at: ${new Date().toISOString()}`,
`- Model: \`${input.model}\``
]
if (input.targetUrl) {
headerLines.push(`- Target: ${input.targetUrl}`)
}
if (input.beforeVideoPath) {
headerLines.push(
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
'- Mode: **Comparative (before/after)**'
)
} else {
headerLines.push(
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
`- Video size: ${formatBytes(input.videoSizeBytes)}`
)
}
headerLines.push('', '## AI Review', '')
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
}
async function reviewVideo(
video: VideoCandidate,
options: CliOptions,
apiKey: string
): Promise<void> {
let prContext = ''
if (options.prContext) {
try {
prContext = await readFile(options.prContext, 'utf-8')
process.stdout.write(
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
)
} catch {
process.stdout.write(
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
)
}
}
const beforeVideoPath = options.beforeVideo
? resolve(options.beforeVideo)
: ''
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
process.stdout.write(
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
)
}
process.stdout.write(
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
)
const reviewText = await requestGeminiReview({
apiKey,
model: options.model,
platformName: video.platformName,
videoPath: video.videoPath,
beforeVideoPath,
timeoutMs: options.requestTimeoutMs,
prContext
})
const videoStat = await stat(video.videoPath)
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
const outputPath = resolve(
options.outputDir,
`${video.platformName}${passSegment}-qa-video-report.md`
)
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
platformName: video.platformName,
model: options.model,
videoPath: video.videoPath,
videoSizeBytes: videoStat.size,
reviewText,
targetUrl: options.targetUrl || undefined
}
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
reportInput.beforeVideoPath = beforeVideoPath
reportInput.beforeVideoSizeBytes = beforeStat.size
}
const reportMarkdown = buildReportMarkdown(reportInput)
await mkdir(dirname(outputPath), { recursive: true })
await writeFile(outputPath, reportMarkdown, 'utf-8')
process.stdout.write(
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
async function main(): Promise<void> {
const options = parseCliOptions(process.argv.slice(2))
const candidates = await collectVideoCandidates(options.artifactsDir)
if (candidates.length === 0) {
process.stdout.write(
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
)
return
}
const selectedVideo = selectVideoCandidateByFile(candidates, {
artifactsDir: options.artifactsDir,
videoFile: options.videoFile
})
process.stdout.write(
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
)
if (options.dryRun) {
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
return
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
}
await reviewVideo(selectedVideo, options, apiKey)
}
if (isExecutedAsScript(import.meta.url)) {
void main().catch((error: unknown) => {
const message = errorToString(error)
process.stderr.write(`qa-video-review failed: ${message}\n`)
process.exit(1)
})
}

View File

@@ -1,513 +0,0 @@
#!/usr/bin/env tsx
/**
* QA CLI — simplified entry point for local & CI QA runs
*
* Usage:
* pnpm qa 10253 # auto-detects issue vs PR
* pnpm qa https://github.com/.../pull/10270
* pnpm qa 10270 -t base # test PR base (reproduce bug)
* pnpm qa 10270 -t both # test base + head
* pnpm qa --uncommitted # test local uncommitted changes
*
* Automatically loads .env.local / .env for GEMINI_API_KEY, ANTHROPIC_API_KEY.
* Results are written to .comfy-qa/<number>/ by default.
*/
import { parseArgs } from 'node:util'
import { config } from 'dotenv'
import { existsSync, mkdirSync, writeFileSync } from 'fs'
import { dirname, resolve } from 'path'
import { execSync, spawn, spawnSync } from 'child_process'
import { fileURLToPath } from 'url'
// ── Constants ──
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const RECORD_SCRIPT = resolve(SCRIPT_DIR, 'qa-record.ts')
const DEFAULT_REPO = 'Comfy-Org/ComfyUI_frontend'
const VALID_TARGETS = ['head', 'base', 'both'] as const
const CLOUD_FALLBACK_URL = 'https://testcloud.comfy.org/'
type PrTarget = (typeof VALID_TARGETS)[number]
type TargetType = 'issue' | 'pr'
// ── Load .env.local / .env ──
for (const f of ['.env.local', '.env']) {
if (existsSync(f)) {
config({ path: f })
break
}
}
// ── Parse CLI ──
const { values, positionals } = tryParseArgs()
if (values.help) {
printUsage()
process.exit(0)
}
const serverUrl =
values.url || process.env.DEV_SERVER_COMFYUI_URL || 'http://127.0.0.1:8188'
const prTarget = values.target as PrTarget
if (!VALID_TARGETS.includes(prTarget)) {
console.error(
`Invalid --target "${prTarget}". Must be one of: ${VALID_TARGETS.join(', ')}`
)
process.exit(1)
}
// ── Ensure server is reachable (may fall back to cloud) ──
const resolvedServerUrl = await ensureServer(serverUrl)
// ── Dispatch by mode ──
if (values.uncommitted) {
runUncommitted()
} else {
const input = positionals[0]
if (!input) {
printUsage()
process.exit(1)
}
runTarget(input)
}
// ── Mode: uncommitted changes ──
function runUncommitted(): never {
const diff = shell('git diff && git diff --staged')
if (!diff.trim()) {
console.error('No uncommitted changes found')
process.exit(1)
}
const outputDir = resolveOutputDir('.comfy-qa/local')
const diffFile = writeTmpFile(outputDir, 'uncommitted.diff', diff)
logHeader({ label: 'uncommitted changes', outputDir })
const code = runQaRecord('after', diffFile, outputDir)
exit(code, outputDir)
}
// ── Mode: issue or PR by number/URL ──
function runTarget(input: string): never {
const { targetType, number, repo } = resolveTarget(input)
const outputDir = resolveOutputDir(`.comfy-qa/${number}`)
logHeader({
label: `${targetType} #${number} (${repo})`,
outputDir,
extra: targetType === 'pr' ? `Target: ${prTarget}` : undefined
})
const diffFile =
targetType === 'issue'
? fetchIssue(number, repo, outputDir)
: fetchPR(number, repo, outputDir)
let exitCode: number
if (targetType === 'issue') {
exitCode = runQaRecord('reproduce', diffFile, outputDir)
} else if (prTarget === 'both') {
exitCode = runPrBoth(diffFile, outputDir)
} else if (prTarget === 'base') {
exitCode = runQaRecord('before', diffFile, outputDir)
} else {
exitCode = runQaRecord('after', diffFile, outputDir)
}
exit(exitCode, outputDir)
}
// ── PR both phases ──
function runPrBoth(diffFile: string, outputDir: string): number {
console.warn('\n=== Phase 1: Reproduce bug on base ===')
const baseDir = resolve(outputDir, 'base')
mkdirSync(baseDir, { recursive: true })
const baseCode = runQaRecord('before', diffFile, baseDir)
if (baseCode !== 0) {
console.warn('Base phase failed, continuing to head...')
}
console.warn('\n=== Phase 2: Demonstrate fix on head ===')
const headDir = resolve(outputDir, 'head')
mkdirSync(headDir, { recursive: true })
return runQaRecord('after', diffFile, headDir)
}
// ── Target resolution ──
function resolveTarget(input: string): {
targetType: TargetType
number: string
repo: string
} {
const urlMatch = input.match(
/github\.com\/([^/]+\/[^/]+)\/(issues|pull)\/(\d+)/
)
if (urlMatch) {
return {
repo: urlMatch[1],
targetType: urlMatch[2] === 'pull' ? 'pr' : 'issue',
number: urlMatch[3]
}
}
if (/^\d+$/.test(input)) {
return {
repo: DEFAULT_REPO,
targetType: detectType(input, DEFAULT_REPO),
number: input
}
}
console.error(`Cannot parse target: ${input}`)
console.error('Expected a GitHub URL or issue/PR number')
printUsage()
process.exit(1)
}
function detectType(number: string, repo: string): TargetType {
try {
const result = execSync(
`gh api repos/${repo}/issues/${number} --jq 'has("pull_request")'`,
{ encoding: 'utf-8', timeout: 15000, stdio: ['pipe', 'pipe', 'pipe'] }
)
return result.trim() === 'true' ? 'pr' : 'issue'
} catch {
return 'issue'
}
}
// ── Data fetching ──
function fetchIssue(number: string, repo: string, outputDir: string): string {
console.warn(`Fetching issue #${number}...`)
const body = shell(
`gh issue view ${number} --repo ${repo} --json title,body,labels --jq '"Title: " + .title + "\\n\\nLabels: " + ([.labels[].name] | join(", ")) + "\\n\\n" + .body'`
)
return writeTmpFile(outputDir, `issue-${number}.txt`, body)
}
function fetchPR(number: string, repo: string, outputDir: string): string {
console.warn(`Fetching PR #${number}...`)
const prJson = shell(
`gh pr view ${number} --repo ${repo} --json title,body,baseRefName,headRefName,baseRefOid,headRefOid`
)
const pr = JSON.parse(prJson) as {
title: string
body: string
baseRefName: string
headRefName: string
baseRefOid: string
headRefOid: string
}
console.warn(` Base: ${pr.baseRefName} (${pr.baseRefOid.slice(0, 8)})`)
console.warn(` Head: ${pr.headRefName} (${pr.headRefOid.slice(0, 8)})`)
let diff = ''
try {
diff = shell(`gh pr diff ${number} --repo ${repo}`)
} catch {
console.warn('Could not fetch PR diff')
}
writeTmpFile(
outputDir,
'refs.json',
JSON.stringify(
{
base: { ref: pr.baseRefName, sha: pr.baseRefOid },
head: { ref: pr.headRefName, sha: pr.headRefOid }
},
null,
2
)
)
return writeTmpFile(
outputDir,
`pr-${number}.txt`,
`Title: ${pr.title}\n\n${pr.body}\n\n--- DIFF ---\n\n${diff}`
)
}
// ── QA record runner ──
function runQaRecord(
mode: string,
diffFile: string,
outputDir: string
): number {
console.warn(`\nStarting QA ${mode} mode...\n`)
const r = spawnSync(
'pnpm',
[
'exec',
'tsx',
RECORD_SCRIPT,
'--mode',
mode,
'--diff',
diffFile,
'--output-dir',
outputDir,
'--url',
resolvedServerUrl
],
{ stdio: 'inherit', env: process.env }
)
return r.status ?? 1
}
// ── Server management ──
async function ensureServer(url: string): Promise<string> {
if (await isReachable(url)) {
console.warn(`Server OK: ${url}`)
return url
}
console.warn(`Server not reachable at ${url}, attempting auto-start...`)
const port = new URL(url).port || '8188'
// Strategy 1: comfy-cli (pip install comfy-cli)
try {
execSync('which comfy', { stdio: 'pipe' })
console.warn('Starting ComfyUI via comfy-cli...')
const proc = spawn(
'comfy',
['launch', '--background', '--', '--cpu', '--port', port],
{
stdio: 'ignore',
detached: true
}
)
proc.unref()
await waitForServer(url, 120000)
return url
} catch {
// comfy-cli not available
}
// Strategy 2: python main.py from TEST_COMFYUI_DIR or .comfy-qa/ComfyUI
const comfyDir = findComfyUIDir()
if (comfyDir) {
console.warn(`Starting ComfyUI from ${comfyDir}...`)
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
cwd: comfyDir,
stdio: 'ignore',
detached: true
})
proc.unref()
await waitForServer(url, 120000)
return url
}
// Strategy 3: clone ComfyUI and start
const cloneDir = resolve('.comfy-qa/ComfyUI')
if (!existsSync(resolve(cloneDir, 'main.py'))) {
console.warn('No ComfyUI installation found, cloning...')
try {
execSync(
`git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git "${cloneDir}"`,
{ stdio: 'inherit', timeout: 120000 }
)
console.warn('Installing ComfyUI dependencies...')
execSync('pip install -r requirements.txt', {
cwd: cloneDir,
stdio: 'inherit',
timeout: 300000
})
} catch (err) {
console.warn(
`Clone/install failed: ${err instanceof Error ? err.message : err}`
)
}
}
if (existsSync(resolve(cloneDir, 'main.py'))) {
console.warn(`Starting ComfyUI from ${cloneDir}...`)
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
cwd: cloneDir,
stdio: 'ignore',
detached: true
})
proc.unref()
await waitForServer(url, 120000)
return url
}
// Strategy 4: fallback to testcloud
console.warn(`Local server failed. Falling back to ${CLOUD_FALLBACK_URL}`)
if (await isReachable(CLOUD_FALLBACK_URL)) {
console.warn(`Cloud server OK: ${CLOUD_FALLBACK_URL}`)
return CLOUD_FALLBACK_URL
}
console.error(`
No ComfyUI server available. Tried:
1. ${url} (not reachable)
2. comfy-cli (not installed)
3. Local ComfyUI installation (not found)
4. ${CLOUD_FALLBACK_URL} (not reachable)
Install: pip install comfy-cli && comfy install && comfy launch --cpu
`)
process.exit(1)
}
function findComfyUIDir(): string | undefined {
const candidates = [
process.env.TEST_COMFYUI_DIR,
resolve('.comfy-qa/ComfyUI'),
'/home/ComfyUI'
]
return candidates.find((d) => d && existsSync(resolve(d, 'main.py')))
}
async function isReachable(url: string): Promise<boolean> {
try {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), 5000)
const res = await fetch(url, { signal: controller.signal })
clearTimeout(timeout)
return res.ok || res.status === 200 || res.status === 304
} catch {
return false
}
}
async function waitForServer(url: string, timeoutMs: number): Promise<void> {
const start = Date.now()
while (Date.now() - start < timeoutMs) {
if (await isReachable(url)) {
console.warn('Server is ready')
return
}
await new Promise((r) => setTimeout(r, 2000))
}
console.error(`Server did not start within ${timeoutMs / 1000}s`)
process.exit(1)
}
// ── Utilities ──
function shell(cmd: string): string {
return execSync(cmd, { encoding: 'utf-8', timeout: 30000 })
}
function writeTmpFile(
outputDir: string,
filename: string,
content: string
): string {
const tmpDir = resolve(outputDir, '.tmp')
mkdirSync(tmpDir, { recursive: true })
const filePath = resolve(tmpDir, filename)
writeFileSync(filePath, content)
return filePath
}
function resolveOutputDir(defaultPath: string): string {
const dir = values.output ? resolve(values.output) : resolve(defaultPath)
mkdirSync(dir, { recursive: true })
return dir
}
function logHeader(opts: { label: string; outputDir: string; extra?: string }) {
console.warn(`QA target: ${opts.label}`)
console.warn(`Output: ${opts.outputDir}`)
console.warn(`Server: ${resolvedServerUrl}`)
if (values.ref) console.warn(`Ref: ${values.ref}`)
if (opts.extra) console.warn(opts.extra)
}
function exit(code: number, outputDir: string): never {
console.warn('\n=== QA Complete ===')
console.warn(`Results: ${outputDir}`)
try {
console.warn(shell(`ls -la "${outputDir}"`))
} catch {
// not critical
}
process.exit(code)
}
function tryParseArgs() {
try {
const parsed = parseArgs({
args: process.argv.slice(2),
options: {
target: { type: 'string', short: 't', default: 'head' },
uncommitted: { type: 'boolean', default: false },
url: { type: 'string', default: '' },
ref: { type: 'string', default: '' },
output: { type: 'string', short: 'o', default: '' },
help: { type: 'boolean', short: 'h', default: false }
},
allowPositionals: true,
strict: true
})
return {
values: parsed.values as {
target: string
uncommitted: boolean
url: string
ref: string
output: string
help: boolean
},
positionals: parsed.positionals
}
} catch (err) {
console.error(`Error: ${err instanceof Error ? err.message : err}\n`)
printUsage()
process.exit(1)
}
}
function printUsage() {
console.warn(`
QA CLI — Reproduce issues & test PRs for ComfyUI frontend
Usage:
pnpm qa <number|url> [options]
pnpm qa --uncommitted
Targets:
10253 Number (auto-detects issue vs PR via gh CLI)
https://github.com/Comfy-Org/ComfyUI_frontend/issues/10253
https://github.com/Comfy-Org/ComfyUI_frontend/pull/10270
Options:
-t, --target <head|base|both>
For PRs: which ref to test (default: head)
head — test the fix (PR head)
base — reproduce the bug (PR base)
both — base then head
--uncommitted Test local uncommitted changes
--url <url> ComfyUI server URL (default: from .env or http://127.0.0.1:8188)
--ref <ref> Git ref to test against
-o, --output <dir> Override output directory (default: .comfy-qa/<number>)
-h, --help Show this help
Environment (auto-loaded from .env.local or .env):
GEMINI_API_KEY Required — used for PR analysis, video review, TTS
ANTHROPIC_API_KEY Optional locally — Claude Agent SDK auto-detects Claude Code session
Examples:
pnpm qa 10253 # reproduce an issue
pnpm qa 10270 # test PR head (the fix)
pnpm qa 10270 -t base # reproduce bug on PR base
pnpm qa 10270 -t both # test base + head
pnpm qa --uncommitted # test local changes
`)
}

View File

@@ -1,361 +0,0 @@
---
name: ticket-intake
description: 'Parse ticket URL (Notion or GitHub), extract all data, initialize pipeline run. Use when starting work on a new ticket or when asked to pick up a ticket.'
---
# Ticket Intake
Parses a ticket URL from supported sources (Notion or GitHub), extracts all relevant information, and creates a ticket in the pipeline API.
> **🚨 CRITICAL REQUIREMENT**: This skill MUST register the ticket in the Pipeline API and update the source (Notion/GitHub). If these steps are skipped, the entire pipeline breaks. See [Mandatory API Calls](#mandatory-api-calls-execute-all-three) below.
## Supported Sources
| Source | URL Pattern | Provider File |
| ------ | --------------------------------------------------- | --------------------- |
| Notion | `https://notion.so/...` `https://www.notion.so/...` | `providers/notion.md` |
| GitHub | `https://github.com/{owner}/{repo}/issues/{n}` | `providers/github.md` |
## Quick Start
When given a ticket URL:
1. **Detect source type** from URL pattern
2. **Load provider-specific logic** from `providers/` directory
3. Fetch ticket content via appropriate API
4. Extract and normalize properties to common schema
5. **Register ticket in pipeline API** ← MANDATORY
6. **Update source** (Notion status / GitHub comment) ← MANDATORY
7. **Run verification script** to confirm API registration
8. Output summary and handoff to `research-orchestrator`
## Configuration
Uses the **production API** by default. No configuration needed for read operations.
**Defaults (no setup required):**
- API URL: `https://api-gateway-856475788601.us-central1.run.app`
- Read-only endpoints at `/public/*` require no authentication
**For write operations** (transitions, creating tickets), set:
```bash
export PIPELINE_API_KEY="..." # Get from GCP Secret Manager or ask admin
```
**Optional (for local working artifacts):**
```bash
PIPELINE_DIR="${PIPELINE_DIR:-$HOME/repos/ticket-to-pr-pipeline}"
```
## Mandatory API Calls (Execute ALL Three)
**⚠️ These three API calls are the ENTIRE POINT of this skill. Without them, the ticket is invisible to the pipeline, downstream skills will fail, and Notion status won't update.**
**You MUST make these HTTP requests.** Use `curl` from bash — do not just read this as documentation.
### Call 1: Create Ticket
```bash
API_URL="${PIPELINE_API_URL:-https://api-gateway-856475788601.us-central1.run.app}"
API_KEY="${PIPELINE_API_KEY}"
curl -s -X POST "${API_URL}/v1/tickets" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"notion_page_id": "NOTION_PAGE_UUID_HERE",
"title": "TICKET_TITLE_HERE",
"source": "notion",
"metadata": {
"description": "DESCRIPTION_HERE",
"priority": "High",
"labels": [],
"acceptanceCriteria": []
}
}'
```
Save the returned `id` — you need it for the next two calls.
### Call 2: Transition to RESEARCH
```bash
TICKET_ID="id-from-step-1"
curl -s -X POST "${API_URL}/v1/tickets/${TICKET_ID}/transition" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"to_state": "RESEARCH",
"reason": "Intake complete, starting research"
}'
```
### Call 3: Queue Source Update
```bash
curl -s -X POST "${API_URL}/v1/sync/queue" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"ticket_id": "TICKET_ID_HERE",
"action": "update_status",
"payload": { "status": "In Progress" },
"priority": "normal"
}'
```
> **Note:** The action MUST be `"update_status"` (not `"UPDATE_NOTION_STATUS"`). Valid actions: `update_status`, `update_pr_url`, `mark_done`.
### TypeScript Equivalent (if using pipeline client)
```typescript
import { PipelineClient } from '@pipeline/client'
const client = new PipelineClient({
apiUrl:
process.env.PIPELINE_API_URL ||
'https://api-gateway-856475788601.us-central1.run.app',
agentId: process.env.AGENT_ID!
})
const ticket = await client.createTicket({
notion_page_id: pageId,
title: ticketTitle,
source: 'notion',
metadata: { description, priority, labels, acceptanceCriteria }
})
await client.transitionState(
ticket.id,
'RESEARCH',
'Intake complete, starting research'
)
await client.queueSync(ticket.id, 'update_status', { status: 'In Progress' })
```
## Workflow
### Step 1: Detect Source Type
Parse the URL to determine source:
```javascript
if (url.includes('notion.so')) {
source = 'notion'
// Load providers/notion.md
} else if (url.match(/github\.com\/[^\/]+\/[^\/]+\/issues\/\d+/)) {
source = 'github'
// Load providers/github.md
} else {
// Error: Unsupported URL format
}
```
### Step 2: Load Provider and Fetch Data
Read the appropriate provider file for source-specific instructions:
- **Notion**: `providers/notion.md` - Uses Notion MCP, handles Slack links
- **GitHub**: `providers/github.md` - Uses `gh` CLI, handles Dosu comments
Follow the provider's instructions for:
- Fetching content
- Extracting properties
- **Updating the source** (Notion status → "In Progress", Assignee → pipeline owner)
### Step 3: Normalize to Common Schema
All providers must extract normalized ticket data following `schema.md`:
```json
{
"id": "abc12345",
"url": "https://...",
"source": "notion | github",
"title": "Ticket title",
"description": "Full description",
"status": "Not Started",
"assignee": "username",
"priority": "High",
"area": "UI",
"labels": ["bug", "frontend"],
"acceptanceCriteria": ["Criterion 1", "Criterion 2"],
"fetchedAt": "2024-01-15T10:30:00Z"
}
```
### Step 4: Register Ticket in Pipeline API (MANDATORY — DO NOT SKIP)
**Execute all three API calls from [Mandatory API Calls](#mandatory-api-calls-execute-all-three) above.**
This is not optional. This is not documentation. You MUST make these HTTP requests right now.
1. `createTicket()` → save the returned ticket ID
2. `transitionState(id, 'RESEARCH')` → confirm state changed
3. `queueSync(id, 'update_status', { status: 'In Progress' })` → confirm queued
**If any call fails**, retry once. If it still fails, report the error prominently — do NOT silently continue.
### Step 5: Run Verification Script
After making the API calls, run the verification script to confirm everything worked:
```bash
bash scripts/verify-intake.sh TICKET_ID_OR_NOTION_PAGE_ID
```
**If the script is not available locally**, verify manually via the public API:
```bash
curl -s "${API_URL}/public/tickets/${TICKET_ID}" | jq '{id, state, title, notion_page_id}'
```
Expected output:
```json
{
"id": "...",
"state": "RESEARCH",
"title": "...",
"notion_page_id": "..."
}
```
**If `state` is not `RESEARCH`, go back to Step 4 and complete the missing calls.**
### Step 6: Output Summary and Handoff
Print a clear summary:
```markdown
## Ticket Intake Complete
**Source:** Notion | GitHub
**Title:** [Ticket title]
**ID:** abc12345
**Status:** In Progress (queued)
**Priority:** High
**Area:** UI
### Description
[Brief description or first 200 chars]
### Acceptance Criteria
- [ ] Criterion 1
- [ ] Criterion 2
### Links
- **Ticket:** [Original URL]
- **Slack:** [Slack thread content fetched via slackdump] (Notion only)
### Pipeline
- **API Ticket ID:** abc12345
- **State:** RESEARCH
- **Verified:** ✅ (via verify-intake.sh or public API)
```
**After printing the summary, immediately handoff** to continue the pipeline. Use the `handoff` tool with all necessary context (ticket ID, source, title, description, slack context if any):
> **Handoff goal:** "Continue pipeline for ticket {ID} ({title}). Ticket is in RESEARCH state. Load skill: `research-orchestrator` to begin research phase. Ticket data: source={source}, notion_page_id={pageId}, priority={priority}. {slack context summary if available}"
**Do NOT wait for human approval to proceed.** The intake phase is complete — handoff immediately.
## Error Handling
### Unsupported URL
```
❌ Unsupported ticket URL format.
Supported formats:
- Notion: https://notion.so/... or https://www.notion.so/...
- GitHub: https://github.com/{owner}/{repo}/issues/{number}
Received: [provided URL]
```
### Provider-Specific Errors
See individual provider files for source-specific error handling:
- `providers/notion.md` - Authentication, page not found
- `providers/github.md` - Auth, rate limits, issue not found
### Missing Properties
Continue with available data and note what's missing:
```
⚠️ Some properties unavailable:
- Priority: not found (using default: Medium)
- Area: not found
Proceeding with available data...
```
### API Call Failures
```
❌ Pipeline API call failed: {method} {endpoint}
Status: {status}
Error: {message}
Retrying once...
❌ Retry also failed. INTAKE IS INCOMPLETE.
The ticket was NOT registered in the pipeline.
Downstream skills will not work until this is fixed.
```
## Notes
- This skill focuses ONLY on intake — it does not do research
- Slack thread content is fetched automatically via the `slackdump` skill — no manual copy-paste needed
- ALL API calls (createTicket, transitionState, queueSync) are MANDATORY — never skip them
- The `queueSync` action must be `"update_status"`, NOT `"UPDATE_NOTION_STATUS"`
- Pipeline state is tracked via the API, not local files
- Working artifacts (research-report.md, plan.md) can be saved locally to `$PIPELINE_DIR/runs/{ticket-id}/`
- The `source` field in the ticket determines which research strategies to use
## API Client Reference
### Available Methods
| Method | Description |
| ----------------------------------------------------------- | ------------------------------------------------------------------- |
| `createTicket({ notion_page_id, title, source, metadata })` | Create a new ticket in the API |
| `getTicket(id)` | Retrieve a ticket by ID |
| `findByNotionId(notionPageId)` | Look up a ticket by its Notion page ID |
| `listTickets({ state, agent_id, limit, offset })` | List tickets with optional filters |
| `transitionState(id, state, reason)` | Move ticket to a new state (e.g., `'RESEARCH'`) |
| `setPRCreated(id, prUrl)` | Mark ticket as having a PR created |
| `queueSync(id, action, payload)` | Queue a sync action (`update_status`, `update_pr_url`, `mark_done`) |
| `registerBranch(id, branch, repo)` | Register working branch for automatic PR detection |
### Error Handling
```typescript
import { PipelineClient, PipelineAPIError } from '@pipeline/client';
try {
await client.createTicket({ ... });
} catch (error) {
if (error instanceof PipelineAPIError) {
console.error(`API Error ${error.status}: ${error.message}`);
}
throw error;
}
```

View File

@@ -1,194 +0,0 @@
# GitHub Provider - Ticket Intake
Provider-specific logic for ingesting tickets from GitHub Issues.
## URL Pattern
```
https://github.com/{owner}/{repo}/issues/{number}
https://www.github.com/{owner}/{repo}/issues/{number}
```
Extract: `owner`, `repo`, `issue_number` from URL.
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
- Access to the repository
## Fetch Issue Content
Use `gh` CLI to fetch issue details:
```bash
# Get issue details in JSON
gh issue view {number} --repo {owner}/{repo} --json title,body,state,labels,assignees,milestone,author,createdAt,comments,linkedPRs
# Get comments separately if needed
gh issue view {number} --repo {owner}/{repo} --comments
```
## Extract Ticket Data
Map GitHub issue fields to normalized ticket data (stored via API):
| GitHub Field | ticket.json Field | Notes |
| ------------ | ----------------- | -------------------------- |
| title | title | Direct mapping |
| body | description | Issue body/description |
| state | status | Map: open → "Not Started" |
| labels | labels | Array of label names |
| assignees | assignee | First assignee login |
| author | author | Issue author login |
| milestone | milestone | Milestone title if present |
| comments | comments | Array of comment objects |
| linkedPRs | linkedPRs | PRs linked to this issue |
### Priority Mapping
Infer priority from labels:
- `priority:critical`, `P0` → "Critical"
- `priority:high`, `P1` → "High"
- `priority:medium`, `P2` → "Medium"
- `priority:low`, `P3` → "Low"
- No priority label → "Medium" (default)
### Area Mapping
Infer area from labels:
- `area:ui`, `frontend`, `component:*` → "UI"
- `area:api`, `backend` → "API"
- `area:docs`, `documentation` → "Docs"
- `bug`, `fix` → "Bug"
- `enhancement`, `feature` → "Feature"
## Update Source
**For GitHub issues, update is optional but recommended.**
Add a comment to indicate work has started:
```bash
gh issue comment {number} --repo {owner}/{repo} --body "🤖 Pipeline started processing this issue."
```
Optionally assign to self:
```bash
gh issue edit {number} --repo {owner}/{repo} --add-assignee @me
```
Log any updates via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
githubWrites: [
...(ticket.metadata?.githubWrites || []),
{
action: 'comment',
issueNumber: 123,
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
## GitHub-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "github",
"githubOwner": "Comfy-Org",
"githubRepo": "ComfyUI_frontend",
"githubIssueNumber": 123,
"githubIssueUrl": "https://github.com/Comfy-Org/ComfyUI_frontend/issues/123",
"labels": ["bug", "area:ui", "priority:high"],
"linkedPRs": [456, 789],
"dosuComment": "..." // Extracted Dosu bot analysis if present
}
```
## Dosu Bot Detection
Many repositories use Dosu bot for automated issue analysis. Check comments for Dosu:
```bash
gh issue view {number} --repo {owner}/{repo} --comments | grep -A 100 "dosu"
```
Look for comments from:
- `dosu[bot]`
- `dosu-bot`
Extract Dosu analysis which typically includes:
- Root cause analysis
- Suggested files to modify
- Related issues/PRs
- Potential solutions
Store in ticket data via API:
```json
{
"dosuComment": {
"found": true,
"analysis": "...",
"suggestedFiles": ["src/file1.ts", "src/file2.ts"],
"relatedIssues": [100, 101]
}
}
```
## Extract Linked Issues/PRs
Parse issue body and comments for references:
- `#123` → Issue or PR reference
- `fixes #123`, `closes #123` → Linked issue
- `https://github.com/.../issues/123` → Full URL reference
Store in ticket data via API for research phase:
```json
{
"referencedIssues": [100, 101, 102],
"referencedPRs": [200, 201]
}
```
## Error Handling
### Authentication Error
```
⚠️ GitHub CLI not authenticated.
Run: gh auth login
```
### Issue Not Found
```
❌ GitHub issue not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this repository
- Run: gh auth status
```
### Rate Limiting
```
⚠️ GitHub API rate limited.
Wait a few minutes and try again.
Check status: gh api rate_limit
```

View File

@@ -1,202 +0,0 @@
# Notion Provider - Ticket Intake
Provider-specific logic for ingesting tickets from Notion.
## URL Pattern
```
https://www.notion.so/workspace/Page-Title-abc123def456...
https://notion.so/Page-Title-abc123def456...
https://www.notion.so/abc123def456...
```
Page ID is the 32-character hex string (with or without hyphens).
## Prerequisites
- Notion MCP connected and authenticated
- If not setup: `claude mcp add --transport http notion https://mcp.notion.com/mcp`
- Authenticate via `/mcp` command if prompted
## Fetch Ticket Content
Use `Notion:notion-fetch` with the page URL or ID:
```
Fetch the full page content including all properties
```
## Extract Ticket Data
Extract these properties (names may vary):
| Property | Expected Name | Type |
| ------------- | ------------------------- | ------------ |
| Title | Name / Title | Title |
| Status | Status | Select |
| Assignee | Assignee / Assigned To | Person |
| Description | - | Page content |
| Slack Link | Slack Link / Slack Thread | URL |
| GitHub PR | GitHub PR / PR Link | URL |
| Priority | Priority | Select |
| Area | Area / Category | Select |
| Related Tasks | Related Tasks | Relation |
**If properties are missing**: Note what's unavailable and continue with available data.
## Update Source (REQUIRED)
**⚠️ DO NOT SKIP THIS STEP. This is a required action, not optional.**
**⚠️ Notion Write Safety rules apply (see `$PIPELINE_DIR/docs/notion-write-safety.md` for full reference):**
- **Whitelist**: Only `Status`, `GitHub PR`, and `Assignee` fields may be written
- **Valid transitions**: Not Started → In Progress, In Progress → In Review, In Review → Done
- **Logging**: Every write attempt MUST be logged with timestamp, field, value, previous value, skill name, and success status
Use `Notion:notion-update-page` to update the ticket:
1. **Status**: Set to "In Progress" (only valid from "Not Started")
2. **Assignee**: Assign to pipeline owner (Notion ID: `175d872b-594c-81d4-ba5a-0002911c5966`)
```json
{
"page_id": "{page_id_from_ticket}",
"command": "update_properties",
"properties": {
"Status": "In Progress",
"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"
}
}
```
**After the update succeeds**, log the write via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
notionWrites: [
...(ticket.metadata?.notionWrites || []),
{
field: 'Status',
value: 'In Progress',
previousValue: 'Not Started',
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
If update fails, log with `success: false` and continue.
## Notion-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "notion",
"notionPageId": "abc123def456...",
"slackLink": "https://slack.com/...",
"relatedTasks": ["page-id-1", "page-id-2"]
}
```
## Slack Thread Handling
If a Slack link exists, use the `slackdump` skill to fetch the thread content programmatically.
### Slack URL Conversion
Notion stores Slack links in `slackMessage://` format:
```
slackMessage://comfy-organization.slack.com/CHANNEL_ID/THREAD_TS/MESSAGE_TS
```
Convert to browser-clickable format:
```
https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS_NO_DOT
```
**Example:**
- Input: `slackMessage://comfy-organization.slack.com/C075ANWQ8KS/1766022478.450909/1764772881.854829`
- Output: `https://comfy-organization.slack.com/archives/C075ANWQ8KS/p1764772881854829`
(Remove the dot from the last timestamp and prefix with `p`)
### Fetching Thread Content
Load the `slackdump` skill and use the **export-thread** workflow:
```bash
# Export thread by URL
slackdump dump "https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS"
# Or by colon notation (channel_id:thread_ts)
slackdump dump CHANNEL_ID:THREAD_TS
```
Save the thread content to `$RUN_DIR/slack-context.md` and include it in the ticket metadata.
> **No manual action required.** The slackdump CLI handles authentication via stored credentials at `~/.cache/slackdump/comfy-organization.bin`.
## Database Reference: Comfy Tasks
The "Comfy Tasks" database has these properties (verify via `notion-search`):
- **Status values**: Not Started, In Progress, In Review, Done
- **Team assignment**: "Frontend Team" for unassigned tickets
- **Filtering note**: Team filtering in Notion may have quirks - handle gracefully
### Pipeline Owner Details
When assigning tickets, use these identifiers:
| Platform | Identifier |
| --------------- | -------------------------------------- |
| Notion User ID | `175d872b-594c-81d4-ba5a-0002911c5966` |
| Notion Name | Christian Byrne |
| Notion Email | cbyrne@comfy.org |
| Slack User ID | U087MJCDHHC |
| GitHub Username | christian-byrne |
**To update Assignee**, use the Notion User ID (not name):
```
properties: {"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"}
```
### Finding Active Tickets
To list your active tickets:
```
Use Notion:notion-search for "Comfy Tasks"
Filter by Assignee = current user OR Team = "Frontend Team"
```
## Error Handling
### Authentication Error
```
⚠️ Notion authentication required.
Run: claude mcp add --transport http notion https://mcp.notion.com/mcp
Then authenticate via /mcp command.
```
### Page Not Found
```
❌ Notion page not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this page
- Try re-authenticating via /mcp
```

View File

@@ -1,81 +0,0 @@
# Ticket Schema
Common schema for normalized ticket data across all sources. This data is stored and retrieved via the Pipeline API, not local files.
## Ticket Data Schema
```json
{
// Required fields (all sources)
"id": "string", // Unique identifier (short form)
"url": "string", // Original URL
"source": "notion | github", // Source type
"title": "string", // Ticket title
"description": "string", // Full description/body
"fetchedAt": "ISO8601", // When ticket was fetched
// Common optional fields
"status": "string", // Current status
"assignee": "string", // Assigned user
"priority": "string", // Priority level
"area": "string", // Category/area
"labels": ["string"], // Tags/labels
"acceptanceCriteria": ["string"] // List of AC items
// Source-specific fields (see providers)
// Notion: notionPageId, slackLink, relatedTasks, notionWrites
// GitHub: githubOwner, githubRepo, githubIssueNumber, linkedPRs, dosuComment, referencedIssues
}
```
## Ticket State Schema (via API)
State is managed via the Pipeline API using `client.transitionState()`:
```json
{
"ticketId": "string",
"state": "intake | research | planning | implementation | pr_created | done | failed",
"stateChangedAt": "ISO8601",
// Timestamps tracked by API
"createdAt": "ISO8601",
"updatedAt": "ISO8601"
}
```
## Priority Normalization
All sources should normalize to these values:
| Normalized | Description |
| ---------- | ------------------------- |
| Critical | Production down, security |
| High | Blocking work, urgent |
| Medium | Normal priority (default) |
| Low | Nice to have, backlog |
## Status Normalization
Pipeline tracks these statuses internally:
| Status | Description |
| -------------- | ---------------------------- |
| research | Gathering context |
| planning | Creating implementation plan |
| implementation | Writing code |
| review | Code review in progress |
| qa | Quality assurance |
| done | PR merged or completed |
## ID Generation
IDs are generated by the API when creating tickets. For reference:
- **Notion**: First 8 characters of page ID
- **GitHub**: `gh-{owner}-{repo}-{issue_number}` (sanitized)
Examples:
- Notion: `abc12345`
- GitHub: `gh-comfy-org-frontend-123`

View File

@@ -46,11 +46,3 @@ ALGOLIA_API_KEY=684d998c36b67a9a9fce8fc2d8860579
# SENTRY_ORG=comfy-org
# SENTRY_PROJECT=cloud-frontend-staging
# SENTRY_PROJECT_PROD= # prod project slug for sourcemap uploads
# ── QA Skill (scripts/qa-record.ts) ──
# Required for automated bug reproduction via `pnpm exec tsx scripts/qa-record.ts`
# GEMINI_API_KEY is required — used for PR analysis, video review, and TTS narration
GEMINI_API_KEY=
# ANTHROPIC_API_KEY is optional locally — Claude Agent SDK auto-detects Claude Code session
# Required in CI (set as GitHub Actions secret)
# ANTHROPIC_API_KEY=

View File

@@ -0,0 +1,31 @@
name: 'Lint and format verify'
description: >
Runs the lint/format/knip verification suite plus a conditional
browser-tests typecheck. Shared by ci-lint-format.yaml (PR) and
ci-lint-format-queue.yaml (merge queue) so both paths run the exact
same checks. The caller is responsible for checkout and frontend setup
before invoking this action.
runs:
using: composite
steps:
- name: Detect browser_tests changes
id: changed-paths
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
browser_tests:
- 'browser_tests/**'
- name: Verify lint and format
shell: bash
run: |
pnpm lint
pnpm stylelint
pnpm format:check
pnpm knip
- name: Typecheck browser tests
if: steps.changed-paths.outputs.browser_tests == 'true'
shell: bash
run: pnpm typecheck:browser

View File

@@ -0,0 +1,29 @@
# Description: Lint and format verification for GitHub merge queue runs.
# Paired with ci-lint-format.yaml — workflow name and job name must match
# so branch protection resolves a single required check in both the
# pull_request and merge_group contexts. This file runs verify-only steps
# with a read-only token; auto-fix and PR comments live in the PR workflow.
name: 'CI: Lint Format'
on:
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
lint-and-format:
runs-on: ubuntu-latest
steps:
- name: Checkout merge group ref
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Verify lint and format
uses: ./.github/actions/lint-format-verify

View File

@@ -1,4 +1,7 @@
# Description: Linting and code formatting validation for pull requests
# Description: Linting and code formatting validation for pull requests.
# Paired with ci-lint-format-queue.yaml - workflow name and job name must
# match so branch protection resolves a single required check in both the
# pull_request and merge_group contexts.
name: 'CI: Lint Format'
on:
@@ -26,14 +29,6 @@ jobs:
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Detect browser_tests changes
id: changed-paths
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
browser_tests:
- 'browser_tests/**'
- name: Run ESLint with auto-fix
run: pnpm lint:fix
@@ -77,16 +72,8 @@ jobs:
echo "See CONTRIBUTING.md for more details."
exit 1
- name: Final validation
run: |
pnpm lint
pnpm stylelint
pnpm format:check
pnpm knip
- name: Typecheck browser tests
if: steps.changed-paths.outputs.browser_tests == 'true'
run: pnpm typecheck:browser
- name: Verify lint and format
uses: ./.github/actions/lint-format-verify
- name: Comment on PR about auto-fix
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name == github.repository

View File

@@ -8,6 +8,7 @@ on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
merge_group:
workflow_dispatch:
concurrency:

View File

@@ -8,6 +8,7 @@ on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}

File diff suppressed because it is too large Load Diff

3
.gitignore vendored
View File

@@ -67,9 +67,6 @@ dist.zip
/temp/
/tmp/
# QA local output
/.comfy-qa/
# Generated JSON Schemas
/schemas/

View File

@@ -84,6 +84,7 @@
"typescript/no-unsafe-declaration-merging": "off",
"typescript/no-unused-vars": "off",
"unicorn/no-empty-file": "off",
"vitest/require-mock-type-parameters": "off",
"unicorn/no-new-array": "off",
"unicorn/no-single-promise-in-promise-methods": "off",
"unicorn/no-useless-fallback-in-spread": "off",
@@ -116,13 +117,60 @@
},
{
"files": ["browser_tests/**/*.ts"],
"jsPlugins": ["eslint-plugin-playwright"],
"rules": {
"typescript/no-explicit-any": "error",
"no-async-promise-executor": "error",
"no-control-regex": "error",
"no-useless-rename": "error",
"no-unused-private-class-members": "error",
"unicorn/no-empty-file": "error"
"unicorn/no-empty-file": "error",
"playwright/consistent-spacing-between-blocks": "error",
"playwright/expect-expect": [
"error",
{
"assertFunctionNames": [
"recordMeasurement",
"logMeasurement",
"builderSaveAs"
],
"assertFunctionPatterns": [
"^expect",
"^assert",
"^verify",
"^searchAndExpect",
"waitForOpen",
"waitForClosed",
"waitForRequest"
]
}
],
"playwright/max-nested-describe": "error",
"playwright/no-duplicate-hooks": "error",
"playwright/no-element-handle": "error",
"playwright/no-eval": "error",
"playwright/no-focused-test": "error",
"playwright/no-force-option": "error",
"playwright/no-networkidle": "error",
"playwright/no-page-pause": "error",
"playwright/no-skipped-test": "error",
"playwright/no-unsafe-references": "error",
"playwright/no-unused-locators": "error",
"playwright/no-useless-await": "error",
"playwright/no-useless-not": "error",
"playwright/no-wait-for-navigation": "error",
"playwright/no-wait-for-selector": "error",
"playwright/no-wait-for-timeout": "error",
"playwright/prefer-hooks-on-top": "error",
"playwright/prefer-locator": "error",
"playwright/prefer-to-have-count": "error",
"playwright/prefer-to-have-length": "error",
"playwright/prefer-web-first-assertions": "error",
"playwright/prefer-native-locators": "error",
"playwright/require-to-pass-timeout": "error",
"playwright/valid-expect": "error",
"playwright/valid-expect-in-promise": "error",
"playwright/valid-title": "error"
}
}
]

View File

@@ -318,6 +318,9 @@ When referencing Comfy-Org repos:
- Find existing `!important` classes that are interfering with the styling and propose corrections of those instead.
- NEVER use arbitrary percentage values like `w-[80%]` when a Tailwind fraction utility exists
- Use `w-4/5` instead of `w-[80%]`, `w-1/2` instead of `w-[50%]`, etc.
- NEVER use font-size classes (`text-xs`, `text-sm`, etc.) to size `icon-[...]` (iconify) icons
- Iconify icons size via `width`/`height: 1.2em`, so font-size produces unpredictable results
- Use `size-*` classes for explicit sizing, or set font-size on the **parent** container and let `1.2em` scale naturally
## Agent-only rules

View File

@@ -17,7 +17,7 @@ const features = computed(() => [
<div class="mx-auto max-w-3xl px-6 text-center">
<!-- Badge -->
<span
class="inline-block rounded-full bg-brand-yellow/10 px-4 py-1.5 text-xs uppercase tracking-widest text-brand-yellow"
class="inline-block rounded-full bg-brand-yellow/10 px-4 py-1.5 text-xs tracking-widest text-brand-yellow uppercase"
>
{{ t('academy.badge', locale) }}
</span>

View File

@@ -40,7 +40,7 @@ const steps = computed(() => [
<!-- Connecting line between steps (desktop only) -->
<div
v-if="index < steps.length - 1"
class="absolute right-0 top-8 hidden w-full translate-x-1/2 border-t border-brand-yellow/20 md:block"
class="absolute top-8 right-0 hidden w-full translate-x-1/2 border-t border-brand-yellow/20 md:block"
/>
<div class="relative">

View File

@@ -31,11 +31,11 @@ const ctaButtons = computed(() => [
<div class="flex w-full items-center justify-center md:w-[55%]">
<div class="relative -ml-12 -rotate-15 md:-ml-24" aria-hidden="true">
<div
class="h-64 w-64 rounded-full border-[40px] border-brand-yellow md:h-[28rem] md:w-[28rem] md:border-[64px] lg:h-[36rem] lg:w-[36rem] lg:border-[80px]"
class="size-64 rounded-full border-40 border-brand-yellow md:h-112 md:w-md md:border-64 lg:h-144 lg:w-xl lg:border-80"
>
<!-- Gap on the right side to form "C" shape -->
<div
class="absolute right-0 top-1/2 h-32 w-24 -translate-y-1/2 translate-x-1/2 bg-black md:h-48 md:w-36 lg:h-64 lg:w-48"
class="absolute top-1/2 right-0 h-32 w-24 translate-x-1/2 -translate-y-1/2 bg-black md:h-48 md:w-36 lg:h-64 lg:w-48"
/>
</div>
</div>
@@ -44,7 +44,7 @@ const ctaButtons = computed(() => [
<!-- Right: Text content -->
<div class="flex w-full flex-col items-start md:w-[45%]">
<h1
class="text-5xl font-bold leading-tight tracking-tight text-white md:text-6xl lg:text-7xl"
class="text-5xl/tight font-bold tracking-tight text-white md:text-6xl lg:text-7xl"
>
{{ t('hero.headline', locale) }}
</h1>

View File

@@ -17,7 +17,7 @@ const { locale = 'en' } = defineProps<{ locale?: Locale }>()
{{ t('manifesto.heading', locale) }}
</h2>
<p class="mx-auto mt-6 max-w-2xl text-lg leading-relaxed text-smoke-700">
<p class="mx-auto mt-6 max-w-2xl text-lg/relaxed text-smoke-700">
{{ t('manifesto.body', locale) }}
</p>

View File

@@ -33,11 +33,11 @@ const features = computed(() => [
<div class="flex flex-col items-center gap-4">
<!-- Play button triangle -->
<div
class="flex h-16 w-16 items-center justify-center rounded-full border-2 border-white/20"
class="flex size-16 items-center justify-center rounded-full border-2 border-white/20"
aria-hidden="true"
>
<div
class="ml-1 h-0 w-0 border-y-8 border-l-[14px] border-y-transparent border-l-white"
class="ml-1 size-0 border-y-8 border-l-14 border-y-transparent border-l-white"
/>
</div>
<p class="text-sm text-smoke-700">
@@ -54,7 +54,7 @@ const features = computed(() => [
class="flex items-center gap-2"
>
<span
class="h-2 w-2 rounded-full bg-brand-yellow"
class="size-2 rounded-full bg-brand-yellow"
aria-hidden="true"
/>
<span class="text-sm text-smoke-700">{{ feature }}</span>

View File

@@ -32,7 +32,7 @@ const metrics = computed(() => [
<div class="mx-auto max-w-7xl px-6">
<!-- Heading -->
<p
class="text-center text-xs font-medium uppercase tracking-widest text-smoke-700"
class="text-center text-xs font-medium tracking-widest text-smoke-700 uppercase"
>
{{ t('social.heading', locale) }}
</p>

View File

@@ -90,7 +90,7 @@ const filteredTestimonials = computed(() => {
:key="testimonial.name"
class="rounded-xl border border-white/10 bg-charcoal-600 p-6"
>
<blockquote class="text-base italic text-white">
<blockquote class="text-base text-white italic">
&ldquo;{{ testimonial.quote }}&rdquo;
</blockquote>

View File

@@ -24,12 +24,12 @@ const activeCategory = ref(0)
<!-- Left placeholder image (desktop only) -->
<div class="hidden flex-1 lg:block">
<div
class="aspect-[2/3] rounded-full border border-white/10 bg-charcoal-600"
class="aspect-2/3 rounded-full border border-white/10 bg-charcoal-600"
/>
</div>
<!-- Center content -->
<div class="flex flex-col items-center text-center lg:flex-[2]">
<div class="flex flex-col items-center text-center lg:flex-2">
<h2 class="text-3xl font-bold text-white">
{{ t('useCase.heading', locale) }}
</h2>
@@ -70,7 +70,7 @@ const activeCategory = ref(0)
<!-- Right placeholder image (desktop only) -->
<div class="hidden flex-1 lg:block">
<div
class="aspect-[2/3] rounded-3xl border border-white/10 bg-charcoal-600"
class="aspect-2/3 rounded-3xl border border-white/10 bg-charcoal-600"
/>
</div>
</div>

View File

@@ -53,7 +53,7 @@ const pillars = computed(() => [
class="rounded-xl border border-white/10 bg-charcoal-600 p-6 transition-colors hover:border-brand-yellow"
>
<div
class="flex h-12 w-12 items-center justify-center rounded-full bg-brand-yellow text-xl"
class="flex size-12 items-center justify-center rounded-full bg-brand-yellow text-xl"
>
{{ pillar.icon }}
</div>

View File

@@ -40,6 +40,39 @@ browser_tests/
- **`fixtures/helpers/`** — Focused helper classes. Domain-specific actions that coordinate multiple page objects (e.g. canvas operations, workflow loading).
- **`fixtures/utils/`** — Pure utility functions. No `Page` dependency; stateless helpers that can be used anywhere.
## Page Object Locator Style
Define UI element locators as `public readonly` properties assigned in the constructor — not as getter methods. Getters that simply return a locator add unnecessary indirection and hide the object shape from IDE auto-complete.
```typescript
// ✅ Correct — public readonly, assigned in constructor
export class MyDialog extends BaseDialog {
public readonly submitButton: Locator
public readonly cancelButton: Locator
constructor(page: Page) {
super(page)
this.submitButton = this.root.getByRole('button', { name: 'Submit' })
this.cancelButton = this.root.getByRole('button', { name: 'Cancel' })
}
}
// ❌ Avoid — getter-based locators
export class MyDialog extends BaseDialog {
get submitButton() {
return this.root.getByRole('button', { name: 'Submit' })
}
}
```
**Keep as getters only when:**
- Lazy initialization is needed (`this._tab ??= new Tab(this.page)`)
- The value is computed from runtime state (e.g. `get id() { return this.userIds[index] }`)
- It's a private convenience accessor (e.g. `private get page() { return this.comfyPage.page }`)
When a class has cached locator properties, prefer reusing them in methods rather than rebuilding locators from scratch.
## Polling Assertions
Prefer `expect.poll()` over `expect(async () => { ... }).toPass()` when the block contains a single async call with a single assertion. `expect.poll()` is more readable and gives better error messages (shows actual vs expected on failure).

View File

@@ -0,0 +1,50 @@
{
"last_node_id": 1,
"last_link_id": 0,
"nodes": [
{
"id": 1,
"type": "ImageCropV2",
"pos": [50, 50],
"size": [400, 500],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": null
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": null
}
],
"properties": {
"Node name for S&R": "ImageCropV2"
},
"widgets_values": [
{
"x": 0,
"y": 0,
"width": 512,
"height": 512
}
]
}
],
"links": [],
"groups": [],
"config": {},
"extra": {
"ds": {
"offset": [0, 0],
"scale": 1
}
},
"version": 0.4
}

View File

@@ -0,0 +1,100 @@
{
"last_node_id": 3,
"last_link_id": 2,
"nodes": [
{
"id": 1,
"type": "LoadImage",
"pos": [50, 50],
"size": [315, 314],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [1]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": ["example.png", "image"]
},
{
"id": 2,
"type": "ImageCropV2",
"pos": [450, 50],
"size": [400, 500],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 1
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [2]
}
],
"properties": {
"Node name for S&R": "ImageCropV2"
},
"widgets_values": [
{
"x": 10,
"y": 10,
"width": 100,
"height": 100
}
]
},
{
"id": 3,
"type": "PreviewImage",
"pos": [900, 50],
"size": [315, 270],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 2
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewImage"
},
"widgets_values": []
}
],
"links": [
[1, 1, 0, 2, 0, "IMAGE"],
[2, 2, 0, 3, 0, "IMAGE"]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"offset": [0, 0],
"scale": 1
}
},
"version": 0.4
}

View File

@@ -26,11 +26,10 @@ export class ComfyMouse implements Omit<Mouse, 'move'> {
static defaultSteps = 5
static defaultOptions: DragOptions = { steps: ComfyMouse.defaultSteps }
constructor(readonly comfyPage: ComfyPage) {}
readonly mouse: Mouse
/** The normal Playwright {@link Mouse} property from {@link ComfyPage.page}. */
get mouse() {
return this.comfyPage.page.mouse
constructor(readonly comfyPage: ComfyPage) {
this.mouse = comfyPage.page.mouse
}
async nextFrame() {

View File

@@ -73,15 +73,13 @@ class ComfyMenu {
public readonly sideToolbar: Locator
public readonly propertiesPanel: ComfyPropertiesPanel
public readonly modeToggleButton: Locator
public readonly buttons: Locator
constructor(public readonly page: Page) {
this.sideToolbar = page.getByTestId(TestIds.sidebar.toolbar)
this.modeToggleButton = page.getByTestId(TestIds.sidebar.modeToggle)
this.propertiesPanel = new ComfyPropertiesPanel(page)
}
get buttons() {
return this.sideToolbar.locator('.side-bar-button')
this.buttons = this.sideToolbar.locator('.side-bar-button')
}
get modelLibraryTab() {
@@ -183,6 +181,7 @@ export class ComfyPage {
public readonly assetApi: AssetHelper
public readonly modelLibrary: ModelLibraryHelper
public readonly cloudAuth: CloudAuthHelper
public readonly visibleToasts: Locator
/** Worker index to test user ID */
public readonly userIds: string[] = []
@@ -225,6 +224,7 @@ export class ComfyPage {
this.workflow = new WorkflowHelper(this)
this.contextMenu = new ContextMenu(page)
this.toast = new ToastHelper(page)
this.visibleToasts = this.toast.visibleToasts
this.dragDrop = new DragDropHelper(page)
this.featureFlags = new FeatureFlagHelper(page)
this.command = new CommandHelper(page)
@@ -237,10 +237,6 @@ export class ComfyPage {
this.cloudAuth = new CloudAuthHelper(page)
}
get visibleToasts() {
return this.toast.visibleToasts
}
async setupUser(username: string) {
const res = await this.request.get(`${this.url}/api/users`)
if (res.status() !== 200)
@@ -325,7 +321,7 @@ export class ComfyPage {
// window.app.extensionManager => GraphView ready
window.app && window.app.extensionManager
)
await this.page.waitForSelector('.p-blockui-mask', { state: 'hidden' })
await this.page.locator('.p-blockui-mask').waitFor({ state: 'hidden' })
await this.nextFrame()
}
@@ -375,7 +371,7 @@ export class ComfyPage {
}
async closeMenu() {
await this.page.click('button.comfy-close-menu-btn')
await this.page.locator('button.comfy-close-menu-btn').click()
await this.nextFrame()
}

View File

@@ -1,30 +1,22 @@
import type { Page } from '@playwright/test'
import type { Locator, Page } from '@playwright/test'
import { test as base } from '@playwright/test'
export class UserSelectPage {
public readonly selectionUrl: string
public readonly container: Locator
public readonly newUserInput: Locator
public readonly existingUserSelect: Locator
public readonly nextButton: Locator
constructor(
public readonly url: string,
public readonly page: Page
) {}
get selectionUrl() {
return this.url + '/user-select'
}
get container() {
return this.page.locator('#comfy-user-selection')
}
get newUserInput() {
return this.container.locator('#new-user-input')
}
get existingUserSelect() {
return this.container.locator('#existing-user-select')
}
get nextButton() {
return this.container.getByText('Next')
) {
this.selectionUrl = url + '/user-select'
this.container = page.locator('#comfy-user-selection')
this.newUserInput = this.container.locator('#new-user-input')
this.existingUserSelect = this.container.locator('#existing-user-select')
this.nextButton = this.container.getByText('Next')
}
}

View File

@@ -7,13 +7,20 @@ import { TestIds } from '@e2e/fixtures/selectors'
import { VueNodeFixture } from '@e2e/fixtures/utils/vueNodeFixtures'
export class VueNodeHelpers {
constructor(private page: Page) {}
/**
* Get locator for all Vue node components in the DOM
*/
get nodes(): Locator {
return this.page.locator('[data-node-id]')
public readonly nodes: Locator
/**
* Get locator for selected Vue node components (using visual selection indicators)
*/
public readonly selectedNodes: Locator
constructor(private page: Page) {
this.nodes = page.locator('[data-node-id]')
this.selectedNodes = page.locator(
'[data-node-id].outline-node-component-outline'
)
}
/**
@@ -23,13 +30,6 @@ export class VueNodeHelpers {
return this.page.locator(`[data-node-id="${nodeId}"]`)
}
/**
* Get locator for selected Vue node components (using visual selection indicators)
*/
get selectedNodes(): Locator {
return this.page.locator('[data-node-id].outline-node-component-outline')
}
/**
* Get locator for Vue nodes by the node's title (displayed name in the header).
* Matches against the actual title element, not the full node body.
@@ -37,7 +37,7 @@ export class VueNodeHelpers {
*/
getNodeByTitle(title: string): Locator {
return this.page.locator('[data-node-id]').filter({
has: this.page.locator('[data-testid="node-title"]', { hasText: title })
has: this.page.getByTestId('node-title').filter({ hasText: title })
})
}
@@ -146,7 +146,7 @@ export class VueNodeHelpers {
expectedCount
)
} else {
await this.page.waitForSelector('[data-node-id]')
await this.page.locator('[data-node-id]').first().waitFor()
}
}

View File

@@ -3,13 +3,11 @@ import type { Locator, Page } from '@playwright/test'
export class ComfyNodeSearchFilterSelectionPanel {
readonly root: Locator
readonly header: Locator
constructor(public readonly page: Page) {
this.root = page.getByRole('dialog')
}
get header() {
return this.root
this.header = this.root
.locator('div')
.filter({ hasText: 'Add node filter condition' })
}
@@ -41,6 +39,8 @@ export class ComfyNodeSearchFilterSelectionPanel {
export class ComfyNodeSearchBox {
public readonly input: Locator
public readonly dropdown: Locator
public readonly filterButton: Locator
public readonly filterChips: Locator
public readonly filterSelectionPanel: ComfyNodeSearchFilterSelectionPanel
constructor(public readonly page: Page) {
@@ -50,13 +50,15 @@ export class ComfyNodeSearchBox {
this.dropdown = page.locator(
'.comfy-vue-node-search-container .p-autocomplete-list'
)
this.filterButton = page.locator(
'.comfy-vue-node-search-container .filter-button'
)
this.filterChips = page.locator(
'.comfy-vue-node-search-container .p-autocomplete-chip-item'
)
this.filterSelectionPanel = new ComfyNodeSearchFilterSelectionPanel(page)
}
get filterButton() {
return this.page.locator('.comfy-vue-node-search-container .filter-button')
}
async fillAndSelectFirstNode(
nodeName: string,
options?: { suggestionIndex?: number; exact?: boolean }
@@ -78,12 +80,6 @@ export class ComfyNodeSearchBox {
await this.filterSelectionPanel.addFilter(filterValue, filterType)
}
get filterChips() {
return this.page.locator(
'.comfy-vue-node-search-container .p-autocomplete-chip-item'
)
}
async removeFilter(index: number) {
await this.filterChips.nth(index).locator('.p-chip-remove-icon').click()
}

View File

@@ -2,18 +2,14 @@ import { expect } from '@playwright/test'
import type { Locator, Page } from '@playwright/test'
export class ContextMenu {
constructor(public readonly page: Page) {}
public readonly primeVueMenu: Locator
public readonly litegraphMenu: Locator
public readonly menuItems: Locator
get primeVueMenu() {
return this.page.locator('.p-contextmenu, .p-menu')
}
get litegraphMenu() {
return this.page.locator('.litemenu')
}
get menuItems() {
return this.page.locator('.p-menuitem, .litemenu-entry')
constructor(public readonly page: Page) {
this.primeVueMenu = page.locator('.p-contextmenu, .p-menu')
this.litegraphMenu = page.locator('.litemenu')
this.menuItems = page.locator('.p-menuitem, .litemenu-entry')
}
async clickMenuItem(name: string): Promise<void> {

View File

@@ -0,0 +1,97 @@
import type { Locator, Page } from '@playwright/test'
import { TestIds } from '../selectors'
const ids = TestIds.outputHistory
export class OutputHistoryComponent {
constructor(private readonly page: Page) {}
get outputs(): Locator {
return this.page.getByTestId(ids.outputs)
}
get welcome(): Locator {
return this.page.getByTestId(ids.welcome)
}
get outputInfo(): Locator {
return this.page.getByTestId(ids.outputInfo)
}
get activeQueue(): Locator {
return this.page.getByTestId(ids.activeQueue)
}
get queueBadge(): Locator {
return this.page.getByTestId(ids.queueBadge)
}
get inProgressItems(): Locator {
return this.page.getByTestId(ids.inProgressItem)
}
get historyItems(): Locator {
return this.page.getByTestId(ids.historyItem)
}
get skeletons(): Locator {
return this.page.getByTestId(ids.skeleton)
}
get latentPreviews(): Locator {
return this.page.getByTestId(ids.latentPreview)
}
get imageOutputs(): Locator {
return this.page.getByTestId(ids.imageOutput)
}
get videoOutputs(): Locator {
return this.page.getByTestId(ids.videoOutput)
}
/** The currently selected (checked) in-progress item. */
get selectedInProgressItem(): Locator {
return this.page.locator(
`[data-testid="${ids.inProgressItem}"][data-state="checked"]`
)
}
/** The currently selected (checked) history item. */
get selectedHistoryItem(): Locator {
return this.page.locator(
`[data-testid="${ids.historyItem}"][data-state="checked"]`
)
}
/** The header-level progress bar. */
get headerProgressBar(): Locator {
return this.page.getByTestId(ids.headerProgressBar)
}
/** The in-progress item's progress bar (inside the thumbnail). */
get itemProgressBar(): Locator {
return this.inProgressItems.first().getByTestId(ids.itemProgressBar)
}
/** Overall progress in the header bar. */
get headerOverallProgress(): Locator {
return this.headerProgressBar.getByTestId(ids.progressOverall)
}
/** Node progress in the header bar. */
get headerNodeProgress(): Locator {
return this.headerProgressBar.getByTestId(ids.progressNode)
}
/** Overall progress in the in-progress item bar. */
get itemOverallProgress(): Locator {
return this.itemProgressBar.getByTestId(ids.progressOverall)
}
/** Node progress in the in-progress item bar. */
get itemNodeProgress(): Locator {
return this.itemProgressBar.getByTestId(ids.progressNode)
}
}

View File

@@ -1,15 +1,22 @@
import type { Page } from '@playwright/test'
import type { Locator, Page } from '@playwright/test'
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import { TestIds } from '@e2e/fixtures/selectors'
import { BaseDialog } from '@e2e/fixtures/components/BaseDialog'
export class SettingDialog extends BaseDialog {
public readonly searchBox: Locator
public readonly categories: Locator
public readonly contentArea: Locator
constructor(
page: Page,
public readonly comfyPage: ComfyPage
) {
super(page, TestIds.dialogs.settings)
this.searchBox = this.root.getByPlaceholder(/Search/)
this.categories = this.root.locator('nav').getByRole('button')
this.contentArea = this.root.getByRole('main')
}
async open() {
@@ -36,27 +43,15 @@ export class SettingDialog extends BaseDialog {
await settingInputDiv.locator('input').click()
}
get searchBox() {
return this.root.getByPlaceholder(/Search/)
}
get categories() {
return this.root.locator('nav').getByRole('button')
}
category(name: string) {
return this.root.locator('nav').getByRole('button', { name })
}
get contentArea() {
return this.root.getByRole('main')
}
async goToAboutPanel() {
const aboutButton = this.root.locator('nav').getByRole('button', {
name: 'About'
})
await aboutButton.click()
await this.page.waitForSelector('.about-container')
await this.page.locator('.about-container').waitFor()
}
}

View File

@@ -5,18 +5,16 @@ import type { WorkspaceStore } from '@e2e/types/globals'
import { TestIds } from '@e2e/fixtures/selectors'
class SidebarTab {
public readonly tabButton: Locator
public readonly selectedTabButton: Locator
constructor(
public readonly page: Page,
public readonly tabId: string
) {}
get tabButton() {
return this.page.locator(`.${this.tabId}-tab-button`)
}
get selectedTabButton() {
return this.page.locator(
`.${this.tabId}-tab-button.side-bar-button-selected`
) {
this.tabButton = page.locator(`.${tabId}-tab-button`)
this.selectedTabButton = page.locator(
`.${tabId}-tab-button.side-bar-button-selected`
)
}
@@ -35,28 +33,19 @@ class SidebarTab {
}
export class NodeLibrarySidebarTab extends SidebarTab {
public readonly nodeLibrarySearchBoxInput: Locator
public readonly nodeLibraryTree: Locator
public readonly nodePreview: Locator
public readonly tabContainer: Locator
public readonly newFolderButton: Locator
constructor(public override readonly page: Page) {
super(page, 'node-library')
}
get nodeLibrarySearchBoxInput() {
return this.page.getByPlaceholder('Search Nodes...')
}
get nodeLibraryTree() {
return this.page.getByTestId(TestIds.sidebar.nodeLibrary)
}
get nodePreview() {
return this.page.locator('.node-lib-node-preview')
}
get tabContainer() {
return this.page.locator('.sidebar-content-container')
}
get newFolderButton() {
return this.tabContainer.locator('.new-folder-button')
this.nodeLibrarySearchBoxInput = page.getByPlaceholder('Search Nodes...')
this.nodeLibraryTree = page.getByTestId(TestIds.sidebar.nodeLibrary)
this.nodePreview = page.locator('.node-lib-node-preview')
this.tabContainer = page.locator('.sidebar-content-container')
this.newFolderButton = this.tabContainer.locator('.new-folder-button')
}
override async open() {
@@ -101,34 +90,25 @@ export class NodeLibrarySidebarTab extends SidebarTab {
}
export class NodeLibrarySidebarTabV2 extends SidebarTab {
public readonly searchInput: Locator
public readonly sidebarContent: Locator
public readonly allTab: Locator
public readonly blueprintsTab: Locator
public readonly sortButton: Locator
constructor(public override readonly page: Page) {
super(page, 'node-library')
}
get searchInput() {
return this.page.getByPlaceholder('Search...')
}
get sidebarContent() {
return this.page.locator('.sidebar-content-container')
this.searchInput = page.getByPlaceholder('Search...')
this.sidebarContent = page.locator('.sidebar-content-container')
this.allTab = this.getTab('All')
this.blueprintsTab = this.getTab('Blueprints')
this.sortButton = this.sidebarContent.getByRole('button', { name: 'Sort' })
}
getTab(name: string) {
return this.sidebarContent.getByRole('tab', { name, exact: true })
}
get allTab() {
return this.getTab('All')
}
get blueprintsTab() {
return this.getTab('Blueprints')
}
get sortButton() {
return this.sidebarContent.getByRole('button', { name: 'Sort' })
}
getFolder(folderName: string) {
return this.sidebarContent
.getByRole('treeitem', { name: folderName })
@@ -154,12 +134,15 @@ export class NodeLibrarySidebarTabV2 extends SidebarTab {
}
export class WorkflowsSidebarTab extends SidebarTab {
public readonly root: Locator
public readonly activeWorkflowLabel: Locator
constructor(public override readonly page: Page) {
super(page, 'workflows')
}
get root() {
return this.page.getByTestId(TestIds.sidebar.workflows)
this.root = page.getByTestId(TestIds.sidebar.workflows)
this.activeWorkflowLabel = this.root.locator(
'.comfyui-workflows-open .p-tree-node-selected .node-label'
)
}
async getOpenedWorkflowNames() {
@@ -168,12 +151,6 @@ export class WorkflowsSidebarTab extends SidebarTab {
.allInnerTexts()
}
get activeWorkflowLabel(): Locator {
return this.root.locator(
'.comfyui-workflows-open .p-tree-node-selected .node-label'
)
}
async getActiveWorkflowName() {
return await this.activeWorkflowLabel.innerText()
}
@@ -228,36 +205,27 @@ export class WorkflowsSidebarTab extends SidebarTab {
}
export class ModelLibrarySidebarTab extends SidebarTab {
public readonly searchInput: Locator
public readonly modelTree: Locator
public readonly refreshButton: Locator
public readonly loadAllFoldersButton: Locator
public readonly folderNodes: Locator
public readonly leafNodes: Locator
public readonly modelPreview: Locator
constructor(public override readonly page: Page) {
super(page, 'model-library')
}
get searchInput() {
return this.page.getByPlaceholder('Search Models...')
}
get modelTree() {
return this.page.locator('.model-lib-tree-explorer')
}
get refreshButton() {
return this.page.getByRole('button', { name: 'Refresh' })
}
get loadAllFoldersButton() {
return this.page.getByRole('button', { name: 'Load All Folders' })
}
get folderNodes() {
return this.modelTree.locator('.p-tree-node:not(.p-tree-node-leaf)')
}
get leafNodes() {
return this.modelTree.locator('.p-tree-node-leaf')
}
get modelPreview() {
return this.page.locator('.model-lib-model-preview')
this.searchInput = page.getByPlaceholder('Search Models...')
this.modelTree = page.locator('.model-lib-tree-explorer')
this.refreshButton = page.getByRole('button', { name: 'Refresh' })
this.loadAllFoldersButton = page.getByRole('button', {
name: 'Load All Folders'
})
this.folderNodes = this.modelTree.locator(
'.p-tree-node:not(.p-tree-node-leaf)'
)
this.leafNodes = this.modelTree.locator('.p-tree-node-leaf')
this.modelPreview = page.locator('.model-lib-model-preview')
}
override async open() {
@@ -281,137 +249,97 @@ export class ModelLibrarySidebarTab extends SidebarTab {
}
export class AssetsSidebarTab extends SidebarTab {
constructor(public override readonly page: Page) {
super(page, 'assets')
}
// --- Tab navigation ---
get generatedTab() {
return this.page.getByRole('tab', { name: 'Generated' })
}
get importedTab() {
return this.page.getByRole('tab', { name: 'Imported' })
}
public readonly generatedTab: Locator
public readonly importedTab: Locator
// --- Empty state ---
public readonly emptyStateMessage: Locator
get emptyStateMessage() {
return this.page.getByText(
// --- Search & filter ---
public readonly searchInput: Locator
public readonly settingsButton: Locator
// --- View mode ---
public readonly listViewOption: Locator
public readonly gridViewOption: Locator
// --- Sort options (cloud-only, shown inside settings popover) ---
public readonly sortNewestFirst: Locator
public readonly sortOldestFirst: Locator
// --- Asset cards ---
public readonly assetCards: Locator
public readonly selectedCards: Locator
// --- List view items ---
public readonly listViewItems: Locator
// --- Selection footer ---
public readonly selectionFooter: Locator
public readonly selectionCountButton: Locator
public readonly deselectAllButton: Locator
public readonly deleteSelectedButton: Locator
public readonly downloadSelectedButton: Locator
// --- Folder view ---
public readonly backToAssetsButton: Locator
// --- Loading ---
public readonly skeletonLoaders: Locator
constructor(public override readonly page: Page) {
super(page, 'assets')
this.generatedTab = page.getByRole('tab', { name: 'Generated' })
this.importedTab = page.getByRole('tab', { name: 'Imported' })
this.emptyStateMessage = page.getByText(
'Upload files or generate content to see them here'
)
this.searchInput = page.getByPlaceholder('Search Assets...')
this.settingsButton = page.getByRole('button', { name: 'View settings' })
this.listViewOption = page.getByText('List view')
this.gridViewOption = page.getByText('Grid view')
this.sortNewestFirst = page.getByText('Newest first')
this.sortOldestFirst = page.getByText('Oldest first')
this.assetCards = page
.getByRole('button')
.and(page.locator('[data-selected]'))
this.selectedCards = page.locator('[data-selected="true"]')
this.listViewItems = page.locator(
'.sidebar-content-container [role="button"][tabindex="0"]'
)
this.selectionFooter = page
.locator('.sidebar-content-container')
.locator('..')
.locator('[class*="h-18"]')
this.selectionCountButton = page.getByText(/Assets Selected: \d+/)
this.deselectAllButton = page.getByText('Deselect all')
this.deleteSelectedButton = page
.getByTestId('assets-delete-selected')
.or(page.locator('button:has(.icon-\\[lucide--trash-2\\])').last())
.first()
this.downloadSelectedButton = page
.getByTestId('assets-download-selected')
.or(page.locator('button:has(.icon-\\[lucide--download\\])').last())
.first()
this.backToAssetsButton = page.getByText('Back to all assets')
this.skeletonLoaders = page.locator(
'.sidebar-content-container .animate-pulse'
)
}
emptyStateTitle(title: string) {
return this.page.getByText(title)
}
// --- Search & filter ---
get searchInput() {
return this.page.getByPlaceholder('Search Assets...')
}
get settingsButton() {
return this.page.getByRole('button', { name: 'View settings' })
}
// --- View mode ---
get listViewOption() {
return this.page.getByText('List view')
}
get gridViewOption() {
return this.page.getByText('Grid view')
}
// --- Sort options (cloud-only, shown inside settings popover) ---
get sortNewestFirst() {
return this.page.getByText('Newest first')
}
get sortOldestFirst() {
return this.page.getByText('Oldest first')
}
// --- Asset cards ---
get assetCards() {
return this.page.locator('[role="button"][data-selected]')
}
getAssetCardByName(name: string) {
return this.page.locator('[role="button"][data-selected]', {
hasText: name
})
return this.assetCards.filter({ hasText: name })
}
get selectedCards() {
return this.page.locator('[data-selected="true"]')
}
// --- List view items ---
get listViewItems() {
return this.page.locator(
'.sidebar-content-container [role="button"][tabindex="0"]'
)
}
// --- Selection footer ---
get selectionFooter() {
return this.page
.locator('.sidebar-content-container')
.locator('..')
.locator('[class*="h-18"]')
}
get selectionCountButton() {
return this.page.getByText(/Assets Selected: \d+/)
}
get deselectAllButton() {
return this.page.getByText('Deselect all')
}
get deleteSelectedButton() {
return this.page
.getByTestId('assets-delete-selected')
.or(this.page.locator('button:has(.icon-\\[lucide--trash-2\\])').last())
.first()
}
get downloadSelectedButton() {
return this.page
.getByTestId('assets-download-selected')
.or(this.page.locator('button:has(.icon-\\[lucide--download\\])').last())
.first()
}
// --- Context menu ---
contextMenuItem(label: string) {
return this.page.locator('.p-contextmenu').getByText(label)
}
// --- Folder view ---
get backToAssetsButton() {
return this.page.getByText('Back to all assets')
}
// --- Loading ---
get skeletonLoaders() {
return this.page.locator('.sidebar-content-container .animate-pulse')
}
// --- Helpers ---
override async open() {
// Remove any toast notifications that may overlay the sidebar button
await this.dismissToasts()
@@ -423,28 +351,24 @@ export class AssetsSidebarTab extends SidebarTab {
async dismissToasts() {
const closeButtons = this.page.locator('.p-toast-close-button')
for (const btn of await closeButtons.all()) {
await btn.click({ force: true }).catch(() => {})
await btn.click().catch(() => {})
}
// Wait for all toast elements to fully animate out and detach from DOM
await expect(this.page.locator('.p-toast-message'))
.toHaveCount(0, { timeout: 5000 })
.toHaveCount(0)
.catch(() => {})
}
async switchToImported() {
await this.dismissToasts()
await this.importedTab.click()
await expect(this.importedTab).toHaveAttribute('aria-selected', 'true', {
timeout: 3000
})
await expect(this.importedTab).toHaveAttribute('aria-selected', 'true')
}
async switchToGenerated() {
await this.dismissToasts()
await this.generatedTab.click()
await expect(this.generatedTab).toHaveAttribute('aria-selected', 'true', {
timeout: 3000
})
await expect(this.generatedTab).toHaveAttribute('aria-selected', 'true')
}
async openSettingsMenu() {
@@ -467,7 +391,7 @@ export class AssetsSidebarTab extends SidebarTab {
async waitForAssets(count?: number) {
if (count !== undefined) {
await expect(this.assetCards).toHaveCount(count, { timeout: 5000 })
await expect(this.assetCards).toHaveCount(count)
} else {
await this.assetCards.first().waitFor({ state: 'visible', timeout: 5000 })
}

View File

@@ -10,6 +10,17 @@ export class SignInDialog extends BaseDialog {
readonly apiKeyButton: Locator
readonly termsLink: Locator
readonly privacyLink: Locator
readonly heading: Locator
readonly signUpLink: Locator
readonly signInLink: Locator
readonly signUpEmailInput: Locator
readonly signUpPasswordInput: Locator
readonly signUpConfirmPasswordInput: Locator
readonly signUpButton: Locator
readonly apiKeyHeading: Locator
readonly apiKeyInput: Locator
readonly backButton: Locator
readonly dividerText: Locator
constructor(page: Page) {
super(page)
@@ -22,6 +33,22 @@ export class SignInDialog extends BaseDialog {
})
this.termsLink = this.root.getByRole('link', { name: 'Terms of Use' })
this.privacyLink = this.root.getByRole('link', { name: 'Privacy Policy' })
this.heading = this.root.getByRole('heading').first()
this.signUpLink = this.root.getByText('Sign up', { exact: true })
this.signInLink = this.root.getByText('Sign in', { exact: true })
this.signUpEmailInput = this.root.locator('#comfy-org-sign-up-email')
this.signUpPasswordInput = this.root.locator('#comfy-org-sign-up-password')
this.signUpConfirmPasswordInput = this.root.locator(
'#comfy-org-sign-up-confirm-password'
)
this.signUpButton = this.root.getByRole('button', {
name: 'Sign up',
exact: true
})
this.apiKeyHeading = this.root.getByRole('heading', { name: 'API Key' })
this.apiKeyInput = this.root.locator('#comfy-org-api-key')
this.backButton = this.root.getByRole('button', { name: 'Back' })
this.dividerText = this.root.getByText('Or continue with')
}
async open() {
@@ -30,48 +57,4 @@ export class SignInDialog extends BaseDialog {
})
await this.waitForVisible()
}
get heading() {
return this.root.getByRole('heading').first()
}
get signUpLink() {
return this.root.getByText('Sign up', { exact: true })
}
get signInLink() {
return this.root.getByText('Sign in', { exact: true })
}
get signUpEmailInput() {
return this.root.locator('#comfy-org-sign-up-email')
}
get signUpPasswordInput() {
return this.root.locator('#comfy-org-sign-up-password')
}
get signUpConfirmPasswordInput() {
return this.root.locator('#comfy-org-sign-up-confirm-password')
}
get signUpButton() {
return this.root.getByRole('button', { name: 'Sign up', exact: true })
}
get apiKeyHeading() {
return this.root.getByRole('heading', { name: 'API Key' })
}
get apiKeyInput() {
return this.root.locator('#comfy-org-api-key')
}
get backButton() {
return this.root.getByRole('button', { name: 'Back' })
}
get dividerText() {
return this.root.getByText('Or continue with')
}
}

View File

@@ -5,10 +5,12 @@ import type { WorkspaceStore } from '@e2e/types/globals'
export class Topbar {
private readonly menuLocator: Locator
private readonly menuTrigger: Locator
readonly newWorkflowButton: Locator
constructor(public readonly page: Page) {
this.menuLocator = page.locator('.comfy-command-menu')
this.menuTrigger = page.locator('.comfy-menu-button-wrapper')
this.newWorkflowButton = page.locator('.new-blank-workflow-button')
}
async getTabNames(): Promise<string[]> {
@@ -50,10 +52,6 @@ export class Topbar {
return classes ? !classes.includes('invisible') : false
}
get newWorkflowButton(): Locator {
return this.page.locator('.new-blank-workflow-button')
}
getWorkflowTab(tabName: string): Locator {
return this.page
.locator(`.workflow-tabs .workflow-label:has-text("${tabName}")`)
@@ -73,7 +71,7 @@ export class Topbar {
async closeWorkflowTab(tabName: string) {
const tab = this.getWorkflowTab(tabName)
await tab.hover()
await tab.locator('.close-button').click({ force: true })
await tab.locator('.close-button').click()
}
getSaveDialog(): Locator {
@@ -107,7 +105,7 @@ export class Topbar {
{ timeout: 3000 }
)
// Wait for the dialog to close.
await this.getSaveDialog().waitFor({ state: 'hidden', timeout: 500 })
await this.getSaveDialog().waitFor({ state: 'hidden' })
// Check if a confirmation dialog appeared (e.g., "Overwrite existing file?")
// If so, return early to let the test handle the confirmation

View File

@@ -3,6 +3,7 @@ import type { Locator, Page } from '@playwright/test'
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import { TestIds } from '@e2e/fixtures/selectors'
import { OutputHistoryComponent } from '@e2e/fixtures/components/OutputHistory'
import { AppModeWidgetHelper } from '@e2e/fixtures/helpers/AppModeWidgetHelper'
import { BuilderFooterHelper } from '@e2e/fixtures/helpers/BuilderFooterHelper'
import { BuilderSaveAsHelper } from '@e2e/fixtures/helpers/BuilderSaveAsHelper'
@@ -14,14 +15,66 @@ export class AppModeHelper {
readonly footer: BuilderFooterHelper
readonly saveAs: BuilderSaveAsHelper
readonly select: BuilderSelectHelper
readonly outputHistory: OutputHistoryComponent
readonly widgets: AppModeWidgetHelper
/** The "Connect an output" popover shown when saving without outputs. */
public readonly connectOutputPopover: Locator
/** The empty-state placeholder shown when no outputs are selected. */
public readonly outputPlaceholder: Locator
/** The linear-mode widget list container (visible in app mode). */
public readonly linearWidgets: Locator
/** The PrimeVue Popover for the image picker (renders with role="dialog"). */
public readonly imagePickerPopover: Locator
/** The Run button in the app mode footer. */
public readonly runButton: Locator
/** The welcome screen shown when app mode has no outputs or no nodes. */
public readonly welcome: Locator
/** The empty workflow message shown when no nodes exist. */
public readonly emptyWorkflowText: Locator
/** The "Build app" button shown when nodes exist but no outputs. */
public readonly buildAppButton: Locator
/** The "Back to workflow" button on the welcome screen. */
public readonly backToWorkflowButton: Locator
/** The "Load template" button shown when no nodes exist. */
public readonly loadTemplateButton: Locator
/** The cancel button for an in-progress run in the output history. */
public readonly cancelRunButton: Locator
constructor(private readonly comfyPage: ComfyPage) {
this.steps = new BuilderStepsHelper(comfyPage)
this.footer = new BuilderFooterHelper(comfyPage)
this.saveAs = new BuilderSaveAsHelper(comfyPage)
this.select = new BuilderSelectHelper(comfyPage)
this.outputHistory = new OutputHistoryComponent(comfyPage.page)
this.widgets = new AppModeWidgetHelper(comfyPage)
this.connectOutputPopover = this.page.getByTestId(
TestIds.builder.connectOutputPopover
)
this.outputPlaceholder = this.page.getByTestId(
TestIds.builder.outputPlaceholder
)
this.linearWidgets = this.page.getByTestId('linear-widgets')
this.imagePickerPopover = this.page
.getByRole('dialog')
.filter({ has: this.page.getByRole('button', { name: 'All' }) })
.first()
this.runButton = this.page
.getByTestId('linear-run-button')
.getByRole('button', { name: /run/i })
this.welcome = this.page.getByTestId(TestIds.appMode.welcome)
this.emptyWorkflowText = this.page.getByTestId(
TestIds.appMode.emptyWorkflow
)
this.buildAppButton = this.page.getByTestId(TestIds.appMode.buildApp)
this.backToWorkflowButton = this.page.getByTestId(
TestIds.appMode.backToWorkflow
)
this.loadTemplateButton = this.page.getByTestId(
TestIds.appMode.loadTemplate
)
this.cancelRunButton = this.page.getByTestId(
TestIds.outputHistory.cancelRun
)
}
private get page(): Page {
@@ -93,61 +146,6 @@ export class AppModeHelper {
await this.toggleAppMode()
}
/** The "Connect an output" popover shown when saving without outputs. */
get connectOutputPopover(): Locator {
return this.page.getByTestId(TestIds.builder.connectOutputPopover)
}
/** The empty-state placeholder shown when no outputs are selected. */
get outputPlaceholder(): Locator {
return this.page.getByTestId(TestIds.builder.outputPlaceholder)
}
/** The linear-mode widget list container (visible in app mode). */
get linearWidgets(): Locator {
return this.page.locator('[data-testid="linear-widgets"]')
}
/** The PrimeVue Popover for the image picker (renders with role="dialog"). */
get imagePickerPopover(): Locator {
return this.page
.getByRole('dialog')
.filter({ has: this.page.getByRole('button', { name: 'All' }) })
.first()
}
/** The Run button in the app mode footer. */
get runButton(): Locator {
return this.page
.getByTestId('linear-run-button')
.getByRole('button', { name: /run/i })
}
/** The welcome screen shown when app mode has no outputs or no nodes. */
get welcome(): Locator {
return this.page.getByTestId(TestIds.appMode.welcome)
}
/** The empty workflow message shown when no nodes exist. */
get emptyWorkflowText(): Locator {
return this.page.getByTestId(TestIds.appMode.emptyWorkflow)
}
/** The "Build app" button shown when nodes exist but no outputs. */
get buildAppButton(): Locator {
return this.page.getByTestId(TestIds.appMode.buildApp)
}
/** The "Back to workflow" button on the welcome screen. */
get backToWorkflowButton(): Locator {
return this.page.getByTestId(TestIds.appMode.backToWorkflow)
}
/** The "Load template" button shown when no nodes exist. */
get loadTemplateButton(): Locator {
return this.page.getByTestId(TestIds.appMode.loadTemplate)
}
/**
* Get the actions menu trigger for a widget in the app mode widget list.
* @param widgetName Text shown in the widget label (e.g. "seed").

View File

@@ -4,48 +4,32 @@ import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import { TestIds } from '@e2e/fixtures/selectors'
export class BuilderFooterHelper {
constructor(private readonly comfyPage: ComfyPage) {}
public readonly nav: Locator
public readonly exitButton: Locator
public readonly nextButton: Locator
public readonly backButton: Locator
public readonly saveButton: Locator
public readonly saveGroup: Locator
public readonly saveAsButton: Locator
public readonly saveAsChevron: Locator
public readonly opensAsPopover: Locator
constructor(private readonly comfyPage: ComfyPage) {
this.nav = this.page.getByTestId(TestIds.builder.footerNav)
this.exitButton = this.buttonByName('Exit app builder')
this.nextButton = this.buttonByName('Next')
this.backButton = this.buttonByName('Back')
this.saveButton = this.page.getByTestId(TestIds.builder.saveButton)
this.saveGroup = this.page.getByTestId(TestIds.builder.saveGroup)
this.saveAsButton = this.page.getByTestId(TestIds.builder.saveAsButton)
this.saveAsChevron = this.page.getByTestId(TestIds.builder.saveAsChevron)
this.opensAsPopover = this.page.getByTestId(TestIds.builder.opensAs)
}
private get page(): Page {
return this.comfyPage.page
}
get nav(): Locator {
return this.page.getByTestId(TestIds.builder.footerNav)
}
get exitButton(): Locator {
return this.buttonByName('Exit app builder')
}
get nextButton(): Locator {
return this.buttonByName('Next')
}
get backButton(): Locator {
return this.buttonByName('Back')
}
get saveButton(): Locator {
return this.page.getByTestId(TestIds.builder.saveButton)
}
get saveGroup(): Locator {
return this.page.getByTestId(TestIds.builder.saveGroup)
}
get saveAsButton(): Locator {
return this.page.getByTestId(TestIds.builder.saveAsButton)
}
get saveAsChevron(): Locator {
return this.page.getByTestId(TestIds.builder.saveAsChevron)
}
get opensAsPopover(): Locator {
return this.page.getByTestId(TestIds.builder.opensAs)
}
private buttonByName(name: string): Locator {
return this.nav.getByRole('button', { name })
}

View File

@@ -3,73 +3,61 @@ import type { Locator, Page } from '@playwright/test'
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
export class BuilderSaveAsHelper {
constructor(private readonly comfyPage: ComfyPage) {}
/** The save-as dialog (scoped by aria-labelledby). */
public readonly dialog: Locator
/** The post-save success dialog (scoped by aria-labelledby). */
public readonly successDialog: Locator
public readonly title: Locator
public readonly radioGroup: Locator
public readonly nameInput: Locator
public readonly saveButton: Locator
public readonly successMessage: Locator
public readonly viewAppButton: Locator
public readonly closeButton: Locator
/** The X button to dismiss the success dialog without any action. */
public readonly dismissButton: Locator
public readonly exitBuilderButton: Locator
public readonly overwriteDialog: Locator
public readonly overwriteButton: Locator
constructor(private readonly comfyPage: ComfyPage) {
this.dialog = this.page.locator('[aria-labelledby="builder-save"]')
this.successDialog = this.page.locator(
'[aria-labelledby="builder-save-success"]'
)
this.title = this.dialog.getByText('Save as')
this.radioGroup = this.dialog.getByRole('radiogroup')
this.nameInput = this.dialog.getByRole('textbox')
this.saveButton = this.dialog.getByRole('button', { name: 'Save' })
this.successMessage = this.successDialog.getByText('Successfully saved')
this.viewAppButton = this.successDialog.getByRole('button', {
name: 'View app'
})
this.closeButton = this.successDialog
.getByRole('button', { name: 'Close', exact: true })
.filter({ hasText: 'Close' })
this.dismissButton = this.successDialog.locator(
'button.p-dialog-close-button'
)
this.exitBuilderButton = this.successDialog.getByRole('button', {
name: 'Exit builder'
})
this.overwriteDialog = this.page.getByRole('dialog', {
name: 'Overwrite existing file?'
})
this.overwriteButton = this.overwriteDialog.getByRole('button', {
name: 'Overwrite'
})
}
private get page(): Page {
return this.comfyPage.page
}
/** The save-as dialog (scoped by aria-labelledby). */
get dialog(): Locator {
return this.page.locator('[aria-labelledby="builder-save"]')
}
/** The post-save success dialog (scoped by aria-labelledby). */
get successDialog(): Locator {
return this.page.locator('[aria-labelledby="builder-save-success"]')
}
get title(): Locator {
return this.dialog.getByText('Save as')
}
get radioGroup(): Locator {
return this.dialog.getByRole('radiogroup')
}
get nameInput(): Locator {
return this.dialog.getByRole('textbox')
}
viewTypeRadio(viewType: 'App' | 'Node graph'): Locator {
return this.dialog.getByRole('radio', { name: viewType })
}
get saveButton(): Locator {
return this.dialog.getByRole('button', { name: 'Save' })
}
get successMessage(): Locator {
return this.successDialog.getByText('Successfully saved')
}
get viewAppButton(): Locator {
return this.successDialog.getByRole('button', { name: 'View app' })
}
get closeButton(): Locator {
return this.successDialog
.getByRole('button', { name: 'Close', exact: true })
.filter({ hasText: 'Close' })
}
/** The X button to dismiss the success dialog without any action. */
get dismissButton(): Locator {
return this.successDialog.locator('button.p-dialog-close-button')
}
get exitBuilderButton(): Locator {
return this.successDialog.getByRole('button', { name: 'Exit builder' })
}
get overwriteDialog(): Locator {
return this.page.getByRole('dialog', { name: 'Overwrite existing file?' })
}
get overwriteButton(): Locator {
return this.overwriteDialog.getByRole('button', { name: 'Overwrite' })
}
async fillAndSave(workflowName: string, viewType: 'App' | 'Node graph') {
await this.nameInput.fill(workflowName)
await this.viewTypeRadio(viewType).click()

View File

@@ -32,7 +32,20 @@ async function dragByIndex(items: Locator, fromIndex: number, toIndex: number) {
}
export class BuilderSelectHelper {
constructor(private readonly comfyPage: ComfyPage) {}
/** All IoItem locators in the current step sidebar. */
public readonly inputItems: Locator
/** All IoItem title locators in the inputs step sidebar. */
public readonly inputItemTitles: Locator
/** All widget label locators in the preview/arrange sidebar. */
public readonly previewWidgetLabels: Locator
constructor(private readonly comfyPage: ComfyPage) {
this.inputItems = this.page.getByTestId(TestIds.builder.ioItem)
this.inputItemTitles = this.page.getByTestId(TestIds.builder.ioItemTitle)
this.previewWidgetLabels = this.page.getByTestId(
TestIds.builder.widgetLabel
)
}
private get page(): Page {
return this.comfyPage.page
@@ -43,12 +56,9 @@ export class BuilderSelectHelper {
* @param title The widget title shown in the IoItem.
*/
getInputItemMenu(title: string): Locator {
return this.page
.getByTestId(TestIds.builder.ioItem)
return this.inputItems
.filter({
has: this.page
.getByTestId(TestIds.builder.ioItemTitle)
.getByText(title, { exact: true })
has: this.inputItemTitles.getByText(title, { exact: true })
})
.getByTestId(TestIds.builder.widgetActionsMenu)
}
@@ -141,6 +151,7 @@ export class BuilderSelectHelper {
const widgetLocator = this.comfyPage.vueNodes
.getNodeLocator(String(nodeRef.id))
.getByLabel(widgetName, { exact: true })
// oxlint-disable-next-line playwright/no-force-option -- Node container has conditional pointer-events:none that blocks actionability
await widgetLocator.click({ force: true })
await this.comfyPage.nextFrame()
}
@@ -150,38 +161,19 @@ export class BuilderSelectHelper {
* Useful for asserting "Widget not visible" on disconnected inputs.
*/
getInputItemSubtitle(title: string): Locator {
return this.page
.getByTestId(TestIds.builder.ioItem)
return this.inputItems
.filter({
has: this.page
.getByTestId(TestIds.builder.ioItemTitle)
.getByText(title, { exact: true })
has: this.inputItemTitles.getByText(title, { exact: true })
})
.getByTestId(TestIds.builder.ioItemSubtitle)
}
/** All IoItem locators in the current step sidebar. */
get inputItems(): Locator {
return this.page.getByTestId(TestIds.builder.ioItem)
}
/** All IoItem title locators in the inputs step sidebar. */
get inputItemTitles(): Locator {
return this.page.getByTestId(TestIds.builder.ioItemTitle)
}
/** All widget label locators in the preview/arrange sidebar. */
get previewWidgetLabels(): Locator {
return this.page.getByTestId(TestIds.builder.widgetLabel)
}
/**
* Drag an IoItem from one index to another in the inputs step.
* Items are identified by their 0-based position among visible IoItems.
*/
async dragInputItem(fromIndex: number, toIndex: number) {
const items = this.page.getByTestId(TestIds.builder.ioItem)
await dragByIndex(items, fromIndex, toIndex)
await dragByIndex(this.inputItems, fromIndex, toIndex)
await this.comfyPage.nextFrame()
}
@@ -208,6 +200,7 @@ export class BuilderSelectHelper {
const nodeLocator = this.comfyPage.vueNodes.getNodeLocator(
String(nodeRef.id)
)
// oxlint-disable-next-line playwright/no-force-option -- Node container has conditional pointer-events:none that blocks actionability
await nodeLocator.click({ force: true })
await this.comfyPage.nextFrame()
}

View File

@@ -3,16 +3,16 @@ import type { Locator, Page } from '@playwright/test'
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
export class BuilderStepsHelper {
constructor(private readonly comfyPage: ComfyPage) {}
public readonly toolbar: Locator
constructor(private readonly comfyPage: ComfyPage) {
this.toolbar = this.page.getByRole('navigation', { name: 'App Builder' })
}
private get page(): Page {
return this.comfyPage.page
}
get toolbar(): Locator {
return this.page.getByRole('navigation', { name: 'App Builder' })
}
async goToInputs() {
await this.toolbar.getByRole('button', { name: 'Inputs' }).click()
await this.comfyPage.nextFrame()

View File

@@ -74,6 +74,51 @@ export class CanvasHelper {
await this.nextFrame()
}
/**
* Convert a canvas-element-relative position to absolute page coordinates.
* Use with `page.mouse` APIs when Vue DOM overlays above the canvas would
* cause Playwright's actionability check to fail on the canvas locator.
*/
private async toAbsolute(position: Position): Promise<Position> {
const box = await this.canvas.boundingBox()
if (!box) throw new Error('Canvas bounding box not available')
return { x: box.x + position.x, y: box.y + position.y }
}
/**
* Click at canvas-element-relative coordinates using `page.mouse.click()`.
* Bypasses Playwright's actionability checks on the canvas locator, which
* can fail when Vue-rendered DOM nodes overlay the `<canvas>` element.
*/
async mouseClickAt(
position: Position,
options?: {
button?: 'left' | 'right' | 'middle'
modifiers?: ('Shift' | 'Control' | 'Alt' | 'Meta')[]
}
): Promise<void> {
const abs = await this.toAbsolute(position)
const modifiers = options?.modifiers ?? []
for (const mod of modifiers) await this.page.keyboard.down(mod)
try {
await this.page.mouse.click(abs.x, abs.y, {
button: options?.button
})
} finally {
for (const mod of modifiers) await this.page.keyboard.up(mod)
}
await this.nextFrame()
}
/**
* Double-click at canvas-element-relative coordinates using `page.mouse`.
*/
async mouseDblclickAt(position: Position): Promise<void> {
const abs = await this.toAbsolute(position)
await this.page.mouse.dblclick(abs.x, abs.y)
await this.nextFrame()
}
async clickEmptySpace(): Promise<void> {
await this.canvas.click({ position: DefaultGraphPositions.emptySpaceClick })
await this.nextFrame()

View File

@@ -0,0 +1,211 @@
import type { WebSocketRoute } from '@playwright/test'
import type { RawJobListItem } from '@/platform/remote/comfyui/jobs/jobTypes'
import type { ComfyPage } from '../ComfyPage'
import { createMockJob } from './AssetsHelper'
/**
* Helper for simulating prompt execution in e2e tests.
*/
export class ExecutionHelper {
private jobCounter = 0
private readonly completedJobs: RawJobListItem[] = []
private readonly page: ComfyPage['page']
private readonly command: ComfyPage['command']
private readonly assets: ComfyPage['assets']
constructor(
comfyPage: ComfyPage,
private readonly ws: WebSocketRoute
) {
this.page = comfyPage.page
this.command = comfyPage.command
this.assets = comfyPage.assets
}
/**
* Intercept POST /api/prompt, execute Comfy.QueuePrompt, and return
* the synthetic job ID.
*
* The app receives a valid PromptResponse so storeJob() fires
* and registers the job against the active workflow path.
*/
async run(): Promise<string> {
const jobId = `test-job-${++this.jobCounter}`
let fulfilled!: () => void
const prompted = new Promise<void>((r) => {
fulfilled = r
})
await this.page.route(
'**/api/prompt',
async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
prompt_id: jobId,
node_errors: {}
})
})
fulfilled()
},
{ times: 1 }
)
await this.command.executeCommand('Comfy.QueuePrompt')
await prompted
return jobId
}
/**
* Send a binary `b_preview_with_metadata` WS message (type 4).
* Encodes the metadata and a tiny 1x1 PNG so the app creates a blob URL.
*/
latentPreview(jobId: string, nodeId: string): void {
const metadata = JSON.stringify({
node_id: nodeId,
display_node_id: nodeId,
parent_node_id: nodeId,
real_node_id: nodeId,
prompt_id: jobId,
image_type: 'image/png'
})
const metadataBytes = new TextEncoder().encode(metadata)
// 1x1 red PNG
const png = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwADhQGAWjR9awAAAABJRU5ErkJggg==',
'base64'
)
// Binary format: [type:uint32][metadataLength:uint32][metadata][imageData]
const buf = new ArrayBuffer(8 + metadataBytes.length + png.length)
const view = new DataView(buf)
view.setUint32(0, 4) // type 4 = PREVIEW_IMAGE_WITH_METADATA
view.setUint32(4, metadataBytes.length)
new Uint8Array(buf, 8, metadataBytes.length).set(metadataBytes)
new Uint8Array(buf, 8 + metadataBytes.length).set(png)
this.ws.send(Buffer.from(buf))
}
/** Send `execution_start` WS event. */
executionStart(jobId: string): void {
this.ws.send(
JSON.stringify({
type: 'execution_start',
data: { prompt_id: jobId, timestamp: Date.now() }
})
)
}
/** Send `executing` WS event to signal which node is currently running. */
executing(jobId: string, nodeId: string | null): void {
this.ws.send(
JSON.stringify({
type: 'executing',
data: { prompt_id: jobId, node: nodeId }
})
)
}
/** Send `executed` WS event with node output. */
executed(
jobId: string,
nodeId: string,
output: Record<string, unknown>
): void {
this.ws.send(
JSON.stringify({
type: 'executed',
data: {
prompt_id: jobId,
node: nodeId,
display_node: nodeId,
output
}
})
)
}
/** Send `execution_success` WS event. */
executionSuccess(jobId: string): void {
this.ws.send(
JSON.stringify({
type: 'execution_success',
data: { prompt_id: jobId, timestamp: Date.now() }
})
)
}
/** Send `execution_error` WS event. */
executionError(jobId: string, nodeId: string, message: string): void {
this.ws.send(
JSON.stringify({
type: 'execution_error',
data: {
prompt_id: jobId,
timestamp: Date.now(),
node_id: nodeId,
node_type: 'Unknown',
exception_message: message,
exception_type: 'RuntimeError',
traceback: []
}
})
)
}
/** Send `progress` WS event. */
progress(jobId: string, nodeId: string, value: number, max: number): void {
this.ws.send(
JSON.stringify({
type: 'progress',
data: { prompt_id: jobId, node: nodeId, value, max }
})
)
}
/**
* Complete a job by adding it to mock history, sending execution_success,
* and triggering a history refresh via a status event.
*
* Requires an {@link AssetsHelper} to be passed in the constructor.
*/
async completeWithHistory(
jobId: string,
nodeId: string,
filename: string
): Promise<void> {
this.completedJobs.push(
createMockJob({
id: jobId,
preview_output: {
filename,
subfolder: '',
type: 'output',
nodeId,
mediaType: 'images'
}
})
)
await this.assets.mockOutputHistory(this.completedJobs)
this.executionSuccess(jobId)
// Trigger queue/history refresh
this.status(0)
}
/** Send `status` WS event to update queue count. */
status(queueRemaining: number): void {
this.ws.send(
JSON.stringify({
type: 'status',
data: { status: { exec_info: { queue_remaining: queueRemaining } } }
})
)
}
}

View File

@@ -8,7 +8,13 @@ import type { Position, Size } from '@e2e/fixtures/types'
import { NodeReference } from '@e2e/fixtures/utils/litegraphUtils'
export class NodeOperationsHelper {
constructor(private comfyPage: ComfyPage) {}
public readonly promptDialogInput: Locator
constructor(private comfyPage: ComfyPage) {
this.promptDialogInput = this.page.locator(
'.p-dialog-content input[type="text"]'
)
}
private get page() {
return this.comfyPage.page
@@ -155,10 +161,6 @@ export class NodeOperationsHelper {
await this.comfyPage.nextFrame()
}
get promptDialogInput(): Locator {
return this.page.locator('.p-dialog-content input[type="text"]')
}
async fillPromptDialog(value: string): Promise<void> {
await this.promptDialogInput.fill(value)
await this.page.keyboard.press('Enter')

View File

@@ -157,7 +157,7 @@ export class SubgraphHelper {
// Wait for the appropriate UI element to appear
if (action === 'rightClick') {
await this.page.waitForSelector('.litemenu-entry', {
await this.page.locator('.litemenu-entry').first().waitFor({
state: 'visible',
timeout: 5000
})

View File

@@ -2,14 +2,12 @@ import { expect } from '@playwright/test'
import type { Locator, Page } from '@playwright/test'
export class ToastHelper {
constructor(private readonly page: Page) {}
public readonly visibleToasts: Locator
public readonly toastErrors: Locator
get visibleToasts(): Locator {
return this.page.locator('.p-toast-message:visible')
}
get toastErrors(): Locator {
return this.page.locator('.p-toast-message.p-toast-message-error')
constructor(private readonly page: Page) {
this.visibleToasts = page.locator('.p-toast-message:visible')
this.toastErrors = page.locator('.p-toast-message.p-toast-message-error')
}
async closeToasts(requireCount = 0): Promise<void> {
@@ -28,6 +26,6 @@ export class ToastHelper {
}
// Assert all toasts are closed
await expect(this.visibleToasts).toHaveCount(0, { timeout: 1000 })
await expect(this.visibleToasts).toHaveCount(0)
}
}

View File

@@ -21,6 +21,10 @@ export const TestIds = {
contextMenu: 'canvas-context-menu',
toggleMinimapButton: 'toggle-minimap-button',
closeMinimapButton: 'close-minimap-button',
minimapContainer: 'minimap-container',
minimapCanvas: 'minimap-canvas',
minimapViewport: 'minimap-viewport',
minimapInteractionOverlay: 'minimap-interaction-overlay',
toggleLinkVisibilityButton: 'toggle-link-visibility-button',
zoomControlsButton: 'zoom-controls-button',
zoomInAction: 'zoom-in-action',
@@ -130,6 +134,24 @@ export const TestIds = {
outputPlaceholder: 'builder-output-placeholder',
connectOutputPopover: 'builder-connect-output-popover'
},
outputHistory: {
outputs: 'linear-outputs',
welcome: 'linear-welcome',
outputInfo: 'linear-output-info',
activeQueue: 'linear-job',
queueBadge: 'linear-job-badge',
inProgressItem: 'linear-in-progress-item',
historyItem: 'linear-history-item',
skeleton: 'linear-skeleton',
latentPreview: 'linear-latent-preview',
imageOutput: 'linear-image-output',
videoOutput: 'linear-video-output',
cancelRun: 'linear-cancel-run',
headerProgressBar: 'linear-header-progress-bar',
itemProgressBar: 'linear-item-progress-bar',
progressOverall: 'linear-progress-overall',
progressNode: 'linear-progress-node'
},
appMode: {
widgetItem: 'app-mode-widget-item',
welcome: 'linear-welcome',
@@ -180,6 +202,7 @@ export type TestIdValue =
| (typeof TestIds.selectionToolbox)[keyof typeof TestIds.selectionToolbox]
| (typeof TestIds.widgets)[keyof typeof TestIds.widgets]
| (typeof TestIds.builder)[keyof typeof TestIds.builder]
| (typeof TestIds.outputHistory)[keyof typeof TestIds.outputHistory]
| (typeof TestIds.appMode)[keyof typeof TestIds.appMode]
| (typeof TestIds.breadcrumb)[keyof typeof TestIds.breadcrumb]
| Exclude<

View File

@@ -14,11 +14,12 @@ function makeMatcher<T>(
) {
await expect(async () => {
const value = await getValue(node)
const assertion = this.isNot
? expect(value, 'Node is ' + type).not
: expect(value, 'Node is not ' + type)
assertion.toBeTruthy()
}).toPass({ timeout: 250, ...options })
if (this.isNot) {
expect(value, 'Node is ' + type).not.toBeTruthy()
} else {
expect(value, 'Node is not ' + type).toBeTruthy()
}
}).toPass({ timeout: 5000, ...options })
return {
pass: !this.isNot,
message: () => 'Node is ' + (this.isNot ? 'not ' : '') + type
@@ -30,7 +31,7 @@ export const comfyExpect = expect.extend({
toBePinned: makeMatcher((n) => n.isPinned(), 'pinned'),
toBeBypassed: makeMatcher((n) => n.isBypassed(), 'bypassed'),
toBeCollapsed: makeMatcher((n) => n.isCollapsed(), 'collapsed'),
async toHaveFocus(locator: Locator, options = { timeout: 256 }) {
async toHaveFocus(locator: Locator, options = {}) {
await expect
.poll(
() => locator.evaluate((el) => el === document.activeElement),

View File

@@ -1,5 +1,4 @@
import { expect } from '@playwright/test'
import type { Page } from '@playwright/test'
import type { NodeId } from '@/platform/workflow/validation/schemas/workflowSchema'
import { ManageGroupNode } from '@e2e/helpers/manageGroupNode'
@@ -356,7 +355,11 @@ export class NodeReference {
}
async click(
position: 'title' | 'collapse',
options?: Parameters<Page['click']>[1] & { moveMouseToEmptyArea?: boolean }
options?: {
button?: 'left' | 'right' | 'middle'
modifiers?: ('Shift' | 'Control' | 'Alt' | 'Meta')[]
moveMouseToEmptyArea?: boolean
}
) {
let clickPos: Position
switch (position) {
@@ -377,12 +380,7 @@ export class NodeReference {
delete options.moveMouseToEmptyArea
}
await this.comfyPage.canvas.click({
...options,
position: clickPos,
force: true
})
await this.comfyPage.nextFrame()
await this.comfyPage.canvasOps.mouseClickAt(clickPos, options)
if (moveMouseToEmptyArea) {
await this.comfyPage.canvasOps.moveMouseToEmptyArea()
}
@@ -499,31 +497,18 @@ export class NodeReference {
await expect(async () => {
// Try just clicking the enter button first
await this.comfyPage.canvas.click({
position: { x: 250, y: 250 },
force: true
})
await this.comfyPage.nextFrame()
await this.comfyPage.canvasOps.mouseClickAt({ x: 250, y: 250 })
await this.comfyPage.canvas.click({
position: subgraphButtonPos,
force: true
})
await this.comfyPage.nextFrame()
await this.comfyPage.canvasOps.mouseClickAt(subgraphButtonPos)
if (await checkIsInSubgraph()) return
for (const position of clickPositions) {
// Clear any selection first
await this.comfyPage.canvas.click({
position: { x: 250, y: 250 },
force: true
})
await this.comfyPage.nextFrame()
await this.comfyPage.canvasOps.mouseClickAt({ x: 250, y: 250 })
// Double-click to enter subgraph
await this.comfyPage.canvas.dblclick({ position, force: true })
await this.comfyPage.nextFrame()
await this.comfyPage.canvasOps.mouseDblclickAt(position)
if (await checkIsInSubgraph()) return
}

View File

@@ -4,38 +4,24 @@ import { TestIds } from '@e2e/fixtures/selectors'
/** DOM-centric helper for a single Vue-rendered node on the canvas. */
export class VueNodeFixture {
constructor(private readonly locator: Locator) {}
public readonly header: Locator
public readonly title: Locator
public readonly titleInput: Locator
public readonly body: Locator
public readonly pinIndicator: Locator
public readonly collapseButton: Locator
public readonly collapseIcon: Locator
public readonly root: Locator
get header(): Locator {
return this.locator.locator('[data-testid^="node-header-"]')
}
get title(): Locator {
return this.locator.locator('[data-testid="node-title"]')
}
get titleInput(): Locator {
return this.locator.locator('[data-testid="node-title-input"]')
}
get body(): Locator {
return this.locator.locator('[data-testid^="node-body-"]')
}
get pinIndicator(): Locator {
return this.locator.getByTestId(TestIds.node.pinIndicator)
}
get collapseButton(): Locator {
return this.locator.locator('[data-testid="node-collapse-button"]')
}
get collapseIcon(): Locator {
return this.collapseButton.locator('i')
}
get root(): Locator {
return this.locator
constructor(private readonly locator: Locator) {
this.header = locator.locator('[data-testid^="node-header-"]')
this.title = locator.getByTestId('node-title')
this.titleInput = locator.getByTestId('node-title-input')
this.body = locator.locator('[data-testid^="node-body-"]')
this.pinIndicator = locator.getByTestId(TestIds.node.pinIndicator)
this.collapseButton = locator.getByTestId('node-collapse-button')
this.collapseIcon = this.collapseButton.locator('i')
this.root = locator
}
async getTitle(): Promise<string> {

View File

@@ -1,53 +1,31 @@
import { test as base } from '@playwright/test'
import type { WebSocketRoute } from '@playwright/test'
export const webSocketFixture = base.extend<{
ws: { trigger(data: unknown, url?: string): Promise<void> }
getWebSocket: () => Promise<WebSocketRoute>
}>({
ws: [
async ({ page }, use) => {
// Each time a page loads, to catch navigations
page.on('load', async () => {
await page.evaluate(function () {
// Create a wrapper for WebSocket that stores them globally
// so we can look it up to trigger messages
const store: Record<string, WebSocket> = (window.__ws__ = {})
window.WebSocket = class extends window.WebSocket {
constructor(
...rest: ConstructorParameters<typeof window.WebSocket>
) {
super(...rest)
store[this.url] = this
}
}
getWebSocket: [
async ({ context }, use) => {
let latest: WebSocketRoute | undefined
let resolve: ((ws: WebSocketRoute) => void) | undefined
await context.routeWebSocket(/\/ws/, (ws) => {
const server = ws.connectToServer()
server.onMessage((message) => {
ws.send(message)
})
latest = ws
resolve?.(ws)
})
await use({
async trigger(data, url) {
// Trigger a websocket event on the page
await page.evaluate(
function ([data, url]) {
if (!url) {
// If no URL specified, use page URL
const u = new URL(window.location.href)
u.hash = ''
u.protocol = 'ws:'
u.pathname = '/'
url = u.toString() + 'ws'
}
const ws: WebSocket = window.__ws__![url]
ws.dispatchEvent(
new MessageEvent('message', {
data
})
)
},
[JSON.stringify(data), url]
)
}
await use(() => {
if (latest) return Promise.resolve(latest)
return new Promise<WebSocketRoute>((r) => {
resolve = r
})
})
},
// We need this to run automatically as the first thing so it adds handlers as soon as the page loads
{ auto: true }
]
})

View File

@@ -82,11 +82,9 @@ export async function builderSaveAs(
viewType: 'App' | 'Node graph' = 'App'
) {
await appMode.footer.saveAsButton.click()
await comfyExpect(appMode.saveAs.nameInput).toBeVisible({ timeout: 5000 })
await comfyExpect(appMode.saveAs.nameInput).toBeVisible()
await appMode.saveAs.fillAndSave(workflowName, viewType)
await comfyExpect(appMode.saveAs.successMessage).toBeVisible({
timeout: 5000
})
await comfyExpect(appMode.saveAs.successMessage).toBeVisible()
}
/**
@@ -124,3 +122,21 @@ export async function saveAndReopenInAppMode(
await comfyPage.appMode.toggleAppMode()
}
/**
* Enter builder, select the given widgets as inputs + SaveImage as output,
* save as an app, and close the success dialog.
*
* Returns on the builder arrange/preview step.
*/
export async function createAndSaveApp(
comfyPage: ComfyPage,
appName: string,
widgetNames: string[] = ['seed']
): Promise<void> {
await setupBuilder(comfyPage, undefined, widgetNames)
await comfyPage.appMode.steps.goToPreview()
await builderSaveAs(comfyPage.appMode, appName)
await comfyPage.appMode.saveAs.closeButton.click()
await comfyPage.nextFrame()
}

View File

@@ -0,0 +1,66 @@
import type { Locator, Page } from '@playwright/test'
import type { TestGraphAccess } from '@e2e/types/globals'
export async function drawStroke(
page: Page,
canvas: Locator,
opts: { startXPct?: number; endXPct?: number; yPct?: number } = {}
): Promise<void> {
const { startXPct = 0.3, endXPct = 0.7, yPct = 0.5 } = opts
const box = await canvas.boundingBox()
if (!box) throw new Error('Canvas bounding box not found')
await page.mouse.move(
box.x + box.width * startXPct,
box.y + box.height * yPct
)
await page.mouse.down()
await page.mouse.move(
box.x + box.width * endXPct,
box.y + box.height * yPct,
{ steps: 10 }
)
await page.mouse.up()
}
export async function hasCanvasContent(canvas: Locator): Promise<boolean> {
return canvas.evaluate((el: HTMLCanvasElement) => {
const ctx = el.getContext('2d')
if (!ctx) return false
const { data } = ctx.getImageData(0, 0, el.width, el.height)
for (let i = 3; i < data.length; i += 4) {
if (data[i] > 0) return true
}
return false
})
}
export async function triggerSerialization(page: Page): Promise<void> {
await page.evaluate(async () => {
const graph = window.graph as TestGraphAccess | undefined
if (!graph) {
throw new Error(
'Global window.graph is absent. Ensure workflow fixture is loaded.'
)
}
const node = graph._nodes_by_id?.['1']
if (!node) {
throw new Error(
'Target node with ID "1" not found in graph._nodes_by_id.'
)
}
const widget = node.widgets?.find((w) => w.name === 'mask')
if (!widget) {
throw new Error('Widget "mask" not found on target node 1.')
}
if (typeof widget.serializeValue !== 'function') {
throw new Error(
'mask widget on node 1 does not have a serializeValue function.'
)
}
await widget.serializeValue(node, 0)
})
}

View File

@@ -1,7 +1,6 @@
import type { Response } from '@playwright/test'
import { expect, mergeTests } from '@playwright/test'
import type { StatusWsMessage } from '@/schemas/apiSchema'
import { comfyPageFixture } from '@e2e/fixtures/ComfyPage'
import { webSocketFixture } from '@e2e/fixtures/ws'
import type { WorkspaceStore } from '@e2e/types/globals'
@@ -18,8 +17,10 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
*/
test('Does not auto-queue multiple changes at a time', async ({
comfyPage,
ws
getWebSocket
}) => {
const ws = await getWebSocket()
// Enable change auto-queue mode
const queueOpts = await comfyPage.actionbar.queueButton.toggleOptions()
await expect.poll(() => queueOpts.getMode()).toBe('disabled')
@@ -62,17 +63,19 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
}
// Trigger a status websocket message
const triggerStatus = async (queueSize: number) => {
await ws.trigger({
type: 'status',
data: {
status: {
exec_info: {
queue_remaining: queueSize
const triggerStatus = (queueSize: number) => {
ws.send(
JSON.stringify({
type: 'status',
data: {
status: {
exec_info: {
queue_remaining: queueSize
}
}
}
}
} as StatusWsMessage)
})
)
}
// Extract the width from the queue response
@@ -104,8 +107,8 @@ test.describe('Actionbar', { tag: '@ui' }, () => {
).toBe(1)
// Trigger a status update so auto-queue re-runs
await triggerStatus(1)
await triggerStatus(0)
triggerStatus(1)
triggerStatus(0)
// Ensure the queued width is the last queued value
expect(

View File

@@ -75,9 +75,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
]
await comfyPage.appMode.enterAppModeWithInputs(inputs)
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
timeout: 5000
})
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
// Scroll to bottom so the codec widget is at the clipping edge
const widgetList = comfyPage.appMode.linearWidgets
@@ -90,7 +88,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
await codecSelect.click()
const overlay = comfyPage.page.locator('.p-select-overlay').first()
await expect(overlay).toBeVisible({ timeout: 5000 })
await expect(overlay).toBeVisible()
await expect
.poll(() =>
@@ -123,9 +121,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
]
await comfyPage.appMode.enterAppModeWithInputs(inputs)
await expect(comfyPage.appMode.linearWidgets).toBeVisible({
timeout: 5000
})
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
// Scroll to bottom so the image widget is at the clipping edge
const widgetList = comfyPage.appMode.linearWidgets
@@ -144,7 +140,7 @@ test.describe('App mode dropdown clipping', { tag: '@ui' }, () => {
// The unstyled PrimeVue Popover renders with role="dialog".
// Locate the one containing the image grid (filter buttons like "All", "Inputs").
const popover = comfyPage.appMode.imagePickerPopover
await expect(popover).toBeVisible({ timeout: 5000 })
await expect(popover).toBeVisible()
await expect
.poll(() =>

View File

@@ -16,7 +16,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
await expect(comfyPage.appMode.welcome).toBeVisible()
await expect(comfyPage.appMode.emptyWorkflowText).toBeVisible()
await expect(comfyPage.appMode.buildAppButton).not.toBeVisible()
await expect(comfyPage.appMode.buildAppButton).toBeHidden()
})
test('Build app button is visible when no outputs selected', async ({
@@ -26,7 +26,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
await expect(comfyPage.appMode.welcome).toBeVisible()
await expect(comfyPage.appMode.buildAppButton).toBeVisible()
await expect(comfyPage.appMode.emptyWorkflowText).not.toBeVisible()
await expect(comfyPage.appMode.emptyWorkflowText).toBeHidden()
})
test('Empty workflow and build app are hidden when app has outputs', async ({
@@ -35,8 +35,8 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
await comfyPage.appMode.enterAppModeWithInputs([['3', 'seed']])
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
await expect(comfyPage.appMode.emptyWorkflowText).not.toBeVisible()
await expect(comfyPage.appMode.buildAppButton).not.toBeVisible()
await expect(comfyPage.appMode.emptyWorkflowText).toBeHidden()
await expect(comfyPage.appMode.buildAppButton).toBeHidden()
})
test('Back to workflow returns to graph mode', async ({ comfyPage }) => {
@@ -46,7 +46,7 @@ test.describe('App mode welcome states', { tag: '@ui' }, () => {
await comfyPage.appMode.backToWorkflowButton.click()
await expect(comfyPage.canvas).toBeVisible()
await expect(comfyPage.appMode.welcome).not.toBeVisible()
await expect(comfyPage.appMode.welcome).toBeHidden()
})
test('Load template opens template selector', async ({ comfyPage }) => {

View File

@@ -26,7 +26,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
await appMode.steps.goToInputs()
const menu = appMode.select.getInputItemMenu('seed')
await expect(menu).toBeVisible({ timeout: 5000 })
await expect(menu).toBeVisible()
await appMode.select.renameInputViaMenu('seed', 'Builder Input Seed')
// Verify in app mode after save/reload
@@ -34,7 +34,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
const workflowName = `${new Date().getTime()} builder-input-menu`
await saveAndReopenInAppMode(comfyPage, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(
appMode.linearWidgets.getByText('Builder Input Seed')
).toBeVisible()
@@ -54,7 +54,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
const workflowName = `${new Date().getTime()} builder-input-dblclick`
await saveAndReopenInAppMode(comfyPage, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(appMode.linearWidgets.getByText('Dblclick Seed')).toBeVisible()
})
@@ -65,7 +65,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
await appMode.steps.goToPreview()
const menu = appMode.select.getPreviewWidgetMenu('seed — New Subgraph')
await expect(menu).toBeVisible({ timeout: 5000 })
await expect(menu).toBeVisible()
await appMode.select.renameWidget(menu, 'Preview Seed')
// Verify in app mode after save/reload
@@ -73,7 +73,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
const workflowName = `${new Date().getTime()} builder-preview`
await saveAndReopenInAppMode(comfyPage, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(appMode.linearWidgets.getByText('Preview Seed')).toBeVisible()
})
@@ -85,7 +85,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
await appMode.footer.exitBuilder()
await appMode.toggleAppMode()
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
const menu = appMode.getAppModeWidgetMenu('seed')
await appMode.select.renameWidget(menu, 'App Mode Seed')
@@ -97,7 +97,7 @@ test.describe('App mode widget rename', { tag: ['@ui', '@subgraph'] }, () => {
const workflowName = `${new Date().getTime()} app-mode`
await saveAndReopenInAppMode(comfyPage, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(appMode.linearWidgets.getByText('App Mode Seed')).toBeVisible()
})
})

View File

@@ -63,7 +63,7 @@ test.describe('App mode widget values in prompt', { tag: '@ui' }, () => {
({ nodeId, widgetName }) => [nodeId, widgetName]
)
await appMode.enterAppModeWithInputs(inputs)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
for (const { nodeId, widgetName, type, fill } of WIDGET_TEST_DATA) {
const key = `${nodeId}:${widgetName}`

View File

@@ -11,7 +11,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
test('should open bottom panel via toggle button', async ({ comfyPage }) => {
const { bottomPanel } = comfyPage
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
await bottomPanel.toggleButton.click()
await expect(bottomPanel.root).toBeVisible()
})
@@ -35,7 +35,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
await expect(bottomPanel.root).toBeVisible()
await bottomPanel.toggleButton.click()
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
})
test('should switch between shortcuts and terminal panels', async ({
@@ -55,7 +55,7 @@ test.describe('Bottom Panel Logs', { tag: '@ui' }, () => {
await expect(logsTab).toBeVisible()
await expect(
comfyPage.page.locator('[id*="tab_shortcuts-essentials"]')
).not.toBeVisible()
).toBeHidden()
})
test('should persist Logs tab content in bottom panel', async ({

View File

@@ -10,11 +10,11 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
test('should toggle shortcuts panel visibility', async ({ comfyPage }) => {
const { bottomPanel } = comfyPage
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
await bottomPanel.keyboardShortcutsButton.click()
await expect(bottomPanel.root).toBeVisible()
await bottomPanel.keyboardShortcutsButton.click()
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
})
test('should display essentials shortcuts tab', async ({ comfyPage }) => {
@@ -182,7 +182,7 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
await expect(bottomPanel.root).toBeVisible()
await bottomPanel.keyboardShortcutsButton.click()
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
})
test('should display shortcuts in organized columns', async ({
@@ -192,9 +192,7 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
await bottomPanel.keyboardShortcutsButton.click()
await expect(
comfyPage.page.locator('[data-testid="shortcuts-columns"]')
).toBeVisible()
await expect(comfyPage.page.getByTestId('shortcuts-columns')).toBeVisible()
const subcategoryTitles = bottomPanel.shortcuts.subcategoryTitles
await expect.poll(() => subcategoryTitles.count()).toBeGreaterThanOrEqual(2)
@@ -205,7 +203,7 @@ test.describe('Bottom Panel Shortcuts', { tag: '@ui' }, () => {
}) => {
const { bottomPanel } = comfyPage
await expect(bottomPanel.root).not.toBeVisible()
await expect(bottomPanel.root).toBeHidden()
await comfyPage.page.keyboard.press('Control+Shift+KeyK')

View File

@@ -6,6 +6,7 @@ import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import type { AppModeHelper } from '@e2e/fixtures/helpers/AppModeHelper'
import {
builderSaveAs,
createAndSaveApp,
openWorkflowFromSidebar,
setupBuilder
} from '@e2e/helpers/builderTestUtils'
@@ -21,7 +22,7 @@ async function saveCloseAndReopenAsApp(
await appMode.steps.goToPreview()
await builderSaveAs(appMode, workflowName)
await appMode.saveAs.closeButton.click()
await expect(appMode.saveAs.successDialog).not.toBeVisible()
await expect(appMode.saveAs.successDialog).toBeHidden()
await appMode.footer.exitBuilder()
await openWorkflowFromSidebar(comfyPage, workflowName)
@@ -122,7 +123,7 @@ test.describe('Builder input reordering', { tag: '@ui' }, () => {
const workflowName = `${Date.now()} reorder-preview`
await saveCloseAndReopenAsApp(comfyPage, appMode, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(appMode.select.previewWidgetLabels).toHaveText([
'steps',
'cfg',
@@ -147,11 +148,58 @@ test.describe('Builder input reordering', { tag: '@ui' }, () => {
const workflowName = `${Date.now()} reorder-persist`
await saveCloseAndReopenAsApp(comfyPage, appMode, workflowName)
await expect(appMode.linearWidgets).toBeVisible({ timeout: 5000 })
await expect(appMode.linearWidgets).toBeVisible()
await expect(appMode.select.previewWidgetLabels).toHaveText([
'steps',
'cfg',
'seed'
])
})
test('Reordering inputs in one app does not corrupt another app', async ({
comfyPage
}) => {
const { appMode } = comfyPage
const suffix = String(Date.now())
const app1Name = `app1-${suffix}`
const app2Name = `app2-${suffix}`
const app2Widgets = ['seed', 'steps']
// Create and save app1 with [seed, steps, cfg]
await createAndSaveApp(comfyPage, app1Name, WIDGETS)
await appMode.footer.exitBuilder()
// Create app2 in a new tab so both apps are open simultaneously
await comfyPage.menu.topbar.triggerTopbarCommand(['New'])
await createAndSaveApp(comfyPage, app2Name, app2Widgets)
await appMode.footer.exitBuilder()
// Switch to app1 tab and enter builder
await comfyPage.menu.topbar.getWorkflowTab(app1Name).click()
await appMode.enterBuilder()
await appMode.steps.goToInputs()
await expect(appMode.select.inputItemTitles).toHaveText(WIDGETS)
// Reorder app1 inputs: drag 'seed' from first to last
await appMode.select.dragInputItem(0, 2)
const app1Reordered = ['steps', 'cfg', 'seed']
await expect(appMode.select.inputItemTitles).toHaveText(app1Reordered)
// Switch to app2 tab and enter builder
await appMode.footer.exitBuilder()
await comfyPage.menu.topbar.getWorkflowTab(app2Name).click()
await appMode.enterBuilder()
await appMode.steps.goToInputs()
// Verify app2 inputs are not corrupted — still [seed, steps]
await expect(appMode.select.inputItemTitles).toHaveText(app2Widgets)
// Switch back to app1 and verify reorder persisted
await appMode.footer.exitBuilder()
await comfyPage.menu.topbar.getWorkflowTab(app1Name).click()
await appMode.enterBuilder()
await appMode.steps.goToInputs()
await expect(appMode.select.inputItemTitles).toHaveText(app1Reordered)
})
})

View File

@@ -21,7 +21,7 @@ async function reSaveAs(
viewType: 'App' | 'Node graph'
) {
await appMode.footer.openSaveAsFromChevron()
await expect(appMode.saveAs.nameInput).toBeVisible({ timeout: 5000 })
await expect(appMode.saveAs.nameInput).toBeVisible()
await appMode.saveAs.fillAndSave(workflowName, viewType)
}
@@ -31,7 +31,7 @@ async function dismissSuccessDialog(
) {
const btn = button === 'close' ? saveAs.closeButton : saveAs.dismissButton
await btn.click()
await expect(saveAs.successDialog).not.toBeVisible()
await expect(saveAs.successDialog).toBeHidden()
}
test.describe('Builder save flow', { tag: ['@ui'] }, () => {
@@ -48,7 +48,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await setupBuilder(comfyPage)
await comfyPage.appMode.footer.saveAsButton.click()
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
await expect(saveAs.dialog).toBeVisible()
await expect(saveAs.nameInput).toBeVisible()
await expect(saveAs.title).toBeVisible()
await expect(saveAs.radioGroup).toBeVisible()
@@ -68,7 +68,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await setupBuilder(comfyPage)
await comfyPage.appMode.footer.saveAsButton.click()
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
await expect(saveAs.dialog).toBeVisible()
await saveAs.nameInput.fill('')
await expect(saveAs.saveButton).toBeDisabled()
})
@@ -78,7 +78,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await setupBuilder(comfyPage)
await comfyPage.appMode.footer.saveAsButton.click()
await expect(saveAs.dialog).toBeVisible({ timeout: 5000 })
await expect(saveAs.dialog).toBeVisible()
const appRadio = saveAs.viewTypeRadio('App')
await expect(appRadio).toHaveAttribute('aria-checked', 'true')
@@ -113,7 +113,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await comfyPage.page.keyboard.press('Escape')
await comfyPage.nextFrame()
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
})
test('Exit builder button exits builder mode', async ({ comfyPage }) => {
@@ -121,7 +121,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await expect(comfyPage.appMode.steps.toolbar).toBeVisible()
await comfyPage.appMode.footer.exitBuilder()
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
})
test('Save button directly saves for previously saved workflow', async ({
@@ -136,12 +136,12 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
// Modify the workflow so the save button becomes enabled
await comfyPage.appMode.steps.goToInputs()
await comfyPage.appMode.select.deleteInput('seed')
await expect(footer.saveButton).toBeEnabled({ timeout: 5000 })
await expect(footer.saveButton).toBeEnabled()
await footer.saveButton.click()
await comfyPage.nextFrame()
await expect(saveAs.dialog).not.toBeVisible({ timeout: 2000 })
await expect(saveAs.dialog).toBeHidden()
await expect(footer.saveButton).toBeDisabled()
})
@@ -156,7 +156,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await footer.openSaveAsFromChevron()
await expect(saveAs.title).toBeVisible({ timeout: 5000 })
await expect(saveAs.title).toBeVisible()
await expect(saveAs.nameInput).toBeVisible()
})
@@ -209,7 +209,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await expect(
comfyPage.page.getByText('Connect an output', { exact: false })
).toBeVisible({ timeout: 5000 })
).toBeVisible()
})
test('save as app produces correct extension and linearMode', async ({
@@ -253,7 +253,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await builderSaveAs(comfyPage.appMode, `${Date.now()} app-view`, 'App')
await comfyPage.appMode.saveAs.viewAppButton.click()
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
await expect(comfyPage.appMode.saveAs.successDialog).toBeHidden()
await expect
.poll(() => comfyPage.workflow.getActiveWorkflowActiveAppMode())
@@ -271,9 +271,9 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
)
await comfyPage.appMode.saveAs.exitBuilderButton.click()
await expect(comfyPage.appMode.saveAs.successDialog).not.toBeVisible()
await expect(comfyPage.appMode.saveAs.successDialog).toBeHidden()
await expect(comfyPage.appMode.steps.toolbar).not.toBeVisible()
await expect(comfyPage.appMode.steps.toolbar).toBeHidden()
})
test('save as with different mode does not modify the original workflow', async ({
@@ -291,7 +291,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
// Re-save as node graph — creates a copy
await reSaveAs(appMode, `${Date.now()} copy`, 'Node graph')
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
await expect(appMode.saveAs.successMessage).toBeVisible()
await expect
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
@@ -325,11 +325,11 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await reSaveAs(appMode, name, 'App')
await expect(appMode.saveAs.overwriteDialog).toBeVisible({ timeout: 5000 })
await expect(appMode.saveAs.overwriteDialog).toBeVisible()
await appMode.saveAs.overwriteButton.click()
await expect(appMode.saveAs.overwriteDialog).not.toBeVisible()
await expect(appMode.saveAs.overwriteDialog).toBeHidden()
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
await expect(appMode.saveAs.successMessage).toBeVisible()
await expect
.poll(() => comfyPage.workflow.getActiveWorkflowPath())
@@ -351,7 +351,7 @@ test.describe('Builder save flow', { tag: ['@ui'] }, () => {
await dismissSuccessDialog(appMode.saveAs)
await reSaveAs(appMode, name, 'Node graph')
await expect(appMode.saveAs.successMessage).toBeVisible({ timeout: 5000 })
await expect(appMode.saveAs.successMessage).toBeVisible()
await expect
.poll(() => comfyPage.workflow.getActiveWorkflowPath())

View File

@@ -70,7 +70,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
await expect(menu).toBeVisible()
await trigger.click()
await comfyPage.nextFrame()
await expect(menu).not.toBeVisible()
await expect(menu).toBeHidden()
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
})
@@ -81,7 +81,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
await expect(menu).toBeVisible()
await handItem.click()
await comfyPage.nextFrame()
await expect(menu).not.toBeVisible()
await expect(menu).toBeHidden()
})
test('closes when Escape is pressed', async ({ comfyPage }) => {
@@ -91,7 +91,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
await expect(menu).toBeVisible()
await selectItem.press('Escape')
await comfyPage.nextFrame()
await expect(menu).not.toBeVisible()
await expect(menu).toBeHidden()
await expect(trigger).toHaveAttribute('aria-expanded', 'false')
})
})
@@ -197,7 +197,7 @@ test.describe('CanvasModeSelector', { tag: '@canvas' }, () => {
await selectItem.press('ArrowDown')
await handItem.press('Escape')
await comfyPage.nextFrame()
await expect(menu).not.toBeVisible()
await expect(menu).toBeHidden()
await expect(trigger).toBeFocused()
})
})

View File

@@ -206,8 +206,8 @@ test.describe('Change Tracker', { tag: '@workflow' }, () => {
// Ensure undo reverts both changes
await comfyPage.keyboard.undo()
await expect(node).not.toBeBypassed({ timeout: 5000 })
await expect(node).not.toBeCollapsed({ timeout: 5000 })
await expect(node).not.toBeBypassed()
await expect(node).not.toBeCollapsed()
await waitForChangeTrackerSettled(comfyPage, {
isModified: false,
redoQueueSize: 1,

View File

@@ -75,15 +75,18 @@ test.describe('Asset-supported node default value', { tag: '@cloud' }, () => {
})
await expect
.poll(async () => {
return await comfyPage.page.evaluate((id) => {
const node = window.app!.graph.getNodeById(id)
const widget = node?.widgets?.find(
(w: { name: string }) => w.name === 'ckpt_name'
)
return String(widget?.value ?? '')
}, nodeId)
})
.poll(
async () => {
return await comfyPage.page.evaluate((id) => {
const node = window.app!.graph.getNodeById(id)
const widget = node?.widgets?.find(
(w: { name: string }) => w.name === 'ckpt_name'
)
return String(widget?.value ?? '')
}, nodeId)
},
{ timeout: 10_000 }
)
.toBe(CLOUD_ASSETS[0].name)
})
})

View File

@@ -149,11 +149,7 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
await comfyPage.canvas.click({ position: { x: 50, y: 500 } })
await comfyPage.nextFrame()
await comfyPage.clipboard.paste()
await expect
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
timeout: 5_000
})
.toBe(3)
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(3)
// Step 2: Paste image onto selected LoadImage node
const loadImageNodes =
@@ -171,13 +167,10 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
await uploadPromise
await expect
.poll(
async () => {
const fileWidget = await loadImageNodes[0].getWidget(0)
return fileWidget.getValue()
},
{ timeout: 5_000 }
)
.poll(async () => {
const fileWidget = await loadImageNodes[0].getWidget(0)
return fileWidget.getValue()
})
.toContain('image32x32')
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(3)
@@ -194,11 +187,7 @@ test.describe('Copy Paste', { tag: ['@screenshot', '@workflow'] }, () => {
)
await uploadPromise2
await expect
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
timeout: 5_000
})
.toBe(4)
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(4)
const allLoadImageNodes =
await comfyPage.nodeOps.getNodeRefsByType('LoadImage')
expect(allLoadImageNodes).toHaveLength(2)

View File

@@ -38,13 +38,13 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
`.${tabId}-tab-button.side-bar-button-selected`
)
await expect(selectedButton).not.toBeVisible()
await expect(selectedButton).toBeHidden()
await comfyPage.canvas.press(key)
await expect(selectedButton).toBeVisible()
await comfyPage.canvas.press(key)
await expect(selectedButton).not.toBeVisible()
await expect(selectedButton).toBeHidden()
})
}
})
@@ -172,7 +172,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
// Toggle off with Alt+m
await comfyPage.page.keyboard.press('Alt+KeyM')
await expect(comfyPage.appMode.linearWidgets).not.toBeVisible()
await expect(comfyPage.appMode.linearWidgets).toBeHidden()
// Toggle on again
await comfyPage.page.keyboard.press('Alt+KeyM')
@@ -189,7 +189,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
await expect(minimap).toBeVisible()
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
await expect(minimap).not.toBeVisible()
await expect(minimap).toBeHidden()
await comfyPage.page.keyboard.press('Alt+Shift+KeyM')
await expect(minimap).toBeVisible()
@@ -198,13 +198,13 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
test("'Ctrl+`' toggles terminal/logs panel", async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
await expect(comfyPage.bottomPanel.root).toBeHidden()
await comfyPage.page.keyboard.press('Control+Backquote')
await expect(comfyPage.bottomPanel.root).toBeVisible()
await comfyPage.page.keyboard.press('Control+Backquote')
await expect(comfyPage.bottomPanel.root).not.toBeVisible()
await expect(comfyPage.bottomPanel.root).toBeHidden()
})
})
@@ -250,7 +250,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
// The Save As dialog should appear (p-dialog overlay)
const dialogOverlay = comfyPage.page.locator('.p-dialog-mask')
await expect(dialogOverlay).toBeVisible({ timeout: 3000 })
await expect(dialogOverlay).toBeVisible()
// Dismiss the dialog
await comfyPage.page.keyboard.press('Escape')
@@ -303,9 +303,7 @@ test.describe('Default Keybindings', { tag: '@keyboard' }, () => {
// After conversion, node count should decrease
// (multiple nodes replaced by single subgraph node)
await expect
.poll(() => comfyPage.nodeOps.getGraphNodesCount(), {
timeout: 5000
})
.poll(() => comfyPage.nodeOps.getGraphNodesCount())
.toBeLessThan(initialCount)
})

View File

@@ -11,9 +11,7 @@ test.beforeEach(async ({ comfyPage }) => {
test.describe('Settings', () => {
test('@mobile Should be visible on mobile', async ({ comfyPage }) => {
await comfyPage.page.keyboard.press('Control+,')
const settingsDialog = comfyPage.page.locator(
'[data-testid="settings-dialog"]'
)
const settingsDialog = comfyPage.page.getByTestId('settings-dialog')
await expect(settingsDialog).toBeVisible()
const contentArea = settingsDialog.locator('main')
await expect(contentArea).toBeVisible()
@@ -26,17 +24,16 @@ test.describe('Settings', () => {
await comfyPage.page.keyboard.down('ControlOrMeta')
await comfyPage.page.keyboard.press(',')
await comfyPage.page.keyboard.up('ControlOrMeta')
const settingsLocator = comfyPage.page.locator(
'[data-testid="settings-dialog"]'
)
const settingsLocator = comfyPage.page.getByTestId('settings-dialog')
await expect(settingsLocator).toBeVisible()
await comfyPage.page.keyboard.press('Escape')
await expect(settingsLocator).not.toBeVisible()
await expect(settingsLocator).toBeHidden()
})
test('Can change canvas zoom speed setting', async ({ comfyPage }) => {
const maxSpeed = 2.5
await comfyPage.settings.setSetting('Comfy.Graph.ZoomSpeed', maxSpeed)
await test.step('Setting should persist', async () => {
await expect
.poll(() => comfyPage.settings.getSetting('Comfy.Graph.ZoomSpeed'))
@@ -49,9 +46,7 @@ test.describe('Settings', () => {
await comfyPage.page.keyboard.press('Control+,')
// Open the keybinding tab
const settingsDialog = comfyPage.page.locator(
'[data-testid="settings-dialog"]'
)
const settingsDialog = comfyPage.page.getByTestId('settings-dialog')
await expect(settingsDialog).toBeVisible()
await settingsDialog
.locator('nav [role="button"]', { hasText: 'Keybinding' })

View File

@@ -307,7 +307,7 @@ test.describe('ManagerDialog', { tag: '@ui' }, () => {
await searchInput.fill('Test Pack B')
await expect(dialog.getByText('Test Pack B')).toBeVisible()
await expect(dialog.getByText('Test Pack A')).not.toBeVisible()
await expect(dialog.getByText('Test Pack A')).toBeHidden()
})
test('Clicking a pack card opens the info panel', async ({ comfyPage }) => {
@@ -360,7 +360,7 @@ test.describe('ManagerDialog', { tag: '@ui' }, () => {
await expect(dialog).toBeVisible()
await comfyPage.page.keyboard.press('Escape')
await expect(dialog).not.toBeVisible()
await expect(dialog).toBeHidden()
})
test('Empty search shows no results message', async ({ comfyPage }) => {

View File

@@ -60,7 +60,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
})
await dialog.getByRole('button', { name: 'Cancel' }).click()
await expect(dialog).not.toBeVisible()
await expect(dialog).toBeHidden()
expect(clearCalled).toBe(false)
await comfyPage.page.unroute('**/api/history')
@@ -83,7 +83,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
})
await dialog.getByLabel('Close').click()
await expect(dialog).not.toBeVisible()
await expect(dialog).toBeHidden()
expect(clearCalled).toBe(false)
await comfyPage.page.unroute('**/api/history')
@@ -106,7 +106,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
const request = await clearPromise
expect(request.postDataJSON()).toEqual({ clear: true })
await expect(dialog).not.toBeVisible()
await expect(dialog).toBeHidden()
})
test('Dialog state resets after close and reopen', async ({ comfyPage }) => {
@@ -114,7 +114,7 @@ test.describe('Queue Clear History Dialog', { tag: '@ui' }, () => {
const dialog = comfyPage.confirmDialog.root
await expect(dialog).toBeVisible()
await dialog.getByRole('button', { name: 'Cancel' }).click()
await expect(dialog).not.toBeVisible()
await expect(dialog).toBeHidden()
await comfyPage.queuePanel.openClearHistoryDialog()
await expect(dialog).toBeVisible()

View File

@@ -61,7 +61,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
await expect(dialog.root).toBeVisible()
await dialog.close()
await expect(dialog.root).not.toBeVisible()
await expect(dialog.root).toBeHidden()
})
test('Escape key closes dialog', async ({ comfyPage }) => {
@@ -70,7 +70,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
await expect(dialog.root).toBeVisible()
await comfyPage.page.keyboard.press('Escape')
await expect(dialog.root).not.toBeVisible()
await expect(dialog.root).toBeHidden()
})
test('Search filters settings list', async ({ comfyPage }) => {
@@ -153,7 +153,7 @@ test.describe('Settings dialog', { tag: '@ui' }, () => {
const expanded = await select.getAttribute('aria-expanded')
if (expanded !== 'true') await select.click()
await expect(select).toHaveAttribute('aria-expanded', 'true')
}).toPass({ timeout: 3000 })
}).toPass({ timeout: 5000 })
// Pick the option that is not the current value
const targetValue = initialValue === 'Top' ? 'Disabled' : 'Top'

View File

@@ -10,7 +10,7 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
test('Collapsed multiline textarea is not visible', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('widgets/collapsed_multiline')
const textareaWidget = comfyPage.page.locator('.comfy-multiline-input')
await expect(textareaWidget).not.toBeVisible()
await expect(textareaWidget).toBeHidden()
})
test('Multiline textarea correctly collapses', async ({ comfyPage }) => {
@@ -25,8 +25,8 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
for (const node of nodes) {
await node.click('collapse')
}
await expect(firstMultiline).not.toBeVisible()
await expect(lastMultiline).not.toBeVisible()
await expect(firstMultiline).toBeHidden()
await expect(lastMultiline).toBeHidden()
})
test(
@@ -35,7 +35,7 @@ test.describe('DOM Widget', { tag: '@widget' }, () => {
async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
await expect(comfyPage.canvas).toHaveScreenshot('focus-mode-on.png')
}
)

View File

@@ -74,7 +74,7 @@ test.describe('Error dialog', () => {
}) => {
const errorDialog = await triggerConfigureError(comfyPage)
await expect(errorDialog).toBeVisible()
await expect(errorDialog.locator('pre')).not.toBeVisible()
await expect(errorDialog.locator('pre')).toBeHidden()
await errorDialog.getByTestId(TestIds.dialogs.errorDialogShowReport).click()
@@ -83,7 +83,7 @@ test.describe('Error dialog', () => {
await expect(reportPre).toHaveText(/\S/)
await expect(
errorDialog.getByTestId(TestIds.dialogs.errorDialogShowReport)
).not.toBeVisible()
).toBeHidden()
})
test('Should copy report to clipboard when "Copy to Clipboard" is clicked', async ({

View File

@@ -100,7 +100,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await errorOverlay
.getByTestId(TestIds.dialogs.errorOverlayDismiss)
.click()
await expect(errorOverlay).not.toBeVisible()
await expect(errorOverlay).toBeHidden()
await comfyPage.canvas.click()
await comfyPage.nextFrame()
@@ -112,10 +112,10 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await comfyPage.nextFrame()
await comfyPage.keyboard.undo()
await expect(errorOverlay).not.toBeVisible({ timeout: 5000 })
await expect(errorOverlay).toBeHidden()
await comfyPage.keyboard.redo()
await expect(errorOverlay).not.toBeVisible({ timeout: 5000 })
await expect(errorOverlay).toBeHidden()
})
})
@@ -156,7 +156,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await overlay.getByTestId(TestIds.dialogs.errorOverlaySeeErrors).click()
await expect(overlay).not.toBeVisible()
await expect(overlay).toBeHidden()
await expect(comfyPage.page.getByTestId('properties-panel')).toBeVisible()
})
@@ -168,7 +168,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await overlay.getByTestId(TestIds.dialogs.errorOverlaySeeErrors).click()
await expect(overlay).not.toBeVisible()
await expect(overlay).toBeHidden()
})
test('"Dismiss" closes overlay without opening panel', async ({
@@ -181,10 +181,8 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await overlay.getByTestId(TestIds.dialogs.errorOverlayDismiss).click()
await expect(overlay).not.toBeVisible()
await expect(
comfyPage.page.getByTestId('properties-panel')
).not.toBeVisible()
await expect(overlay).toBeHidden()
await expect(comfyPage.page.getByTestId('properties-panel')).toBeHidden()
})
test('Close button (X) dismisses overlay', async ({ comfyPage }) => {
@@ -195,7 +193,7 @@ test.describe('Error overlay', { tag: '@ui' }, () => {
await overlay.getByRole('button', { name: /close/i }).click()
await expect(overlay).not.toBeVisible()
await expect(overlay).toBeHidden()
})
})
})

View File

@@ -63,19 +63,13 @@ test.describe(
await comfyPage.command.executeCommand('Comfy.QueueSelectedOutputNodes')
await expect
.poll(async () => (await input.getWidget(0)).getValue(), {
timeout: 2_000
})
.poll(async () => (await input.getWidget(0)).getValue())
.toBe('foo')
await expect
.poll(async () => (await output1.getWidget(0)).getValue(), {
timeout: 2_000
})
.poll(async () => (await output1.getWidget(0)).getValue())
.toBe('foo')
await expect
.poll(async () => (await output2.getWidget(0)).getValue(), {
timeout: 2_000
})
.poll(async () => (await output2.getWidget(0)).getValue())
.toBe('')
})
}

View File

@@ -69,7 +69,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
expect(flags?.data).not.toBeNull()
expect(flags?.data).toHaveProperty('supports_preview_metadata')
expect(typeof flags?.data?.supports_preview_metadata).toBe('boolean')
}).toPass()
}).toPass({ timeout: 5000 })
// Verify server sent feature flags back
await expect(async () => {
@@ -82,7 +82,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
expect(flags).toHaveProperty('max_upload_size')
expect(typeof flags?.max_upload_size).toBe('number')
expect(Object.keys(flags ?? {}).length).toBeGreaterThan(0)
}).toPass()
}).toPass({ timeout: 5000 })
await newPage.close()
})
@@ -102,7 +102,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
expect(typeof flags.supports_preview_metadata).toBe('boolean')
expect(flags).toHaveProperty('max_upload_size')
expect(typeof flags.max_upload_size).toBe('number')
}).toPass()
}).toPass({ timeout: 5000 })
})
test('serverSupportsFeature method works with real backend flags', async ({
@@ -182,7 +182,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
)
expect(typeof maxUpload).toBe('number')
expect(maxUpload as number).toBeGreaterThan(0)
}).toPass()
}).toPass({ timeout: 5000 })
// Test getServerFeature with default value for non-existent feature
await expect
@@ -210,7 +210,7 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
expect(typeof features.supports_preview_metadata).toBe('boolean')
expect(features).toHaveProperty('max_upload_size')
expect(Object.keys(features).length).toBeGreaterThan(0)
}).toPass()
}).toPass({ timeout: 5000 })
})
test('Client feature flags are immutable', async ({ comfyPage }) => {
@@ -348,14 +348,14 @@ test.describe('Feature Flags', { tag: ['@slow', '@settings'] }, () => {
expect(flags).toHaveProperty('supports_preview_metadata')
expect(typeof flags?.supports_preview_metadata).toBe('boolean')
expect(flags).toHaveProperty('max_upload_size')
}).toPass()
}).toPass({ timeout: 5000 })
// Verify feature flags were received and API was initialized
await expect(async () => {
const readiness = await newPage.evaluate(() => window.__appReadiness)
expect(readiness?.featureFlagsReceived).toBe(true)
expect(readiness?.apiInitialized).toBe(true)
}).toPass()
}).toPass({ timeout: 5000 })
await newPage.close()
})

View File

@@ -14,12 +14,12 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
await comfyPage.setFocusMode(true)
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
})
test('Focus mode restores UI chrome', async ({ comfyPage }) => {
await comfyPage.setFocusMode(true)
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
await comfyPage.setFocusMode(false)
await expect(comfyPage.menu.sideToolbar).toBeVisible()
@@ -29,7 +29,7 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
await expect(comfyPage.menu.sideToolbar).toBeVisible()
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
await comfyPage.command.executeCommand('Workspace.ToggleFocusMode')
await expect(comfyPage.menu.sideToolbar).toBeVisible()
@@ -41,7 +41,7 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
await comfyPage.setFocusMode(true)
await expect(topMenu).not.toBeVisible()
await expect(topMenu).toBeHidden()
})
test('Canvas remains visible in focus mode', async ({ comfyPage }) => {
@@ -52,12 +52,12 @@ test.describe('Focus Mode', { tag: '@ui' }, () => {
test('Focus mode can be toggled multiple times', async ({ comfyPage }) => {
await comfyPage.setFocusMode(true)
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
await comfyPage.setFocusMode(false)
await expect(comfyPage.menu.sideToolbar).toBeVisible()
await comfyPage.setFocusMode(true)
await expect(comfyPage.menu.sideToolbar).not.toBeVisible()
await expect(comfyPage.menu.sideToolbar).toBeHidden()
})
})

View File

@@ -109,6 +109,6 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
expect(r.switchOutputLinkIds).toEqual(
expect.arrayContaining([r.cfg85LinkId, r.cfg86LinkId])
)
}).toPass()
}).toPass({ timeout: 5000 })
})
})

View File

@@ -94,6 +94,6 @@ test.describe('Graph Canvas Menu', { tag: ['@screenshot', '@canvas'] }, () => {
await backdrop.click()
// Modal should be hidden
await expect(zoomModal).not.toBeVisible()
await expect(zoomModal).toBeHidden()
})
})

View File

@@ -88,7 +88,10 @@ test.describe('Group Node', { tag: '@node' }, () => {
.getNode(groupNodeName)
.locator('.bookmark-button')
.click()
await comfyPage.page.hover('.p-tree-node-label.tree-explorer-node-label')
await comfyPage.page
.locator('.p-tree-node-label.tree-explorer-node-label')
.first()
.hover()
await expect(
comfyPage.page.locator('.node-lib-node-preview')
).toBeVisible()
@@ -99,6 +102,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
.click()
})
})
test(
'Can be added to canvas using search',
{ tag: '@screenshot' },
@@ -154,7 +158,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
await comfyPage.nextFrame()
await expect(manage1.selectedNodeTypeSelect).toHaveValue('g1')
await manage1.close()
await expect(manage1.root).not.toBeVisible()
await expect(manage1.root).toBeHidden()
const manage2 = await group2.manageGroupNode()
await expect(manage2.selectedNodeTypeSelect).toHaveValue('g2')
@@ -241,7 +245,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(1)
await expect(
comfyPage.page.getByTestId(TestIds.dialogs.errorOverlay)
).not.toBeVisible()
).toBeHidden()
})
test.describe('Copy and paste', () => {
@@ -349,6 +353,7 @@ test.describe('Group Node', { tag: '@node' }, () => {
await comfyPage.page.keyboard.press('Alt+g')
await expect(comfyPage.toast.visibleToasts).toHaveCount(1)
})
test('Convert to group node, selected 1 node', async ({ comfyPage }) => {
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
await comfyPage.canvas.click({

View File

@@ -1,9 +1,10 @@
import type { Locator, Page } from '@playwright/test'
import { expect } from '@playwright/test'
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
test.describe('Image Compare', () => {
test.describe('Image Compare', { tag: '@widget' }, () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.workflow.loadWorkflow('widgets/image_compare_widget')
@@ -21,7 +22,12 @@ test.describe('Image Compare', () => {
async function setImageCompareValue(
comfyPage: ComfyPage,
value: { beforeImages: string[]; afterImages: string[] }
value: {
beforeImages: string[]
afterImages: string[]
beforeAlt?: string
afterAlt?: string
}
) {
await comfyPage.page.evaluate(
({ value }) => {
@@ -37,6 +43,48 @@ test.describe('Image Compare', () => {
await comfyPage.nextFrame()
}
async function moveToPercentage(
page: Page,
containerLocator: Locator,
percentage: number
) {
const box = await containerLocator.boundingBox()
if (!box) throw new Error('Container not found')
await page.mouse.move(
box.x + box.width * (percentage / 100),
box.y + box.height / 2
)
}
async function waitForImagesLoaded(node: Locator) {
await expect
.poll(() =>
node.evaluate((el) => {
const imgs = el.querySelectorAll('img')
return (
imgs.length > 0 &&
Array.from(imgs).every(
(img) => img.complete && img.naturalWidth > 0
)
)
})
)
.toBe(true)
}
async function getClipPathInsetRightPercent(imgLocator: Locator) {
return imgLocator.evaluate((el) => {
// Accessing raw style avoids cross-browser getComputedStyle normalization issues
// Format is uniformly "inset(0 60% 0 0)" per Vue runtime inline style bindings
const parts = (el as HTMLElement).style.clipPath.split(' ')
return parts.length > 1 ? parseFloat(parts[1]) : -1
})
}
// ---------------------------------------------------------------------------
// Rendering
// ---------------------------------------------------------------------------
test(
'Shows empty state when no images are set',
{ tag: '@smoke' },
@@ -46,10 +94,14 @@ test.describe('Image Compare', () => {
await expect(node).toContainText('No images to compare')
await expect(node.locator('img')).toHaveCount(0)
await expect(node.locator('[role="presentation"]')).toHaveCount(0)
await expect(node.getByRole('presentation')).toHaveCount(0)
}
)
// ---------------------------------------------------------------------------
// Slider defaults
// ---------------------------------------------------------------------------
test(
'Slider defaults to 50% with both images set',
{ tag: ['@smoke', '@screenshot'] },
@@ -67,15 +119,444 @@ test.describe('Image Compare', () => {
await expect(beforeImg).toBeVisible()
await expect(afterImg).toBeVisible()
const handle = node.locator('[role="presentation"]')
const handle = node.getByRole('presentation')
await expect(handle).toBeVisible()
expect(
await handle.evaluate((el) => (el as HTMLElement).style.left)
await handle.evaluate((el) => (el as HTMLElement).style.left),
'Slider should default to 50% before screenshot'
).toBe('50%')
await expect(beforeImg).toHaveCSS('clip-path', /50%/)
await expect
.poll(() => getClipPathInsetRightPercent(beforeImg))
.toBeCloseTo(50, 0)
await waitForImagesLoaded(node)
await comfyPage.page.mouse.move(0, 0)
await expect(node).toHaveScreenshot('image-compare-default-50.png')
}
)
// ---------------------------------------------------------------------------
// Slider interaction
// ---------------------------------------------------------------------------
test(
'Mouse hover moves slider position',
{ tag: '@smoke' },
async ({ comfyPage }) => {
const beforeUrl = createTestImageDataUrl('Before', '#c00')
const afterUrl = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [beforeUrl],
afterImages: [afterUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const handle = node.getByRole('presentation')
const beforeImg = node.locator('img[alt="Before image"]')
const afterImg = node.locator('img[alt="After image"]')
await expect(afterImg).toBeVisible()
// Left edge: sliderPosition ≈ 5 → clip-path inset right ≈ 95%
await moveToPercentage(comfyPage.page, afterImg, 5)
await expect
.poll(() => getClipPathInsetRightPercent(beforeImg))
.toBeGreaterThan(90)
await expect
.poll(() =>
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
)
.toBeLessThan(10)
// Right edge: sliderPosition ≈ 95 → clip-path inset right ≈ 5%
await moveToPercentage(comfyPage.page, afterImg, 95)
await expect
.poll(() => getClipPathInsetRightPercent(beforeImg))
.toBeLessThan(10)
await expect
.poll(() =>
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
)
.toBeGreaterThan(90)
}
)
test('Slider preserves last position when mouse leaves widget', async ({
comfyPage
}) => {
const beforeUrl = createTestImageDataUrl('Before', '#c00')
const afterUrl = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [beforeUrl],
afterImages: [afterUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const handle = node.getByRole('presentation')
const afterImg = node.locator('img[alt="After image"]')
await expect(afterImg).toBeVisible()
await moveToPercentage(comfyPage.page, afterImg, 30)
// Wait for Vue to commit the slider update
await expect
.poll(() =>
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
)
.toBeCloseTo(30, 0)
const positionWhileInside = parseFloat(
await handle.evaluate((el) => (el as HTMLElement).style.left)
)
await comfyPage.page.mouse.move(0, 0)
// Position must not reset to default 50%
await expect
.poll(() =>
handle.evaluate((el) => parseFloat((el as HTMLElement).style.left))
)
.toBeCloseTo(positionWhileInside, 0)
})
test('Slider clamps to 0% at left edge of container', async ({
comfyPage
}) => {
const beforeUrl = createTestImageDataUrl('Before', '#c00')
const afterUrl = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [beforeUrl],
afterImages: [afterUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const handle = node.getByRole('presentation')
const afterImg = node.locator('img[alt="After image"]')
await expect(afterImg).toBeVisible()
const box = await afterImg.boundingBox()
if (!box) throw new Error('Container not found')
// Move to the leftmost pixel (elementX = 0 → sliderPosition = 0)
await comfyPage.page.mouse.move(box.x, box.y + box.height / 2)
await expect
.poll(() => handle.evaluate((el) => (el as HTMLElement).style.left))
.toBe('0%')
})
// ---------------------------------------------------------------------------
// Single image modes
// ---------------------------------------------------------------------------
test('Only before image shows without slider when afterImages is empty', async ({
comfyPage
}) => {
const url = createTestImageDataUrl('Before', '#c00')
await setImageCompareValue(comfyPage, {
beforeImages: [url],
afterImages: []
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img')).toHaveCount(1)
await expect(node.getByRole('presentation')).toBeHidden()
})
test('Only after image shows without slider when beforeImages is empty', async ({
comfyPage
}) => {
const url = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [],
afterImages: [url]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img')).toHaveCount(1)
await expect(node.getByRole('presentation')).toBeHidden()
})
// ---------------------------------------------------------------------------
// Batch navigation
// ---------------------------------------------------------------------------
test(
'Batch navigation appears when before side has multiple images',
{ tag: '@smoke' },
async ({ comfyPage }) => {
const url1 = createTestImageDataUrl('A1', '#c00')
const url2 = createTestImageDataUrl('A2', '#0c0')
const url3 = createTestImageDataUrl('A3', '#00c')
const afterUrl = createTestImageDataUrl('B1', '#888')
await setImageCompareValue(comfyPage, {
beforeImages: [url1, url2, url3],
afterImages: [afterUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const beforeBatch = node.getByTestId('before-batch')
await expect(node.getByTestId('batch-nav')).toBeVisible()
await expect(beforeBatch.getByTestId('batch-counter')).toHaveText('1 / 3')
// after-batch renders only when afterBatchCount > 1
await expect(node.getByTestId('after-batch')).toBeHidden()
await expect(beforeBatch.getByTestId('batch-prev')).toBeDisabled()
}
)
test('Batch navigation is hidden when both sides have single images', async ({
comfyPage
}) => {
const url = createTestImageDataUrl('Image', '#c00')
await setImageCompareValue(comfyPage, {
beforeImages: [url],
afterImages: [url]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.getByTestId('batch-nav')).toBeHidden()
})
test(
'Navigate forward through before images',
{ tag: '@smoke' },
async ({ comfyPage }) => {
const url1 = createTestImageDataUrl('A1', '#c00')
const url2 = createTestImageDataUrl('A2', '#0c0')
const url3 = createTestImageDataUrl('A3', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [url1, url2, url3],
afterImages: [createTestImageDataUrl('B1', '#888')]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const beforeBatch = node.getByTestId('before-batch')
const counter = beforeBatch.getByTestId('batch-counter')
const nextBtn = beforeBatch.getByTestId('batch-next')
const prevBtn = beforeBatch.getByTestId('batch-prev')
await nextBtn.click()
await expect(counter).toHaveText('2 / 3')
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
'src',
url2
)
await expect(prevBtn).toBeEnabled()
await nextBtn.click()
await expect(counter).toHaveText('3 / 3')
await expect(nextBtn).toBeDisabled()
}
)
test('Navigate backward through before images', async ({ comfyPage }) => {
const url1 = createTestImageDataUrl('A1', '#c00')
const url2 = createTestImageDataUrl('A2', '#0c0')
const url3 = createTestImageDataUrl('A3', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [url1, url2, url3],
afterImages: [createTestImageDataUrl('B1', '#888')]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const beforeBatch = node.getByTestId('before-batch')
const counter = beforeBatch.getByTestId('batch-counter')
const nextBtn = beforeBatch.getByTestId('batch-next')
const prevBtn = beforeBatch.getByTestId('batch-prev')
await nextBtn.click()
await nextBtn.click()
await expect(counter).toHaveText('3 / 3')
await prevBtn.click()
await expect(counter).toHaveText('2 / 3')
await expect(prevBtn).toBeEnabled()
await expect(nextBtn).toBeEnabled()
})
test('Before and after batch navigation are independent', async ({
comfyPage
}) => {
const url1 = createTestImageDataUrl('A1', '#c00')
const url2 = createTestImageDataUrl('A2', '#0c0')
const url3 = createTestImageDataUrl('A3', '#00c')
const urlA = createTestImageDataUrl('B1', '#880')
const urlB = createTestImageDataUrl('B2', '#008')
await setImageCompareValue(comfyPage, {
beforeImages: [url1, url2, url3],
afterImages: [urlA, urlB]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const beforeBatch = node.getByTestId('before-batch')
const afterBatch = node.getByTestId('after-batch')
await beforeBatch.getByTestId('batch-next').click()
await afterBatch.getByTestId('batch-next').click()
await expect(beforeBatch.getByTestId('batch-counter')).toHaveText('2 / 3')
await expect(afterBatch.getByTestId('batch-counter')).toHaveText('2 / 2')
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
'src',
url2
)
await expect(node.locator('img[alt="After image"]')).toHaveAttribute(
'src',
urlB
)
})
// ---------------------------------------------------------------------------
// Visual regression screenshots
// ---------------------------------------------------------------------------
for (const { pct, expectedClipMin, expectedClipMax } of [
{ pct: 25, expectedClipMin: 70, expectedClipMax: 80 },
{ pct: 75, expectedClipMin: 20, expectedClipMax: 30 }
]) {
test(
`Screenshot at ${pct}% slider position`,
{ tag: '@screenshot' },
async ({ comfyPage }) => {
const beforeUrl = createTestImageDataUrl('Before', '#c00')
const afterUrl = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [beforeUrl],
afterImages: [afterUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
const beforeImg = node.locator('img[alt="Before image"]')
const afterImg = node.locator('img[alt="After image"]')
await waitForImagesLoaded(node)
await moveToPercentage(comfyPage.page, afterImg, pct)
await expect
.poll(() => getClipPathInsetRightPercent(beforeImg))
.toBeGreaterThan(expectedClipMin)
await expect
.poll(() => getClipPathInsetRightPercent(beforeImg))
.toBeLessThan(expectedClipMax)
await expect(node).toHaveScreenshot(`image-compare-slider-${pct}.png`)
}
)
}
// ---------------------------------------------------------------------------
// Edge cases
// ---------------------------------------------------------------------------
test('Widget remains stable with broken image URLs', async ({
comfyPage
}) => {
await setImageCompareValue(comfyPage, {
beforeImages: ['https://example.invalid/broken.png'],
afterImages: ['https://example.invalid/broken2.png']
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img')).toHaveCount(2)
await expect(node.getByRole('presentation')).toBeVisible()
await expect
.poll(() =>
node.evaluate((el) => {
const imgs = el.querySelectorAll('img')
let errors = 0
imgs.forEach((img) => {
if (img.complete && img.naturalWidth === 0 && img.src) errors++
})
return errors
})
)
.toBe(2)
})
test('Rapid value updates show latest images and reset batch index', async ({
comfyPage
}) => {
const redUrl = createTestImageDataUrl('Red', '#c00')
const green1Url = createTestImageDataUrl('G1', '#0c0')
const green2Url = createTestImageDataUrl('G2', '#090')
const blueUrl = createTestImageDataUrl('Blue', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [redUrl, green1Url],
afterImages: [blueUrl]
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await node.getByTestId('before-batch').getByTestId('batch-next').click()
await expect(
node.getByTestId('before-batch').getByTestId('batch-counter')
).toHaveText('2 / 2')
await setImageCompareValue(comfyPage, {
beforeImages: [green1Url, green2Url],
afterImages: [blueUrl]
})
await expect(node.locator('img[alt="Before image"]')).toHaveAttribute(
'src',
green1Url
)
await expect(
node.getByTestId('before-batch').getByTestId('batch-counter')
).toHaveText('1 / 2')
})
test('Legacy string value shows single image without slider', async ({
comfyPage
}) => {
const url = createTestImageDataUrl('Legacy', '#c00')
await comfyPage.page.evaluate(
({ url }) => {
const node = window.app!.graph.getNodeById(1)
const widget = node?.widgets?.find((w) => w.type === 'imagecompare')
if (widget) {
widget.value = url
widget.callback?.(url)
}
},
{ url }
)
await comfyPage.nextFrame()
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img')).toHaveCount(1)
await expect(node.getByRole('presentation')).toBeHidden()
})
test('Custom beforeAlt and afterAlt are used as img alt text', async ({
comfyPage
}) => {
const beforeUrl = createTestImageDataUrl('Before', '#c00')
const afterUrl = createTestImageDataUrl('After', '#00c')
await setImageCompareValue(comfyPage, {
beforeImages: [beforeUrl],
afterImages: [afterUrl],
beforeAlt: 'Custom before',
afterAlt: 'Custom after'
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(node.locator('img[alt="Custom before"]')).toBeVisible()
await expect(node.locator('img[alt="Custom after"]')).toBeVisible()
})
test('Large batch sizes show correct counter', async ({ comfyPage }) => {
const images = Array.from({ length: 20 }, (_, i) =>
createTestImageDataUrl(String(i + 1), '#c00')
)
await setImageCompareValue(comfyPage, {
beforeImages: images,
afterImages: images
})
const node = comfyPage.vueNodes.getNodeLocator('1')
await expect(
node.getByTestId('before-batch').getByTestId('batch-counter')
).toHaveText('1 / 20')
await expect(
node.getByTestId('after-batch').getByTestId('batch-counter')
).toHaveText('1 / 20')
})
})

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

View File

@@ -180,6 +180,48 @@ test.describe('Node Interaction', () => {
})
})
test.describe('Node Duplication', () => {
test.beforeEach(async ({ comfyPage }) => {
// Pin this suite to the legacy canvas path so Alt+drag exercises
// LGraphCanvas, not the Vue node drag handler.
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
await comfyPage.nextFrame()
})
test('Can duplicate a regular node via Alt+drag', async ({ comfyPage }) => {
const before = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
expect(
before,
'Expected exactly 2 CLIPTextEncode nodes in default graph'
).toHaveLength(2)
const target = before[0]
const pos = await target.getPosition()
const src = { x: pos.x + 16, y: pos.y + 16 }
await comfyPage.page.mouse.move(src.x, src.y)
await comfyPage.page.keyboard.down('Alt')
try {
await comfyPage.page.mouse.down()
await comfyPage.nextFrame()
await comfyPage.page.mouse.move(src.x + 120, src.y + 80, { steps: 20 })
await comfyPage.page.mouse.up()
await comfyPage.nextFrame()
} finally {
await comfyPage.page.keyboard.up('Alt')
}
await comfyPage.canvasOps.moveMouseToEmptyArea()
await expect
.poll(
async () =>
(await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')).length
)
.toBe(3)
expect(await target.exists()).toBe(true)
})
})
test.describe('Edge Interaction', { tag: '@screenshot' }, () => {
test.beforeEach(async ({ comfyPage }) => {
await comfyPage.settings.setSetting(
@@ -351,16 +393,20 @@ test.describe('Node Interaction', () => {
await expect(comfyPage.canvas).toHaveScreenshot(
'text-encode-toggled-off.png'
)
// Re-expand: clicking the canvas toggler on a collapsed node is
// unreliable because DOM widget overlays may intercept the pointer
// event. Use programmatic collapse() for the expand step.
// TODO(#11006): Restore click-to-expand once DOM widget overlay pointer interception is fixed
await comfyPage.page.evaluate((nodeId) => {
const node = window.app!.graph.getNodeById(nodeId)!
node.collapse()
window.app!.canvas.setDirty(true, true)
}, targetNode.id)
await comfyPage.nextFrame()
// Wait for the double-click window (300ms) to expire so the next
// click at the same position isn't interpreted as a double-click.
await expect
.poll(() =>
comfyPage.page.evaluate(() => {
const pointer = window.app!.canvas.pointer
if (!pointer.eLastDown) return true
return performance.now() - pointer.eLastDown.timeStamp > 300
})
)
.toBe(true)
await comfyPage.canvas.click({
position: togglerPos
})
await expect.poll(() => targetNode.isCollapsed()).toBe(false)
// Move mouse away to avoid hover highlight differences.
await comfyPage.canvasOps.moveMouseToEmptyArea()
@@ -1248,7 +1294,7 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
test('Space + left-click drag should pan canvas', async ({ comfyPage }) => {
// Click canvas to focus it
await comfyPage.page.click('canvas')
await comfyPage.canvas.click()
await comfyPage.nextFrame()
await comfyPage.page.keyboard.down('Space')
@@ -1317,7 +1363,7 @@ test.describe('Canvas Navigation', { tag: '@screenshot' }, () => {
'panning'
)
await comfyPage.page.click('canvas')
await comfyPage.canvas.click()
await comfyPage.nextFrame()
await expect(comfyPage.canvas).toHaveScreenshot('standard-initial.png')

View File

@@ -25,7 +25,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
await openMoreOptionsPopover(comfyPage)
await expect(
comfyPage.page.locator('[data-testid="docked-job-history-action"]')
comfyPage.page.getByTestId('docked-job-history-action')
).toBeVisible()
})
@@ -34,9 +34,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
}) => {
await openMoreOptionsPopover(comfyPage)
const action = comfyPage.page.locator(
'[data-testid="docked-job-history-action"]'
)
const action = comfyPage.page.getByTestId('docked-job-history-action')
await expect(action).toBeVisible()
await expect(action).not.toBeEmpty()
})
@@ -45,7 +43,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
await openMoreOptionsPopover(comfyPage)
await expect(
comfyPage.page.locator('[data-testid="show-run-progress-bar-action"]')
comfyPage.page.getByTestId('show-run-progress-bar-action')
).toBeVisible()
})
@@ -53,20 +51,18 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
await openMoreOptionsPopover(comfyPage)
await expect(
comfyPage.page.locator('[data-testid="clear-history-action"]')
comfyPage.page.getByTestId('clear-history-action')
).toBeVisible()
})
test('Clicking docked job history closes popover', async ({ comfyPage }) => {
await openMoreOptionsPopover(comfyPage)
const action = comfyPage.page.locator(
'[data-testid="docked-job-history-action"]'
)
const action = comfyPage.page.getByTestId('docked-job-history-action')
await expect(action).toBeVisible()
await action.click()
await expect(action).not.toBeVisible()
await expect(action).toBeHidden()
})
test('Clicking show run progress bar toggles setting', async ({
@@ -78,9 +74,7 @@ test.describe('Job History Actions', { tag: '@ui' }, () => {
await openMoreOptionsPopover(comfyPage)
const action = comfyPage.page.locator(
'[data-testid="show-run-progress-bar-action"]'
)
const action = comfyPage.page.getByTestId('show-run-progress-bar-action')
await action.click()
await expect

View File

@@ -79,9 +79,7 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
await expect(presetTrigger).toContainText('test-preset')
// Wait for toast to auto-dismiss, then close settings via Escape
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
timeout: 5000
})
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
await page.keyboard.press('Escape')
await comfyPage.settingDialog.waitForHidden()
@@ -133,9 +131,7 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
await expect(presetTrigger).toContainText('test-preset')
// Wait for toast to auto-dismiss
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
timeout: 5000
})
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
// Export via ellipsis menu
await menuButton.click()
@@ -183,9 +179,7 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
await expect(presetTrigger).toContainText('test-preset')
// Wait for toast to auto-dismiss
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
timeout: 5000
})
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
// Delete via ellipsis menu
await menuButton.click()
@@ -223,9 +217,7 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
await expect(presetTrigger).toContainText('test-preset')
// Wait for toast to auto-dismiss
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
timeout: 5000
})
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
// Save as new preset via ellipsis menu
await menuButton.click()
@@ -237,9 +229,7 @@ test.describe('Keybinding Presets', { tag: '@keyboard' }, () => {
await promptInput.press('Enter')
// Wait for toast to auto-dismiss
await expect(comfyPage.toast.visibleToasts).toHaveCount(0, {
timeout: 5000
})
await expect(comfyPage.toast.visibleToasts).toHaveCount(0)
// Verify preset trigger shows my-custom-preset
await expect(presetTrigger).toContainText('my-custom-preset')

Some files were not shown because too many files have changed in this diff Show More