Commit Graph

9 Commits

Author SHA1 Message Date
Christian Byrne
eaf1f9ac77 test: add FeatureFlagHelper and QueueHelper for E2E test infrastructure (#9554)
## Summary

Add 2 reusable test helpers for Playwright E2E tests, integrated into
the ComfyPage fixture. These provide standardized patterns for mocking
feature flags and queue state across all E2E tests.

## Changes

- **`FeatureFlagHelper.ts`** — manage localStorage `ff:` prefixed
feature flags (`seedFlags` for init-time, `setFlags` for runtime) and
mock `/api/features` route
- **`QueueHelper.ts`** — mock `/api/queue` and `/api/history` routes
with configurable running/pending counts and success/error job entries
- **`ComfyPage.ts`** — integrate both helpers as
`comfyPage.featureFlags` and `comfyPage.queue`

## Review Focus

- Helper API design: are `seedFlags`/`setFlags`/`mockServerFeatures` the
right abstractions for feature flag testing?
- Queue mock fidelity: does the mock history shape match real ComfyUI
API responses closely enough?
- These are test-only infrastructure — no production code changes.

## Stack

This is the base PR for the Playwright E2E coverage stack. Waves 1-4 all
branch from this and can merge independently once this lands:
- **→ This PR**: Test infrastructure helpers
- #9555: Toasts, error overlay, selection toolbox, linear mode,
selection rectangle
- #9556: Node search, bottom panel, focus mode, job history, side panel
- #9557: Errors tab, node headers, queue notifications, settings sidebar
- #9558: Minimap, widget copy, floating menus, node library essentials

---------

Co-authored-by: GitHub Action <action@github.com>
2026-03-16 06:51:22 -07:00
Christian Byrne
c420cff4f0 feat: add TBT/frameDuration metrics and new perf test scenarios (#9910)
## Summary

Adds Total Blocking Time (TBT) and frame duration metrics to the
performance testing infrastructure, plus three new test scenarios
covering zoom, pan, and many-nodes-idle.

## Changes

### New Metrics
- **`totalBlockingTimeMs`** — Computed from PerformanceObserver
`longtask` entries: `sum(duration - 50ms)` for tasks >50ms. Measures
main thread blocking.
- **`frameDurationMs`** — Average frame duration via rAF timing (16.67ms
= 60fps target). Measures rendering smoothness.

### New Test Scenarios
| Scenario | Description |
|---|---|
| `canvas-zoom-sweep` | 10 zoom-in + 10 zoom-out cycles on default
workflow |
| `canvas-pan-many-nodes` | 10 pan sweeps over 100-node workflow |
| `canvas-many-nodes-idle` | 2-second idle measurement with 100 nodes
rendered |

### Infrastructure
- `PerformanceHelper.ts`: Installs PerformanceObserver for longtask,
collects TBT, measures frame duration via rAF
- `perf-report.ts`: Reports TBT and frame duration in PR comment tables
- `browser_tests/assets/perf/many_nodes_100.json`: 100-node (10×10 grid)
test fixture

## Review Focus
- TBT collection clears entries at `startMeasuring()` and reads at
`stopMeasuring()` — ensure no race with observer buffering
- Frame duration sampling uses 10 frames — enough for signal without
slowing tests

Depends on: #9886, #9887

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9910-feat-add-TBT-frameDuration-metrics-and-new-perf-test-scenarios-3236d73d365081488ae3c594a8bf7cff)
by [Unito](https://www.unito.io)
2026-03-15 08:54:00 -07:00
Christian Byrne
21069d54e7 feat: expand CDP perf metrics — add DOM nodes, script duration, event listeners (#9887)
## Summary

Expands the performance testing infrastructure to collect 4 additional
CDP metrics that are already returned by `Performance.getMetrics` but
were not being read. This is a zero-cost expansion — no additional CDP
calls, just reading more fields from the existing response.

## New Metrics

| Metric | CDP Source | What It Detects |
|---|---|---|
| `domNodes` | `Nodes` | DOM node count delta — widget DOM leaks during
node create/destroy |
| `jsHeapTotalBytes` | `JSHeapTotalSize` | Total heap delta — combined
with `heapDeltaBytes` shows GC pressure |
| `scriptDurationMs` | `ScriptDuration` | JS execution time vs total
task time — script vs rendering balance |
| `eventListeners` | `JSEventListeners` | Listener count delta — detects
listener accumulation across lifecycle |

## Changes

### `browser_tests/fixtures/helpers/PerformanceHelper.ts`
- Added 4 fields to `PerfSnapshot` interface
- Added 4 fields to `PerfMeasurement` interface
- Wired through `getSnapshot()` and `stopMeasuring()`

### `scripts/perf-report.ts`
- Added 4 fields to `PerfMeasurement` interface
- Expanded `MetricKey` type and `REPORTED_METRICS` array with 3 new
reported metrics (`domNodes`, `scriptDurationMs`, `eventListeners`)
- `jsHeapTotalBytes` is collected but not in `REPORTED_METRICS` — it's
used alongside `heapDeltaBytes` for GC pressure ratio analysis

## Why These 4

From a gap analysis of all ~30 CDP metrics, these were identified as
highest priority for ComfyUI:
- **`Nodes`** (P0): ComfyUI dynamically creates/destroys widget DOM. DOM
bloat from leaked widgets is a key performance risk, especially for Vue
Nodes 2.0.
- **`ScriptDuration`** (P1): Separates JS execution from layout/paint.
Reveals whether perf issues are script-heavy or rendering-heavy.
- **`JSEventListeners`** (P1): Widget lifecycle can leak listeners
across node add/remove cycles.
- **`JSHeapTotalSize`** (P1): With `JSHeapUsedSize`, the ratio shows GC
fragmentation pressure.

## Backward Compatibility

The `PerfMeasurement` interface is extended (not changed). Old baseline
`perf-metrics.json` files without these fields will have `undefined`
values, which the report script handles gracefully (shows `—` for
missing data).

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9887-feat-expand-CDP-perf-metrics-add-DOM-nodes-script-duration-event-listeners-3226d73d3650818abea1d4a441667c38)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-14 23:24:02 -07:00
Christian Byrne
8e215b3174 feat: add performance testing infrastructure with CDP metrics (#9170)
## Summary

Add a permanent, non-failing performance regression detection system
using Chrome DevTools Protocol metrics, with automatic PR commenting.

## Changes

- **What**: Performance testing infrastructure — `PerformanceHelper`
fixture class using CDP `Performance.getMetrics` to collect
`RecalcStyleCount`, `LayoutCount`, `LayoutDuration`, `TaskDuration`,
`JSHeapUsedSize`. Adds `@perf` Playwright project (Chromium-only,
single-threaded, 60s timeout), 4 baseline perf tests, CI workflow with
sticky PR comment reporting, and `perf-report.js` script for generating
markdown comparison tables.

## Review Focus

- `PerformanceHelper` uses `page.context().newCDPSession(page)` — CDP is
Chromium-only, so perf metrics are not collected on Firefox. This is
intentional since CDP gives us browser-level style recalc/layout counts
that `performance.mark/measure` cannot capture.
- The CI workflow uses `continue-on-error: true` so perf tests never
block merging.
- Baseline comparison uses `dawidd6/action-download-artifact` to
download metrics from the target branch, following the same pattern as
`pr-size-report.yaml`.

## Stack

This is the foundation PR for the Firefox performance fix stack:
1. **→ This PR: perf testing infrastructure**
2. `perf/fix-cursor-cache` — cursor style caching (depends on this)
3. `perf/fix-subgraph-svg` — SVG pre-rasterization (depends on this)
4. `perf/fix-clippath-raf` — RAF batching for clip-path (depends on
this)

PRs 2-4 are independent of each other.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9170-feat-add-performance-testing-infrastructure-with-CDP-metrics-3116d73d3650817cb43def6f8e9917f8)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-02-25 20:09:57 -08:00
Alexander Brown
c25f9a0e93 feat: synthetic widgets getter for SubgraphNode (proxy-widget-v2) (#8856)
## Summary

Replace the Proxy-based proxy widget system with a store-driven
architecture where `promotionStore` and `widgetValueStore` are the
single sources of truth for subgraph widget promotion and widget values,
and `SubgraphNode.widgets` is a synthetic getter composing lightweight
`PromotedWidgetView` objects from store state.

## Motivation

The subgraph widget promotion system previously scattered state across
multiple unsynchronized layers:

- **Persistence**: `node.properties.proxyWidgets` (tuples on the
LiteGraph node)
- **Runtime**: Proxy-based `proxyWidget.ts` with `Overlay` objects,
`DisconnectedWidget` singleton, and `isProxyWidget` type guards
- **UI**: Each Vue component independently calling `parseProxyWidgets()`
via `customRef` hacks
- **Mutation flags**: Imperative `widget.promoted = true/false` set on
`subgraph-opened` events

This led to 4+ independent parsings of the same data, complex cache
invalidation, and no reactive contract between the promotion state and
the rendering layer. Widget values were similarly owned by LiteGraph
with no Vue-reactive backing.

The core principle driving these changes: **Vue owns truth**. Pinia
stores are the canonical source; LiteGraph objects delegate to stores
via getters/setters; Vue components react to store state directly.

## Changes

### New stores (single sources of truth)

- **`promotionStore`** — Reactive `Map<NodeId, PromotionEntry[]>`
tracking which interior widgets are promoted on which SubgraphNode
instances. Graph-scoped by root graph ID to prevent cross-workflow state
collision. Replaces `properties.proxyWidgets` parsing, `customRef`
hacks, `widget.promoted` mutation, and the `subgraph-opened` event
listener.
- **`widgetValueStore`** — Graph-scoped `Map<WidgetKey, WidgetState>`
that is the canonical owner of widget values. `BaseWidget.value`
delegates to this store via getter/setter when a node ID is assigned.
Eliminates the need for Proxy-based value forwarding.

### Synthetic widgets getter (SubgraphNode)

`SubgraphNode.widgets` is now a getter that reads
`promotionStore.getPromotions(rootGraphId, nodeId)` and returns cached
`PromotedWidgetView` objects. No stubs, no Proxies, no fake widgets
persisted in the array. The setter is a no-op — mutations go through
`promotionStore`.

### PromotedWidgetView

A class behind a `createPromotedWidgetView` factory, implementing the
`PromotedWidgetView` interface. Delegates value/type/options/drawing to
the resolved interior widget and stores. Owns positional state (`y`,
`computedHeight`) for canvas layout. Cached by
`PromotedWidgetViewManager` for object-identity stability across frames.

### DOM widget promotion

Promoted DOM widgets (textarea, image upload, etc.) render on the
SubgraphNode surface via `positionOverride` in `domWidgetStore`.
`DomWidgets.vue` checks for overrides and uses the SubgraphNode's
coordinates instead of the interior node's.

### Promoted previews

New `usePromotedPreviews` composable resolves image/audio/video preview
widgets from promoted entries, enabling SubgraphNodes to display
previews of interior preview nodes.

### Deleted

- `proxyWidget.ts` (257 lines) — Proxy handler, `Overlay`,
`newProxyWidget`, `isProxyWidget`
- `DisconnectedWidget.ts` (39 lines) — Singleton Proxy target
- `useValueTransform.ts` (32 lines) — Replaced by store delegation

### Key architectural changes

- `BaseWidget.value` getter/setter delegates to `widgetValueStore` when
node ID is set
- `LGraph.add()` reordered: `node.graph` assigned before widget
`setNodeId` (enables store registration)
- `LGraph.clear()` cleans up graph-scoped stores to prevent stale
entries across workflow switches
- `promotionStore` and `widgetValueStore` state nested under root graph
UUID for multi-workflow isolation
- `SubgraphNode.serialize()` writes promotions back to
`properties.proxyWidgets` for persistence compatibility
- Legacy `-1` promotion entries resolved and migrated on first load with
dev warning

## Test coverage

- **3,700+ lines of new/updated tests** across 36 test files
- **Unit**: `promotionStore.test.ts`, `widgetValueStore.test.ts`,
`promotedWidgetView.test.ts` (921 lines),
`subgraphNodePromotion.test.ts`, `proxyWidgetUtils.test.ts`,
`DomWidgets.test.ts`, `PromotedWidgetViewManager.test.ts`,
`usePromotedPreviews.test.ts`, `resolvePromotedWidget.test.ts`,
`subgraphPseudoWidgetCache.test.ts`
- **E2E**: `subgraphPromotion.spec.ts` (622 lines) — promote/demote,
manual/auto promotion, paste preservation, seed control augmentation,
image preview promotion; `imagePreview.spec.ts` extended with
multi-promoted-preview coverage
- **Fixtures**: 2 new subgraph workflow fixtures for preview promotion
scenarios

## Review focus

- Graph-scoped store keying (`rootGraphId`) — verify isolation across
workflows/tabs and cleanup on `LGraph.clear()`
- `PromotedWidgetView` positional stability — `_arrangeWidgets` writes
to `y`/`computedHeight` on cached objects; getter returns fresh array
but stable object references
- DOM widget position override lifecycle — overrides set on promote,
cleared on demote/removal/subgraph navigation
- Legacy `-1` entry migration — resolved and written back on first load;
unresolvable entries dropped with dev warning
- Serialization round-trip — `promotionStore` state →
`properties.proxyWidgets` on serialize, hydrated back on configure

## Diff breakdown (excluding lockfile)

- 153 files changed, ~7,500 insertions, ~1,900 deletions (excluding
pnpm-lock.yaml churn)
- ~3,700 lines are tests
- ~300 lines deleted (proxyWidget.ts, DisconnectedWidget.ts,
useValueTransform.ts)

<!-- Fixes #ISSUE_NUMBER -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-8856-feat-synthetic-widgets-getter-for-SubgraphNode-proxy-widget-v2-3076d73d365081c7b517f5ec7cb514f3)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: GitHub Action <action@github.com>
2026-02-23 13:33:41 -08:00
Johnpaul Chiwetelu
02e926471f fix: replace as-unknown-as casts with safer patterns (#9107)
## Summary

- Replace 83 `as unknown as` double casts with safer alternatives across
33 files
- Use `as Partial<X> as X` pattern where TypeScript allows it
- Create/reuse factory functions from `litegraphTestUtils.ts` for mock
objects
- Widen `getWorkflowDataFromFile` return type to include `ComfyMetadata`
directly
- Reduce total `as unknown as` count from ~153 to 71

The remaining 71 occurrences are genuinely necessary due to cross-schema
casts, generic variance, missing index signatures, Float64Array-to-tuple
conversions, and DOM type incompatibilities.

## Test plan

- [x] `pnpm typecheck` passes
- [x] `pnpm lint` passes
- [x] All affected unit tests pass

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9107-fix-replace-as-unknown-as-casts-with-safer-patterns-3106d73d3650815cb5bcd613ad635bd7)
by [Unito](https://www.unito.io)
2026-02-22 20:46:12 -08:00
Dante
bd95150f82 test: remove unused DebugHelper from browser_tests (#9017)
## Summary

- Remove unused `DebugHelper` class and its import from browser test
fixtures

Fixes #8581

## Test plan

- [x] Verify browser tests still pass without `DebugHelper`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9017-test-remove-unused-DebugHelper-from-browser_tests-30d6d73d36508158b8c9fc83670df4f3)
by [Unito](https://www.unito.io)
2026-02-22 11:52:54 +09:00
pythongosssss
6feb2022a4 Add support for search aliases on subgraphs (#8608)
## Summary

- add commands for setting search aliases and description when in
subgraph
- in future we can add these fields to the dialog when publishing a
subgraph
- map workflow extra metadata on save/load from from/to subgraph node to
allow access via `canvas.subgraph.extra`

## Changes

**What**: 
- new core commands for Comfy.Subgraph.SetSearchAliases &
Comfy.Subgraph.SetDescription to be called when in a subgraph context
- update Publish command to allow command metadata arg for name
- update test executeCommand to allow passing metadata arg

## Review Focus

- When saving a subgraph, the outer workflow "wrapper" is created at the
point of publishing. So unlike a normal workflow `extra` property that
is available at any point, for a subgraph this is not accessible.
To workaround this, the `extra` property that exists on the inner
subgraph node is copied to the top level on save, and restored on load
so extra properties can be set via `canvas.subgraph.extra`.
- I have kept the existing naming format matching `BlueprintDescription`
for `BlueprintSearchAliases` but i'm not sure if the description was
ever used before

## Screenshots


https://github.com/user-attachments/assets/4d4df9c1-2281-4589-aa56-ab07cdecd353

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-8608-Add-support-for-search-aliases-on-subgraphs-2fd6d73d365081d083caebd6befcacdd)
by [Unito](https://www.unito.io)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Set subgraph search aliases (comma-separated) and descriptions;
aliases enable discovery by alternative names.
  * Publish subgraphs using a provided name.
* Node definitions now support search aliases so nodes can be found by
alternate names.
  * UI strings added for entering descriptions and search aliases.

* **Tests**
* Added end-to-end and unit tests covering aliases, descriptions, search
behavior, publishing, and persistence.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-04 12:52:30 -08:00
Alexander Brown
f2d5bfab73 test(browser): refactor browser tests for reliability and maintainability (#8510)
## Summary

Major refactoring of browser tests to improve reliability,
maintainability, and type safety.

## Changes

### Test Infrastructure Decomposition
- Decomposed `ComfyPage.ts` (~1000 lines) into focused helpers:
- `CanvasHelper`, `DebugHelper`, `SubgraphHelper`,
`NodeOperationsHelper`
- `SettingsHelper`, `WorkflowHelper`, `ClipboardHelper`,
`KeyboardHelper`
- Created `ContextMenu` page object, `BaseDialog` base class, and
`BottomPanel` page object
- Extracted `DefaultGraphPositions` constants

### Locator Stability
- Added `data-testid` attributes to Vue components (sidebar, dialogs,
node library)
- Created centralized `selectors.ts` with test ID constants
- Replaced fragile CSS selectors (`.nth()`, `:nth-child()`) with
`getByTestId`/`getByRole`

### Performance & Reliability
- Removed `setTimeout` anti-patterns (replaced with `waitForFunction`)
- Replaced `waitForTimeout` with retrying assertions
- Replaced hardcoded coordinates with computed `NodeReference` positions
- Enforced LF line endings for all text files

### Type Safety
- Enabled `no-explicit-any` lint rule for browser_tests via oxlint
- Purged `as any` casts from browser_tests
- Added Window type augmentation for standardized window access
- Added proper type annotations throughout

### Bug Fixes
- Restored `ExtensionManager` API contract
- Removed test-only settings from production schema
- Fixed flaky selectors and missing test setup

## Testing
- All browser tests pass
- Typecheck passes


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Overhauled browser E2E test infrastructure with many new
helpers/fixtures, updated test APIs, and CI test container image bumped
for consistency.

* **Chores**
* Standardized line endings and applied stricter lint rules for browser
tests; workspace dependency version updated.

* **Documentation**
* Updated Playwright and TypeScript testing guidance and test-run
commands.

* **UI**
* Added stable data-testids to multiple components to improve
testability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-03 12:29:40 -08:00