Files
ComfyUI_frontend/browser_tests/fixtures/helpers/SubgraphHelper.ts
Alexander Brown c25f9a0e93 feat: synthetic widgets getter for SubgraphNode (proxy-widget-v2) (#8856)
## Summary

Replace the Proxy-based proxy widget system with a store-driven
architecture where `promotionStore` and `widgetValueStore` are the
single sources of truth for subgraph widget promotion and widget values,
and `SubgraphNode.widgets` is a synthetic getter composing lightweight
`PromotedWidgetView` objects from store state.

## Motivation

The subgraph widget promotion system previously scattered state across
multiple unsynchronized layers:

- **Persistence**: `node.properties.proxyWidgets` (tuples on the
LiteGraph node)
- **Runtime**: Proxy-based `proxyWidget.ts` with `Overlay` objects,
`DisconnectedWidget` singleton, and `isProxyWidget` type guards
- **UI**: Each Vue component independently calling `parseProxyWidgets()`
via `customRef` hacks
- **Mutation flags**: Imperative `widget.promoted = true/false` set on
`subgraph-opened` events

This led to 4+ independent parsings of the same data, complex cache
invalidation, and no reactive contract between the promotion state and
the rendering layer. Widget values were similarly owned by LiteGraph
with no Vue-reactive backing.

The core principle driving these changes: **Vue owns truth**. Pinia
stores are the canonical source; LiteGraph objects delegate to stores
via getters/setters; Vue components react to store state directly.

## Changes

### New stores (single sources of truth)

- **`promotionStore`** — Reactive `Map<NodeId, PromotionEntry[]>`
tracking which interior widgets are promoted on which SubgraphNode
instances. Graph-scoped by root graph ID to prevent cross-workflow state
collision. Replaces `properties.proxyWidgets` parsing, `customRef`
hacks, `widget.promoted` mutation, and the `subgraph-opened` event
listener.
- **`widgetValueStore`** — Graph-scoped `Map<WidgetKey, WidgetState>`
that is the canonical owner of widget values. `BaseWidget.value`
delegates to this store via getter/setter when a node ID is assigned.
Eliminates the need for Proxy-based value forwarding.

### Synthetic widgets getter (SubgraphNode)

`SubgraphNode.widgets` is now a getter that reads
`promotionStore.getPromotions(rootGraphId, nodeId)` and returns cached
`PromotedWidgetView` objects. No stubs, no Proxies, no fake widgets
persisted in the array. The setter is a no-op — mutations go through
`promotionStore`.

### PromotedWidgetView

A class behind a `createPromotedWidgetView` factory, implementing the
`PromotedWidgetView` interface. Delegates value/type/options/drawing to
the resolved interior widget and stores. Owns positional state (`y`,
`computedHeight`) for canvas layout. Cached by
`PromotedWidgetViewManager` for object-identity stability across frames.

### DOM widget promotion

Promoted DOM widgets (textarea, image upload, etc.) render on the
SubgraphNode surface via `positionOverride` in `domWidgetStore`.
`DomWidgets.vue` checks for overrides and uses the SubgraphNode's
coordinates instead of the interior node's.

### Promoted previews

New `usePromotedPreviews` composable resolves image/audio/video preview
widgets from promoted entries, enabling SubgraphNodes to display
previews of interior preview nodes.

### Deleted

- `proxyWidget.ts` (257 lines) — Proxy handler, `Overlay`,
`newProxyWidget`, `isProxyWidget`
- `DisconnectedWidget.ts` (39 lines) — Singleton Proxy target
- `useValueTransform.ts` (32 lines) — Replaced by store delegation

### Key architectural changes

- `BaseWidget.value` getter/setter delegates to `widgetValueStore` when
node ID is set
- `LGraph.add()` reordered: `node.graph` assigned before widget
`setNodeId` (enables store registration)
- `LGraph.clear()` cleans up graph-scoped stores to prevent stale
entries across workflow switches
- `promotionStore` and `widgetValueStore` state nested under root graph
UUID for multi-workflow isolation
- `SubgraphNode.serialize()` writes promotions back to
`properties.proxyWidgets` for persistence compatibility
- Legacy `-1` promotion entries resolved and migrated on first load with
dev warning

## Test coverage

- **3,700+ lines of new/updated tests** across 36 test files
- **Unit**: `promotionStore.test.ts`, `widgetValueStore.test.ts`,
`promotedWidgetView.test.ts` (921 lines),
`subgraphNodePromotion.test.ts`, `proxyWidgetUtils.test.ts`,
`DomWidgets.test.ts`, `PromotedWidgetViewManager.test.ts`,
`usePromotedPreviews.test.ts`, `resolvePromotedWidget.test.ts`,
`subgraphPseudoWidgetCache.test.ts`
- **E2E**: `subgraphPromotion.spec.ts` (622 lines) — promote/demote,
manual/auto promotion, paste preservation, seed control augmentation,
image preview promotion; `imagePreview.spec.ts` extended with
multi-promoted-preview coverage
- **Fixtures**: 2 new subgraph workflow fixtures for preview promotion
scenarios

## Review focus

- Graph-scoped store keying (`rootGraphId`) — verify isolation across
workflows/tabs and cleanup on `LGraph.clear()`
- `PromotedWidgetView` positional stability — `_arrangeWidgets` writes
to `y`/`computedHeight` on cached objects; getter returns fresh array
but stable object references
- DOM widget position override lifecycle — overrides set on promote,
cleared on demote/removal/subgraph navigation
- Legacy `-1` entry migration — resolved and written back on first load;
unresolvable entries dropped with dev warning
- Serialization round-trip — `promotionStore` state →
`properties.proxyWidgets` on serialize, hydrated back on configure

## Diff breakdown (excluding lockfile)

- 153 files changed, ~7,500 insertions, ~1,900 deletions (excluding
pnpm-lock.yaml churn)
- ~3,700 lines are tests
- ~300 lines deleted (proxyWidget.ts, DisconnectedWidget.ts,
useValueTransform.ts)

<!-- Fixes #ISSUE_NUMBER -->

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-8856-feat-synthetic-widgets-getter-for-SubgraphNode-proxy-widget-v2-3076d73d365081c7b517f5ec7cb514f3)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: GitHub Action <action@github.com>
2026-02-23 13:33:41 -08:00

326 lines
11 KiB
TypeScript

import type { Page } from '@playwright/test'
import type {
CanvasPointerEvent,
Subgraph
} from '@/lib/litegraph/src/litegraph'
import type { ComfyPage } from '../ComfyPage'
import type { NodeReference } from '../utils/litegraphUtils'
import { SubgraphSlotReference } from '../utils/litegraphUtils'
export class SubgraphHelper {
constructor(private readonly comfyPage: ComfyPage) {}
private get page(): Page {
return this.comfyPage.page
}
/**
* Core helper method for interacting with subgraph I/O slots.
* Handles both input/output slots and both right-click/double-click actions.
*
* @param slotType - 'input' or 'output'
* @param action - 'rightClick' or 'doubleClick'
* @param slotName - Optional specific slot name to target
*/
private async interactWithSubgraphSlot(
slotType: 'input' | 'output',
action: 'rightClick' | 'doubleClick',
slotName?: string
): Promise<void> {
const foundSlot = await this.page.evaluate(
async (params) => {
const { slotType, action, targetSlotName } = params
const app = window.app!
const currentGraph = app.canvas!.graph!
// Check if we're in a subgraph
if (!('inputNode' in currentGraph)) {
throw new Error(
'Not in a subgraph - this method only works inside subgraphs'
)
}
const subgraph = currentGraph as Subgraph
// Get the appropriate node and slots
const node =
slotType === 'input' ? subgraph.inputNode : subgraph.outputNode
const slots = slotType === 'input' ? subgraph.inputs : subgraph.outputs
if (!node) {
throw new Error(`No ${slotType} node found in subgraph`)
}
if (!slots || slots.length === 0) {
throw new Error(`No ${slotType} slots found in subgraph`)
}
// Filter slots based on target name and action type
const slotsToTry = targetSlotName
? slots.filter((slot) => slot.name === targetSlotName)
: action === 'rightClick'
? slots
: [slots[0]] // Right-click tries all, double-click uses first
if (slotsToTry.length === 0) {
throw new Error(
targetSlotName
? `${slotType} slot '${targetSlotName}' not found`
: `No ${slotType} slots available to try`
)
}
// Handle the interaction based on action type
if (action === 'rightClick') {
// Right-click: try each slot until one works
for (const slot of slotsToTry) {
if (!slot.pos) continue
const event = {
canvasX: slot.pos[0],
canvasY: slot.pos[1],
button: 2, // Right mouse button
preventDefault: () => {},
stopPropagation: () => {}
}
if (node.onPointerDown) {
node.onPointerDown(
event as Partial<CanvasPointerEvent> as CanvasPointerEvent,
app.canvas.pointer,
app.canvas.linkConnector
)
return {
success: true,
slotName: slot.name,
x: slot.pos[0],
y: slot.pos[1]
}
}
}
} else if (action === 'doubleClick') {
// Double-click: use first slot with bounding rect center
const slot = slotsToTry[0]
if (!slot.boundingRect) {
throw new Error(`${slotType} slot bounding rect not found`)
}
const rect = slot.boundingRect
const testX = rect[0] + rect[2] / 2 // x + width/2
const testY = rect[1] + rect[3] / 2 // y + height/2
const event = {
canvasX: testX,
canvasY: testY,
button: 0, // Left mouse button
preventDefault: () => {},
stopPropagation: () => {}
}
if (node.onPointerDown) {
node.onPointerDown(
event as Partial<CanvasPointerEvent> as CanvasPointerEvent,
app.canvas.pointer,
app.canvas.linkConnector
)
// Trigger double-click
if (app.canvas.pointer.onDoubleClick) {
app.canvas.pointer.onDoubleClick(
event as Partial<CanvasPointerEvent> as CanvasPointerEvent
)
}
}
return { success: true, slotName: slot.name, x: testX, y: testY }
}
return { success: false }
},
{ slotType, action, targetSlotName: slotName }
)
if (!foundSlot.success) {
const actionText =
action === 'rightClick' ? 'open context menu for' : 'double-click'
throw new Error(
slotName
? `Could not ${actionText} ${slotType} slot '${slotName}'`
: `Could not find any ${slotType} slot to ${actionText}`
)
}
// Wait for the appropriate UI element to appear
if (action === 'rightClick') {
await this.page.waitForSelector('.litemenu-entry', {
state: 'visible',
timeout: 5000
})
} else {
await this.comfyPage.nextFrame()
}
}
/**
* Right-clicks on a subgraph input slot to open the context menu.
* Must be called when inside a subgraph.
*
* This method uses the actual slot positions from the subgraph.inputs array,
* which contain the correct coordinates for each input slot. These positions
* are different from the visual node positions and are specifically where
* the slots are rendered on the input node.
*
* @param inputName Optional name of the specific input slot to target (e.g., 'text').
* If not provided, tries all available input slots until one works.
* @returns Promise that resolves when the context menu appears
*/
async rightClickInputSlot(inputName?: string): Promise<void> {
return this.interactWithSubgraphSlot('input', 'rightClick', inputName)
}
/**
* Right-clicks on a subgraph output slot to open the context menu.
* Must be called when inside a subgraph.
*
* Similar to rightClickInputSlot but for output slots.
*
* @param outputName Optional name of the specific output slot to target.
* If not provided, tries all available output slots until one works.
* @returns Promise that resolves when the context menu appears
*/
async rightClickOutputSlot(outputName?: string): Promise<void> {
return this.interactWithSubgraphSlot('output', 'rightClick', outputName)
}
/**
* Double-clicks on a subgraph input slot to rename it.
* Must be called when inside a subgraph.
*
* @param inputName Optional name of the specific input slot to target (e.g., 'text').
* If not provided, tries the first available input slot.
* @returns Promise that resolves when the rename dialog appears
*/
async doubleClickInputSlot(inputName?: string): Promise<void> {
return this.interactWithSubgraphSlot('input', 'doubleClick', inputName)
}
/**
* Double-clicks on a subgraph output slot to rename it.
* Must be called when inside a subgraph.
*
* @param outputName Optional name of the specific output slot to target.
* If not provided, tries the first available output slot.
* @returns Promise that resolves when the rename dialog appears
*/
async doubleClickOutputSlot(outputName?: string): Promise<void> {
return this.interactWithSubgraphSlot('output', 'doubleClick', outputName)
}
/**
* Get a reference to a subgraph input slot
*/
getInputSlot(slotName?: string): SubgraphSlotReference {
return new SubgraphSlotReference('input', slotName || '', this.comfyPage)
}
/**
* Get a reference to a subgraph output slot
*/
getOutputSlot(slotName?: string): SubgraphSlotReference {
return new SubgraphSlotReference('output', slotName || '', this.comfyPage)
}
/**
* Connect a regular node output to a subgraph input.
* This creates a new input slot on the subgraph if targetInputName is not provided.
*/
async connectToInput(
sourceNode: NodeReference,
sourceSlotIndex: number,
targetInputName?: string
): Promise<void> {
const sourceSlot = await sourceNode.getOutput(sourceSlotIndex)
const targetSlot = this.getInputSlot(targetInputName)
const targetPosition = targetInputName
? await targetSlot.getPosition() // Connect to existing slot
: await targetSlot.getOpenSlotPosition() // Create new slot
await this.comfyPage.canvasOps.dragAndDrop(
await sourceSlot.getPosition(),
targetPosition
)
await this.comfyPage.nextFrame()
}
/**
* Connect a subgraph input to a regular node input.
* This creates a new input slot on the subgraph if sourceInputName is not provided.
*/
async connectFromInput(
targetNode: NodeReference,
targetSlotIndex: number,
sourceInputName?: string
): Promise<void> {
const sourceSlot = this.getInputSlot(sourceInputName)
const targetSlot = await targetNode.getInput(targetSlotIndex)
const sourcePosition = sourceInputName
? await sourceSlot.getPosition() // Connect from existing slot
: await sourceSlot.getOpenSlotPosition() // Create new slot
const targetPosition = await targetSlot.getPosition()
await this.comfyPage.canvasOps.dragAndDrop(sourcePosition, targetPosition)
await this.comfyPage.nextFrame()
}
/**
* Connect a regular node output to a subgraph output.
* This creates a new output slot on the subgraph if targetOutputName is not provided.
*/
async connectToOutput(
sourceNode: NodeReference,
sourceSlotIndex: number,
targetOutputName?: string
): Promise<void> {
const sourceSlot = await sourceNode.getOutput(sourceSlotIndex)
const targetSlot = this.getOutputSlot(targetOutputName)
const targetPosition = targetOutputName
? await targetSlot.getPosition() // Connect to existing slot
: await targetSlot.getOpenSlotPosition() // Create new slot
await this.comfyPage.canvasOps.dragAndDrop(
await sourceSlot.getPosition(),
targetPosition
)
await this.comfyPage.nextFrame()
}
/**
* Connect a subgraph output to a regular node input.
* This creates a new output slot on the subgraph if sourceOutputName is not provided.
*/
async connectFromOutput(
targetNode: NodeReference,
targetSlotIndex: number,
sourceOutputName?: string
): Promise<void> {
const sourceSlot = this.getOutputSlot(sourceOutputName)
const targetSlot = await targetNode.getInput(targetSlotIndex)
const sourcePosition = sourceOutputName
? await sourceSlot.getPosition() // Connect from existing slot
: await sourceSlot.getOpenSlotPosition() // Create new slot
await this.comfyPage.canvasOps.dragAndDrop(
sourcePosition,
await targetSlot.getPosition()
)
await this.comfyPage.nextFrame()
}
}