Compare commits

..

36 Commits

Author SHA1 Message Date
Nathaniel Parson Koroso
7adfaa9079 test: stabilize every-node auto-run across environments; document the tiers
The cannotRunAlone baseline (per pack, in the manifest) records nodes
that cannot execute standalone on a bare backend, asserted both ways so
entries cannot rot: an unlisted failure is a regression, a listed node
that runs clean must be removed. queuePrompt rejection is retried once
before classifying VALIDATION_FAIL - pack JS hooking the queue path can
refuse transiently, and the backend log proved several apparent rejects
never reached the server. Nodes whose execution depends on their own
pack JS preprocessing widget values (rgthree Power widgets, KJ editors)
are excluded unconditionally with the mechanism recorded, since whether
a page applies pack JS varies by serving setup; ML-session initializers
and unstable executed-set reporters join them. ADDING_PACKS and the
README document the every-node tiers and all five exception ledgers.

Verified 65/65 in both documented environments: dev server and
dist-serving CI parity, twice consecutively on the latter.
2026-07-06 16:16:51 -07:00
Nathaniel Parson Koroso
1e36107109 test: every-node coverage - mount, save/reload, connect, and auto-run for all pack nodes
All-nodes tiers discover each pack's full node list from the live
backend: chunked mount checks in both renderers (batch size 24, chosen
by the committed calibration tool), chunked save/reload round-trips,
a connectivity corpus widened from the curated sentinels to every
registered node, and an auto-run tier that classifies every node
(AUTO_RUNNABLE / NEEDS_WIRES / NEEDS_MODELS / NO_SINK) and executes the
runnable ones in batches with per-node bisection on failure.

Hard-won harness rules baked in: queuePrompt rejects classify instantly
as VALIDATION_FAIL instead of burning the timeout; a timed-out batch
interrupts and verifies the queue drained so one hung node cannot jam
every later run; a pre-flight queue check fails fast with the real
cause; and three reviewable exception ledgers carry reasons inline
(AUTO_RUN_EXCLUDE for runtime-downloaders like RemBGSession+, a scoped
console-noise allowlist for KJNodes' undefined-filename previews,
connectivity CONNECT_REJECTED/ROUNDTRIP_LOST entries for pack JS that
vetoes or drops links).
2026-07-06 16:16:23 -07:00
Nathaniel Parson Koroso
1d5514c90e docs: rename ADDING_PACKS to ADDING_CUSTOM_NODES
The doc onboards custom nodes; name it what it is.
2026-07-06 16:16:09 -07:00
Nathaniel Parson Koroso
61d1cbfdb0 docs: fix ADDING_PACKS extensions probe and misattributed CI triage
grep -c on the single-line /extensions JSON could only say 0 or 1; count
entries properly with the same python one-liner style the doc already
uses. The Step 7 triage claimed our CI failure was upstream drift; it was
the dev-server blindspot - reorder the advice to reproduce under 6b
before diagnosing.
2026-07-02 18:49:20 -07:00
Nathaniel Parson Koroso
14666b09c4 docs: fold 5-pack onboarding lessons into ADDING_PACKS
Detect frontend-JS packs at install time, split local verification into
the fast dev-server loop and the CI-parity dist run (required when the
pack ships frontend JS), spell out that workflow media paths resolve
against the backend's working directory, and add upstream-drift triage
for unpinned packs. Checklist updated to match.
2026-07-02 18:46:05 -07:00
Nathaniel Parson Koroso
efb0365bc3 test: make connectivity instance-aware; slot drags survive pack page chrome
Two CI failures with the 7-pack backend, both from pack frontend JS that
never loads under the Vite dev server (its /extensions list is core-only):

- rgthree's Seed rebuilds its declared seed input as a widget-only
  control, so the planned BatchCount+.INT -> Seed.seed pair has no socket
  on the instance. The sweep now classifies that as
  WIDGET_ONLY_ON_INSTANCE, logged and excluded like wildcards; a name
  missing from both slots and widgets still fails hard, and the drag test
  picks the first in-pack pair that materializes on real instances.

- rgthree's progress bar shifts the canvas element 16px down, and
  NodeSlotReference.getPosition returned canvas-relative coordinates, so
  every slot drag grabbed the node title instead of the slot dot. Slot
  positions now include the canvas element's page offset (a no-op when
  the canvas sits at 0,0).

Documents the dev-server blindspot and the CI-parity loop (build dist,
--front-end-root) in the suite README and ADDING_PACKS. Verified 36/36
green against both the dev server and a dist-serving 7-pack backend.
2026-07-02 18:42:48 -07:00
Nathaniel Parson Koroso
065bc0c336 test: fail manifest load when a run tier has no workflow
A run row with an empty workflow would skip locally and rely on CI's
skip gate to notice the lost coverage; enforce the documented contract
at load time instead.
2026-07-02 18:13:26 -07:00
Nathaniel Parson Koroso
1248c4628a test: onboard 5 packs (rgthree, essentials, KJNodes, Custom-Scripts, WAS) with vueNodesCompatible flag and ADDING_PACKS guide
Five new manifest rows, each covering load, connectivity, and run tiers
with hand-authored model-free workflows verified against a live backend.
New optional vueNodesCompatible manifest field: a pack proven unable to
mount under Vue Nodes 2.0 runs its LiteGraph assertions only - never a
test.skip, so the zero-skip CI gate stays honest. All five packs mount
under Vue Nodes 2.0 empirically, so no row sets the flag; the decision
helper is unit-tested instead. ADDING_PACKS.md is the authoritative
step-by-step onboarding process, validated against live /object_info.
Manifest rows now also fail fast on an empty repo field.
2026-07-02 18:05:20 -07:00
Nathaniel Parson Koroso
ee83d67834 test: single tier source of truth; fix skip diagnostic to find nested skips
Derive CustomNodeTier from the VALID_TIERS array (as const) so adding a tier
is one edit and the type/runtime lists can't drift. The forbid-skips
diagnostic now recurses the report and prints only specs that actually
skipped - the old dump printed every title and a single-level filter would
miss specs nested under describe() blocks (which the regression spec uses).
2026-07-02 16:07:04 -07:00
Nathaniel Parson Koroso
f63b7d866e ci: gate custom-node job with changes-filter, not trigger paths
A required check gated by a trigger-level paths filter never creates a check
run on a PR that touches none of those paths, leaving branch protection stuck
Pending. Move the gating to a job-level if via the changes-filter action (a
skipped job counts as passing), mirroring ci-tests-unit.yaml, so this can be
marked required without stalling docs-only PRs. Keeps the same-repo fork guard
in the same if.
2026-07-02 16:03:37 -07:00
Nathaniel Parson Koroso
068191ea47 test: harden custom-node CI and manifest per review
Security: the pack-install job now runs only for same-repo PRs and pushes, so
a fork PR can't point the manifest's repo URLs at attacker-controlled code
that the job would clone and pip-install. Fork PRs keep the env-agnostic
coverage via the main e2e shards.

Stability: pack requirements install under a pip constraint pinning the CPU
torch stack, so no pack can swap torch for a GPU/incompatible build on the
--cpu runner.

Correctness: manifest validation rejects unknown tier values (a 'connectivty'
typo would otherwise silently drop that tier's coverage). Connectivity's
'pack installed' predicate is extracted to one isEntryInstalled helper used by
both the breadth and drag tests.
2026-07-02 15:56:17 -07:00
Nathaniel Parson Koroso
07c4b230b2 ci: make the custom-node job gating - fail on pack-install error or any skip
A regression gate that lets a broken pack through as a skip is theater. Pack
clone/dependency failures now fail the job (array+loop instead of a
failure-swallowing jq|while pipe), and a post-run check fails the job if any
test was skipped - on this backend every tier is meant to run, so a skip
means a pack or devtools did not load. Drops the informational framing;
mark custom-nodes-e2e required in branch protection to block merges.
2026-07-02 15:36:19 -07:00
Nathaniel Parson Koroso
9ed51f1e4b ci: run the custom-node suite against a backend with the packs installed
Phase 5. A new informational (non-gating) workflow that reuses the repo's
setup-frontend/setup-playwright/setup-comfyui-server actions, then installs
every pack the manifest declares (jq loop over customNodeManifest.json, so a
new pack row installs itself with no workflow change) and boots ComfyUI with
--multi-user --cache-none before running browser_tests/tests/customNodes.

This makes the load and run tiers actually execute in CI instead of skipping
for want of the packs - the whole point of the suite. A pack whose deps fail
degrades to an honest skip rather than reddening the job.
2026-07-02 15:25:31 -07:00
Nathaniel Parson Koroso
4a91fa4849 test: name connectivity tests in plain language
T-conn was planning-doc shorthand for the connectivity tier; test titles and
logs now say connectivity outright so CI output reads without tribal
knowledge.
2026-07-02 14:11:56 -07:00
Nathaniel Parson Koroso
0991905a89 test: exclude COMBO literals from connectivity auto-pairing
CI caught what a pack-rich local backend masked: isValidConnection compares
only the string COMBO while every combo slot carries its own option set, so
the planner would wire a checkpoint dropdown into a scheduler dropdown and
call it proof, and combo outputs declare a non-string output_name whose
instance slot name never matches (DevToolsNodeWithOutputCombo failed 5
pairs on CI as SLOT_CONTRACT_MISMATCH). Combo slots are now recorded and
counted like wildcards instead of paired, the normalizer coerces slot names
to strings, and a pure spec locks both behaviors. Targeted fixtures remain
the way to cover combo semantics.
2026-07-02 14:10:37 -07:00
Nathaniel Parson Koroso
df6764762b test: make the custom-node suite work on multi-user backends
The Comfy.userId=default settings override broke every test on multi-user
backends (the repo's stated browser-test prerequisite): devtools
set_settings wrote to a user no session reads, so Comfy.TutorialCompleted
never landed, the templates dialog never opened, and the beforeEach wait
timed out - CI sessions even inherited leftover settings (a zh locale) from
earlier tests on the same worker user. Dropping the override lets the
fixture target the real per-worker user everywhere; the harness backend now
runs --multi-user like CI. Connectivity's per-pack guards and drag
derivation apply only to installed packs, so a backend without the manifest
packs reports the absence instead of hard-failing while the core sweep,
native drag, and self-checks still run.
2026-07-02 13:45:34 -07:00
Nathaniel Parson Koroso
2d2b318450 test: drop exports from internal-only custom-node types
knip flags exported types with no external consumers; CustomNodeTier,
ObjectInfoNode, NormalizedSlot, and SlotRef are referenced only within
their own modules.
2026-07-02 13:15:58 -07:00
GitHub Action
0f94da8746 [automated] Apply ESLint and Oxfmt fixes 2026-07-02 19:32:18 +00:00
Nathaniel Parson Koroso
d80427d014 test: assert breadth-sweep console errors and tighten manifest shape checks
The breadth sweep now fails on any console error captured during the
connect/serialize/prompt loop, matching the fidelity test. The wildcard
predicate is exported from typePairing and reused instead of re-derived.
assertEntry validates real shapes (non-empty pack/expectedNodes/tiers,
arrays, boolean requiresGpu, finite positive timeoutMs); workflow stays
allowed as an empty string until a pack gains a run-tier fixture.
2026-07-02 12:28:23 -07:00
Nathaniel Parson Koroso
d02e665290 test: address review feedback on the custom-node suite
Resolve the manifest path from import.meta.url so tests are cwd-independent,
and validate requiresGpu at manifest load. Reuse the centralized TestIds for
the error overlay, error dialog, and templates dialog selectors. Extract the
shared suite settings and templates-dialog dismissal into
fixtures/utils/customNodeSuite so the three specs cannot drift. Rename
spikeDesktop.spec.ts to coreSmoke.spec.ts to match its maintained purpose,
document the full manifest schema in the README, and describe the gate
outcome without a hardcoded test count.
2026-07-02 12:24:33 -07:00
Nathaniel Parson Koroso
dc83cc4df6 test: prove the connectivity executor can reject, and drag every pack
A permanent self-check feeds the shared pair executor a type-incompatible
pair and a fabricated slot name and requires CONNECT_REJECTED and
SLOT_CONTRACT_MISMATCH back, so a green sweep can never come from a
classifier that lost the ability to fail. The breadth test asserts every
connectivity-tier pack contributes pairs, guarding pack attribution. The
drag tier's widget-primitive exclusion is removed: widget-backed inputs
render real slot dots under Vue Nodes (verified empirically), so every pack
now gets an in-pack drag in both renderers, asserted present.
2026-07-02 12:19:40 -07:00
Nathaniel Parson Koroso
8b81a4f359 test: add connectivity tier proving the slot/type contract
A type-pairing generator indexes /object_info producers and consumers and
plans one representative typed edge per slot, excluding wildcard slots
(isValidConnection short-circuits on * before the real type compare, so a
wildcard link proves reachability, not interop). The breadth sweep connects
every planned edge through the real validator in-page and requires each link
to survive serialize/configure and appear in graphToPrompt output; verified
up front that graphToPrompt emits links even when other required inputs
dangle. A curated subset is dragged slot-dot to slot-dot under both
renderers, addressed by data-slot-key so shared labels cannot misfire.
Orphan types are reported, never failed; connect vetoes must match a
committed allow-list. Manifest packs opt in via a connectivity tier that
needs no extra assets.
2026-07-02 12:14:39 -07:00
GitHub Action
8f567e8ef0 [automated] Apply ESLint and Oxfmt fixes 2026-07-02 18:44:36 +00:00
Nathaniel Parson Koroso
4fb282f853 docs: link custom-node suite README from browser_tests README 2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
d17a387ddb test: name each custom-node check as a pnpm script and document the suite
One script per pack tier (impact-render/impact-run/vhs-render/vhs-run) plus
the self-check, all opening the Playwright Inspector so anyone can step
through what the robot does. README covers prerequisites, every script, a
worked example, the zero-visible-errors contract, and how to add a pack.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
68ba0aa613 test: add pnpm scripts for the custom-node suite
test:custom-nodes runs the whole suite headless (the gate); :watch opens a
headed slow-motion run of the browser tiers; :debug steps through them in the
Playwright Inspector. All target the local dev server on :5173 and use the
committed system-Chrome config (no bundled-chromium download). Pass -g to
:watch / :debug to run a single test, e.g. -g 'VideoHelperSuite.*T1'.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
675140c164 test: drop io tier scaffold until assertion-node pack exists
T2a gated on the ComfyUI-test-framework 'Assert Executed' nodes, which are
not published for any backend yet, so the tier could only ever skip. A test
that cannot run anywhere is reporting noise; restore it from history when
the assertion pack lands. Suite is now 16 passed, zero skips, zero failures.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
64706c53c3 test: enforce zero visible errors across custom-node suite
Every browser tier now asserts the app's user-facing error surfaces (error
overlay, error dialog, node render errors, error toasts) are absent at test
start and after each pass, so a run is green only if a human watching the
screen sees zero errors. The harness self-check asserts the overlay IS
visible after a forced execution error, keeping the selectors provably live.

Sessions boot with a blank graph (Comfy.TutorialCompleted=false) because the
bundled default template references models absent on a scoped backend; the
tutorial path's auto-opened template browser is dismissed per test. Settings
now reach the session on single-user server-storage backends by routing
devtools set_settings to the default user, and the errors tab stays enabled
so error indicators are never suppressed in this suite. The smoke test loads
a core-only model-free workflow instead of the SD1.5 default asset.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
bfa94d4118 test: enable run tier for Impact and VHS with model-free workflows
Impact runs ImpactInt and ImpactFloat into PreviewAny as a group; VHS decodes
the existing plain_video.mp4 asset through VHS_LoadVideoPath into
VHS_VideoInfo. The executed-set check asserts each expected node individually
executed, so group workflows still verify per-node execution. Requires a
cache-disabled backend (--cache-none) with the video staged in its input dir;
documented on the manifest workflow field.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
b7708d5ad0 test: validate timeoutMs and requiresModels at manifest load 2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
564de12d46 test: harden custom-node suite per review findings
Normalize the executing event tap: its CustomEvent detail is a bare node-id
string, so the previous object-spread left the executed-set permanently empty.
The self-check now asserts a non-empty executed-set to keep that path live.

T0 now clears the graph per renderer pass, asserts exact node counts, and
verifies each added pack node's own data-node-id mounts in the Vue pass
(default-workflow nodes can no longer satisfy the assertions). Console capture
starts before the renderer toggle. Added an object_info sanity floor so a
depleted getNodeDefs fails loudly instead of skipping everything.

Run/io tiers gain test.setTimeout, requiresModels gating, and an empty-workflow
guard. Interrupted runs get pure-spec coverage; the shared console collector
moves to fixtures/utils.
2026-07-02 11:39:53 -07:00
Nathaniel Parson Koroso
5a1f788230 test: custom-node E2E regression suite (load/render, both renderers)
Data-driven Playwright harness verifying custom-node packs load and render
under both LiteGraph 1.0 and Vue Nodes 2.0 against a real ComfyUI backend.
Pure classifier/validator/manifest logic is unit-tested; the regression spec
renders each pack's nodes in both renderers (Vue via data-node-id DOM) and a
self-check runs a workflow to confirm execution-error capture. Proven against
Impact Pack + VideoHelperSuite.

Makes ComfyPage.createUser idempotent so the suite runs against a persistent
backend (Desktop server user storage).
2026-07-02 11:39:53 -07:00
imick-io
a6db1ab3d6 fix(website): restore node-link.svg intrinsic sizing (#13384)
## Summary

Restore the original `node-link.svg` asset, which PR #13095 accidentally
overwrote with a stretch-to-fill Figma export, breaking the node
connector across the marketing site.

## Changes

- **What**: Revert `apps/website/public/icons/node-link.svg` to its
intrinsic **20×32** form (`fill="#F2FF59"`). PR #13283 had replaced it
with a raw Figma export (`preserveAspectRatio="none"`, `width="100%"
height="100%"`, `fill="var(--fill-0, …)"`). Every consumer loads it as a
bare `<img src>` and relies on the intrinsic size plus
`scale-*`/`rotate` classes — with no intrinsic dimensions the connector
expanded to fill its container and distorted.

## Review Focus

- The overwrite originated in the first commit of #13283's stack and
rode through the squash merge; nothing in that PR actually referenced
this file (the MCP page uses the separate `NodeUnionIcon.vue`), so
restoring the shared asset fixes all consumers (`BuildWhatSection`,
`ProductShowcaseSection`, `OurValuesSection`, `GalleryDetailModal`)
without touching the MCP page.
- `apps/website/dist/icons/node-link.svg` is stale build output and
regenerates on the next `pnpm build`.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions <github-actions@github.com>
2026-07-02 13:07:00 +00:00
Benjamin Lu
2ec2a0e091 feat: attribute payment intent through paywall, checkout, and top-up telemetry (#13363)
## Summary

Answers "why did this user want to pay?" by capturing the triggering
product moment at every paywall/upsell entry point and carrying it
through checkout and success telemetry.

## Changes

- **What**:
- Widen `SubscriptionDialogReason` from 4 coarse values to 13 grounded
intent sources (`subscribe_to_run`, `upgrade_to_add_credits`,
`invite_member_upsell`, `settings_billing_panel`, etc.)
- Fire `app:subscription_required_modal_opened` from
`useSubscriptionDialog` (the choke point all dialog variants pass
through) — the workspace/unified path previously emitted nothing; remove
the now-duplicate emitters in `useSubscription` and
`usePricingTableUrlLoader`
- Add `payment_intent_source` to
`BeginCheckoutMetadata`/`SubscriptionSuccessMetadata`, threaded via the
existing `reason` prop: dialog → `PricingTable` →
`performSubscriptionCheckout` → pending-attempt record, so legacy
`app:monthly_subscription_succeeded` carries intent alongside
`checkout_attempt_id`
- Fire `begin_checkout` on the workspace checkout path
(`useSubscriptionCheckout`, personal + team confirm) and the team
deep-link util — both previously emitted nothing; `tier` widened to
`TierKey | 'team'`
- Implement `trackBeginCheckout` in `PostHogTelemetryProvider` (was
GTM/host-only, so `begin_checkout` never reached PostHog)
- Thread `showSubscriptionDialog(options)` through the billing-context
adapters and pass a reason at ~14 call sites; add `source` to
`app:add_api_credit_button_clicked`

## Review Focus

- `modal_opened` now fires once per dialog actually shown, so a
free-tier user clicking Upgrade emits two events (free-tier dialog, then
pricing table) where the legacy path emitted one
- Intent is threaded explicitly via props/params rather than shared
state; `useSubscriptionCheckout` gained an optional second parameter
2026-07-02 03:11:21 +00:00
Mobeen Abdullah
9cf5c9a93f refactor(website): tidy customer story review nits (#13324)
## Summary

Small follow-up to #13289 applying two non-blocking review nits from
Alex's review.

## Changes

- **What**: drop the redundant `before:content-['']` on the
customer-story list bullet (Tailwind emits the empty `content`
automatically once another `before:` utility is present), and rename
`HEADER_OFFSET` to `HEADER_OFFSET_PX` in `ArticleNav` so the scroll
constants use consistent unit suffixes.

## Review Focus

Both changes are cosmetic with no behavior change. Confirmed in the
browser that the list bullet still renders identically (6px yellow dot)
without the explicit `content` utility.

## Notes from the #13289 review (left as-is here, open to discussion)

Three other comments from the review are intentionally not changed in
this PR; reasoning below so the decisions are on record:

- **`Category` type in `ArticleNav`**: kept the `ComponentProps<typeof
CategoryNav>` derivation. AGENTS.md says to derive component types via
`vue-component-type-helpers` rather than redefining them, so the current
form follows the styleguide. Happy to switch to a plain named type if
preferred.
- **Section ids in frontmatter vs the body `<Section>`**: kept the
`customers.content.test.ts` parity test. The short TOC labels live only
in frontmatter and Astro can't introspect the rendered MDX body to build
the nav, so the frontmatter `sections` list and the body anchor ids
can't be trivially deduplicated. A real fix would need a remark plugin
(larger, separate change). The test guards against silent drift in the
meantime.
- **`nextStory` throw**: left as a fail-loud, build-time invariant. The
slug always comes from the same `getStaticPaths` collection, so the
throw is effectively unreachable; it surfaces a future-refactor bug
loudly instead of linking to the wrong story.
2026-07-01 12:45:24 +00:00
jaeone94
9e5fb67b76 Show app mode run validation warning (#12557)
## Summary
Adds an app mode validation warning so users can see when a workflow has
errors before running and jump directly back to graph mode to review
them.

## Changes
- **What**: Adds a reusable app mode warning banner above the Run button
when the execution error store reports workflow errors, including
validation and missing asset states.
- **What**: Reuses the existing graph-error navigation flow so the
warning action switches out of app mode and opens the Errors panel in
graph mode.
- **What**: Updates the app mode Run button icon and accessible label in
the warning state while keeping the Run action non-blocking.
- **What**: Adds unit coverage for the warning render/accessibility
state and an E2E flow that triggers a validation failure, dismisses the
overlay, and opens graph errors from the app mode warning.
- **Breaking**: None.
- **Dependencies**: None.

## Review Focus
The warning intentionally mirrors graph mode behavior: it surfaces the
error state but does not prevent the user from clicking Run. This avoids
turning display-level validation signals into hard execution blockers.

The warning is driven by the existing `hasAnyError` aggregate, so
missing nodes, missing models, and missing media are included alongside
prompt/node/execution errors.

## Tests
- `pnpm format`
- `pnpm lint`
- `pnpm typecheck`
- `pnpm test:unit`
- `pnpm knip`
- `pnpm test:browser:local
browser_tests/tests/appModeValidationWarning.spec.ts`

## Screenshots

<img width="461" height="994" alt="스크린샷 2026-06-25 오후 7 00 55"
src="https://github.com/user-attachments/assets/f8fc20bf-d572-46b5-9fa4-312e7c4c8076"
/>
2026-07-01 15:24:45 +09:00
115 changed files with 5514 additions and 2077 deletions

View File

@@ -0,0 +1,154 @@
# Runs the custom-node regression suite against a backend that has the manifest
# packs actually installed, so the load/run tiers execute for real. This is a
# GATING check: if a pack fails to install or any tier is skipped, the job goes
# red - a regression gate that let a broken pack through as a "skip" would be
# pointless. Mark `custom-nodes-e2e` as a required status check in branch
# protection to block merges on failure.
name: 'CI: Tests Custom Nodes'
on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
push:
branches: [main, master]
merge_group:
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Path gating lives here, not in a trigger-level `paths:` filter: a required
# check gated by trigger paths never creates a check run on an unrelated PR
# and leaves branch protection stuck Pending. A job-level `if:` still creates
# the check and marks it Skipped (= passing). Mirrors ci-tests-unit.yaml.
changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
should-run: ${{ steps.changes.outputs.should-run }}
steps:
- uses: actions/checkout@v6
- id: changes
uses: ./.github/actions/changes-filter
custom-nodes-e2e:
needs: changes
# Run only when non-docs code changed AND the PR is same-repo. Fork PRs can
# edit the manifest's repo/pin URLs, and this job clones and pip-installs
# whatever they point at (setup.py runs at install time), so an untrusted
# fork must not be able to aim the clone at an attacker-controlled repo.
# Fork PRs still get the environment-agnostic coverage via the main e2e
# shards. A skipped job counts as passing, so this stays required-safe.
if: >-
needs.changes.outputs.should-run == 'true' &&
(github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository)
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
with:
include_build_step: true
- name: Setup Playwright
uses: ./.github/actions/setup-playwright
# Checks out ComfyUI, installs Python/torch/requirements and ComfyUI_devtools.
# launch_server:false so we can add the manifest packs before booting.
- name: Setup ComfyUI server
uses: ./.github/actions/setup-comfyui-server
with:
launch_server: 'false'
# Install every pack the manifest declares (DRY: a new pack row installs
# itself here, no workflow change). A clone or dependency failure fails the
# job - if a pack can't be installed, its coverage can't run, and that is a
# gate failure, not something to paper over. The `jq | while` pipe hides
# failures in a subshell, so read into an array and loop with `set -e`.
- name: Install manifest custom nodes
shell: bash
run: |
set -euo pipefail
# Pin the CPU torch stack that setup-comfyui-server installed so no
# pack's requirements.txt can pull a GPU/incompatible torch onto this
# --cpu runner. A pack that genuinely needs a different torch fails
# the constrained install loudly rather than silently swapping it.
pip freeze | grep -iE '^(torch|torchvision|torchaudio)==' \
> /tmp/torch-constraints.txt || true
manifest=browser_tests/fixtures/data/customNodeManifest.json
mapfile -t entries < <(jq -c '.[]' "$manifest")
for entry in "${entries[@]}"; do
repo=$(jq -r '.repo' <<<"$entry")
pin=$(jq -r '.pin' <<<"$entry")
name=$(basename "$repo")
dir="ComfyUI/custom_nodes/$name"
echo "::group::install $name"
git clone --depth 1 "$repo" "$dir"
if [ -n "$pin" ]; then
git -C "$dir" fetch --depth 1 origin "$pin"
git -C "$dir" checkout "$pin"
fi
if [ -f "$dir/requirements.txt" ]; then
pip install -r "$dir/requirements.txt" -c /tmp/torch-constraints.txt
fi
echo "::endgroup::"
done
# The VHS run-tier workflow reads input/plain_video.mp4.
- name: Stage run-tier assets
shell: bash
run: cp browser_tests/assets/plain_video.mp4 ComfyUI/input/plain_video.mp4
# --cache-none so retried run-tier tests re-execute every node (a cached
# node emits no `executing` event and would false-fail PARTIAL).
- name: Start ComfyUI server
shell: bash
working-directory: ComfyUI
run: |
python main.py --cpu --multi-user --cache-none --front-end-root ../dist &
wait-for-it --service 127.0.0.1:8188 -t 600
- name: Run custom-node suite
env:
PLAYWRIGHT_JSON_OUTPUT_NAME: custom-nodes-results.json
run: |
pnpm exec playwright test browser_tests/tests/customNodes/ \
--project=chromium --reporter=list,json
# A skip here means a pack or devtools did not load: on this backend every
# tier is meant to run, so a skip is a gate failure, not an honest pass.
- name: Forbid skipped tests
if: always()
shell: bash
run: |
set -euo pipefail
skipped=$(jq '.stats.skipped' custom-nodes-results.json)
echo "skipped tests: $skipped"
if [ "$skipped" != "0" ]; then
echo "::error::$skipped test(s) skipped - a manifest pack or devtools failed to load; skips are not acceptable in the gating job"
# Recurse so specs nested under describe() blocks are found, and
# print only the specs that actually skipped.
jq -r '.. | objects
| select(has("title") and has("tests"))
| select(any(.tests[]?; .status == "skipped"))
| .title' custom-nodes-results.json | sort -u | head -40
exit 1
fi
- name: Upload Playwright report
if: always()
uses: actions/upload-artifact@v6
with:
name: playwright-report-custom-nodes
path: playwright-report/
retention-days: 7
if-no-files-found: warn

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

View File

@@ -1,3 +1,3 @@
<svg preserveAspectRatio="none" width="100%" height="100%" overflow="visible" style="display: block;" viewBox="0 0 20 32" fill="none" xmlns="http://www.w3.org/2000/svg">
<path id="Vector" d="M20 32V0C20 5.39616 15.5172 9.78053 10 9.78053C4.48276 9.78053 0 5.416 0 0V32C0 26.6038 4.48276 22.2195 10 22.2195C15.5172 22.2195 20 26.6038 20 32Z" fill="var(--fill-0, #F2FF59)"/>
<svg width="20" height="32" viewBox="0 0 20 32" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M20 32V0C20 5.39616 15.5172 9.78053 10 9.78053C4.48276 9.78053 0 5.416 0 0V32C0 26.6038 4.48276 22.2195 10 22.2195C15.5172 22.2195 20 26.6038 20 32Z" fill="#F2FF59"/>
</svg>

Before

Width:  |  Height:  |  Size: 380 B

After

Width:  |  Height:  |  Size: 279 B

View File

@@ -15,7 +15,7 @@ const { categories } = defineProps<{
const activeSection = ref(categories[0]?.value ?? '')
const HEADER_OFFSET = -144
const HEADER_OFFSET_PX = -144
const BOTTOM_THRESHOLD_PX = 4
const SCROLL_SAFETY_MS = 1500
@@ -52,7 +52,7 @@ function scrollToSection(id: string) {
const el = document.getElementById(id)
if (el) {
scrollTo(el, {
offset: HEADER_OFFSET,
offset: HEADER_OFFSET_PX,
duration: 0.8,
immediate: prefersReducedMotion(),
onComplete: clearScrollLock

View File

@@ -1,5 +1,5 @@
<li
class="flex items-start gap-2 text-primary-comfy-canvas before:mt-1.5 before:size-1.5 before:shrink-0 before:rounded-full before:bg-primary-comfy-yellow before:content-['']"
class="flex items-start gap-2 text-primary-comfy-canvas before:mt-1.5 before:size-1.5 before:shrink-0 before:rounded-full before:bg-primary-comfy-yellow"
>
<slot />
</li>

View File

@@ -123,6 +123,15 @@ Browser tests in this project follow a specific organization pattern:
- **Utilities**: Located in `utils/` - Common utility functions
- `litegraphUtils.ts` - Utilities for working with LiteGraph nodes
### Custom-node regression suite
`tests/customNodes/` holds the manifest-driven suite that proves community
custom-node packs load, render in both renderers (LiteGraph canvas and Vue
Nodes 2.0), and execute real workflows. It has its own prerequisites, pnpm
scripts (`pnpm test:custom-nodes` and per-pack variants), and a
one-JSON-row process for adding packs - see
[tests/customNodes/README.md](tests/customNodes/README.md).
## Writing Effective Tests
When writing new tests, follow these patterns:

View File

@@ -0,0 +1,53 @@
{
"last_node_id": 2,
"last_link_id": 1,
"nodes": [
{
"id": 1,
"type": "PrimitiveInt",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 80 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "PrimitiveInt"
},
"widgets_values": [42, "fixed"]
},
{
"id": 2,
"type": "PreviewAny",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 1
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
}
],
"links": [[1, 1, 0, 2, 0, "INT"]],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,60 @@
{
"last_node_id": 2,
"last_link_id": 1,
"nodes": [
{
"id": 1,
"type": "StringFunction|pysssss",
"pos": { "0": 20, "1": 60 },
"size": { "0": 300, "1": 240 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "STRING",
"type": "STRING",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "StringFunction|pysssss"
},
"widgets_values": ["append", "yes", "hello", " world", ""]
},
{
"id": 2,
"type": "ShowText|pysssss",
"pos": { "0": 380, "1": 60 },
"size": { "0": 220, "1": 80 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "text",
"type": "STRING",
"link": 1
}
],
"outputs": [
{
"name": "STRING",
"type": "STRING",
"links": null,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ShowText|pysssss"
}
}
],
"links": [[1, 1, 0, 2, 0, "STRING"]],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,61 @@
{
"last_node_id": 2,
"last_link_id": 1,
"nodes": [
{
"id": 1,
"type": "SimpleMathInt+",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 60 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "SimpleMathInt+"
},
"widgets_values": [5]
},
{
"id": 2,
"type": "DisplayAny",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 80 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "input",
"type": "*",
"link": 1
}
],
"outputs": [
{
"name": "STRING",
"type": "STRING",
"links": null,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "DisplayAny"
},
"widgets_values": ["raw value"]
}
],
"links": [[1, 1, 0, 2, 0, "INT"]],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,98 @@
{
"last_node_id": 4,
"last_link_id": 2,
"nodes": [
{
"id": 1,
"type": "ImpactInt",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 60 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ImpactInt"
},
"widgets_values": [42]
},
{
"id": 2,
"type": "PreviewAny",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 1
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
},
{
"id": 3,
"type": "ImpactFloat",
"pos": { "0": 20, "1": 220 },
"size": { "0": 250, "1": 60 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "FLOAT",
"type": "FLOAT",
"links": [2],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ImpactFloat"
},
"widgets_values": [3.14]
},
{
"id": 4,
"type": "PreviewAny",
"pos": { "0": 340, "1": 220 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 2
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
}
],
"links": [
[1, 1, 0, 2, 0, "INT"],
[2, 3, 0, 4, 0, "FLOAT"]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,98 @@
{
"last_node_id": 4,
"last_link_id": 2,
"nodes": [
{
"id": 1,
"type": "INTConstant",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 60 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "value",
"type": "INT",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "INTConstant"
},
"widgets_values": [42]
},
{
"id": 2,
"type": "PreviewAny",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 1
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
},
{
"id": 3,
"type": "FloatConstant",
"pos": { "0": 20, "1": 220 },
"size": { "0": 250, "1": 60 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "value",
"type": "FLOAT",
"links": [2],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "FloatConstant"
},
"widgets_values": [3.14]
},
{
"id": 4,
"type": "PreviewAny",
"pos": { "0": 340, "1": 220 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 2
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
}
],
"links": [
[1, 1, 0, 2, 0, "INT"],
[2, 3, 0, 4, 0, "FLOAT"]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,53 @@
{
"last_node_id": 2,
"last_link_id": 1,
"nodes": [
{
"id": 1,
"type": "Seed (rgthree)",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 130 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "SEED",
"type": "INT",
"links": [1],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "Seed (rgthree)"
},
"widgets_values": [12345]
},
{
"id": 2,
"type": "Display Any (rgthree)",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 1
}
],
"outputs": [],
"properties": {
"Node name for S&R": "Display Any (rgthree)"
}
}
],
"links": [[1, 1, 0, 2, 0, "INT"]],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,107 @@
{
"last_node_id": 3,
"last_link_id": 2,
"nodes": [
{
"id": 1,
"type": "VHS_LoadVideoPath",
"pos": { "0": 20, "1": 60 },
"size": { "0": 320, "1": 260 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": null
},
{
"name": "frame_count",
"type": "INT",
"links": null
},
{
"name": "audio",
"type": "AUDIO",
"links": null
},
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"links": [1],
"slot_index": 3
}
],
"properties": {
"Node name for S&R": "VHS_LoadVideoPath"
},
"widgets_values": ["input/plain_video.mp4", 0, 0, 0, 0, 0, 1]
},
{
"id": 2,
"type": "VHS_VideoInfo",
"pos": { "0": 400, "1": 60 },
"size": { "0": 240, "1": 260 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"link": 1
}
],
"outputs": [
{
"name": "source_fps🟨",
"type": "FLOAT",
"links": [2],
"slot_index": 0
},
{ "name": "source_frame_count🟨", "type": "INT", "links": null },
{ "name": "source_duration🟨", "type": "FLOAT", "links": null },
{ "name": "source_width🟨", "type": "INT", "links": null },
{ "name": "source_height🟨", "type": "INT", "links": null },
{ "name": "loaded_fps🟦", "type": "FLOAT", "links": null },
{ "name": "loaded_frame_count🟦", "type": "INT", "links": null },
{ "name": "loaded_duration🟦", "type": "FLOAT", "links": null },
{ "name": "loaded_width🟦", "type": "INT", "links": null },
{ "name": "loaded_height🟦", "type": "INT", "links": null }
],
"properties": {
"Node name for S&R": "VHS_VideoInfo"
}
},
{
"id": 3,
"type": "PreviewAny",
"pos": { "0": 700, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "source",
"type": "*",
"link": 2
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewAny"
}
}
],
"links": [
[1, 1, 3, 2, 0, "VHS_VIDEOINFO"],
[2, 2, 0, 3, 0, "FLOAT"]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,103 @@
{
"last_node_id": 3,
"last_link_id": 2,
"nodes": [
{
"id": 1,
"type": "Constant Number",
"pos": { "0": 20, "1": 60 },
"size": { "0": 250, "1": 100 },
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "NUMBER",
"type": "NUMBER",
"links": [1],
"slot_index": 0
},
{
"name": "FLOAT",
"type": "FLOAT",
"links": null,
"slot_index": 1
},
{
"name": "INT",
"type": "INT",
"links": null,
"slot_index": 2
}
],
"properties": {
"Node name for S&R": "Constant Number"
},
"widgets_values": ["integer", 7]
},
{
"id": 2,
"type": "Number to Text",
"pos": { "0": 340, "1": 60 },
"size": { "0": 220, "1": 60 },
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "number",
"type": "NUMBER",
"link": 1
}
],
"outputs": [
{
"name": "STRING",
"type": "STRING",
"links": [2],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "Number to Text"
}
},
{
"id": 3,
"type": "Text to Console",
"pos": { "0": 640, "1": 60 },
"size": { "0": 250, "1": 80 },
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "text",
"type": "STRING",
"link": 2
}
],
"outputs": [
{
"name": "STRING",
"type": "STRING",
"links": null,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "Text to Console"
},
"widgets_values": ["Text Output"]
}
],
"links": [
[1, 1, 0, 2, 0, "NUMBER"],
[2, 2, 0, 3, 0, "STRING"]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}

View File

@@ -0,0 +1,45 @@
{
"last_node_id": 9,
"last_link_id": 9,
"nodes": [
{
"id": 9,
"type": "SaveImage",
"pos": {
"0": 64,
"1": 104
},
"size": {
"0": 210,
"1": 58
},
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": null
}
],
"outputs": [],
"properties": {},
"widgets_values": ["ComfyUI"]
}
],
"links": [],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 1,
"offset": [0, 0]
},
"linearData": {
"inputs": [],
"outputs": ["9"]
}
},
"version": 0.4
}

View File

@@ -268,8 +268,16 @@ export class ComfyPage {
data: { username }
})
if (resp.status() !== 200)
throw new Error(`Failed to create user: ${await resp.text()}`)
if (resp.status() !== 200) {
const body = await resp.text()
// Persistent backends (Comfy Desktop server user storage) keep the user
// across runs and do not list it via GET /api/users, so a duplicate means
// it already exists. Returns the username since the generated id is not
// retrievable here; only reached on single-user / default-resolving backends.
if (resp.status() === 400 && body.includes('Duplicate username.'))
return username
throw new Error(`Failed to create user: ${body}`)
}
return await resp.json()
}

View File

@@ -0,0 +1,141 @@
import type { Page } from '@playwright/test'
import type { ObjectInfo } from '@e2e/fixtures/customNode/objectInfoValidator'
import type {
ExecutionError,
PromptEvent,
RunResult
} from '@e2e/fixtures/customNode/runResult'
import { classifyRun } from '@e2e/fixtures/customNode/runResult'
interface RawEvent {
type: string
node?: string | null
exception_type?: string
node_id?: string
node_type?: string
traceback?: string[]
}
const TERMINAL = [
'execution_success',
'execution_error',
'execution_interrupted'
]
function toPromptEvent(raw: RawEvent): PromptEvent {
if (raw.type === 'executing')
return { type: 'executing', node: raw.node ?? null }
if (raw.type === 'execution_error' || raw.type === 'execution_interrupted') {
const error: ExecutionError = {
exceptionType: raw.exception_type,
nodeId: raw.node_id,
nodeType: raw.node_type,
traceback: raw.traceback
}
return { type: raw.type, error }
}
return { type: raw.type as 'execution_start' | 'execution_success' }
}
/**
* Drives a real ComfyUI backend through the running frontend. The verdict logic
* lives in the pure `classifyRun`; this class is only the in-page IO plumbing.
*/
export class LocalDesktopTarget {
async getObjectInfo(page: Page): Promise<ObjectInfo> {
return await page.evaluate(async () => {
const defs = await window.app!.api.getNodeDefs()
const out: Record<
string,
{ input?: { required?: Record<string, unknown> } }
> = {}
for (const [name, def] of Object.entries(defs)) {
const required = (
def as { input?: { required?: Record<string, unknown> } }
).input?.required
out[name] = { input: { required } }
}
return out
})
}
async runWorkflow(
page: Page,
opts: { expectedNodeIds: string[]; timeoutMs: number }
): Promise<RunResult> {
await page.evaluate(
(types) => {
const sink = window as unknown as {
__cnEvents: RawEvent[]
__cnTapInstalled?: boolean
}
sink.__cnEvents = []
if (sink.__cnTapInstalled) return
sink.__cnTapInstalled = true
for (const type of types)
(window.app!.api as EventTarget).addEventListener(
type,
(event: Event) => {
const detail: unknown = (event as CustomEvent).detail
// `executing` dispatches a bare node-id string (api.ts
// dispatchCustomEvent('executing', msg.data.node)); the other
// events dispatch object payloads.
sink.__cnEvents.push(
detail !== null && typeof detail === 'object'
? { type, ...(detail as Record<string, unknown>) }
: { type, node: (detail as string | undefined) ?? null }
)
}
)
},
['execution_start', ...TERMINAL, 'executing']
)
// Browser path: app.queuePrompt runs graphToPrompt internally. Do NOT call
// app.api.queuePrompt, which submits an already-serialized (empty) prompt.
// A backend validation reject emits NO events at all - without checking the
// queue result, every rejected prompt would burn the full wait and
// masquerade as TIMEOUT. But a false return is NOT proof of a backend
// reject: pack JS hooking the queue path (Impact's wildcard processing)
// can refuse transiently right after nodes are created. Retry once -
// genuine backend rejects are deterministic and fail both attempts.
let queued = await page.evaluate(() => window.app!.queuePrompt(0))
if (queued === false) {
await page.evaluate(
() => new Promise((resolve) => setTimeout(resolve, 250))
)
queued = await page.evaluate(() => window.app!.queuePrompt(0))
if (queued === false)
return { outcome: 'VALIDATION_FAIL', executedNodes: [] }
}
await page
.waitForFunction(
(terminal) => {
const events =
(window as unknown as { __cnEvents?: { type: string }[] })
.__cnEvents ?? []
return events.some((event) => terminal.includes(event.type))
},
TERMINAL,
{ timeout: opts.timeoutMs }
)
.catch((error: unknown) => {
// Only a Playwright wait timeout means "no terminal event"; surface any
// other fault instead of masquerading it as a run TIMEOUT.
if (error instanceof Error && error.name === 'TimeoutError') return
throw error
})
const raw = await page.evaluate(
() => (window as unknown as { __cnEvents?: RawEvent[] }).__cnEvents ?? []
)
const timedOut = !raw.some((event) => TERMINAL.includes(event.type))
return classifyRun({
events: raw.map(toPromptEvent),
expectedNodeIds: opts.expectedNodeIds,
timedOut
})
}
}

View File

@@ -0,0 +1,108 @@
// Classifies which registered nodes can execute with zero hand-authored
// fixtures: every required input satisfiable by a widget default, plus an
// observable terminus (the node is an OUTPUT_NODE, or an output we can wire
// to core PreviewAny). Those run for real on the backend; the rest are
// recorded with the reason they cannot run alone - never silently dropped.
import type { RawNodeDef } from './typePairing'
export type AutoRunClass =
// Runs alone: widgets cover every required input, terminus available.
| 'AUTO_RUNNABLE'
// A required input is a socket (custom type or forceInput); execution
// needs wiring, which the curated run workflows cover instead.
| 'NEEDS_WIRES'
// A required combo has zero options on this backend - a model/file scan
// came back empty, so no valid default exists (CPU gate has no models).
| 'NEEDS_MODELS'
// Widgets suffice but nothing observable to queue: no outputs and not an
// OUTPUT_NODE, so the executor would never touch it.
| 'NO_SINK'
export interface AutoRunVerdict {
key: string
verdict: AutoRunClass
// Set for AUTO_RUNNABLE: wire output 0 to PreviewAny (false = the node is
// its own OUTPUT_NODE terminus and runs standalone).
needsPreviewSink?: boolean
reason: string
}
const WIDGET_TYPES = new Set(['INT', 'FLOAT', 'STRING', 'BOOLEAN'])
type InputSpec = [unknown, Record<string, unknown>?] | unknown
function classifyInput(
name: string,
spec: InputSpec
): 'widget' | 'socket' | 'empty-combo' {
const specArray = Array.isArray(spec) ? spec : [spec]
const rawType = specArray[0]
const options = specArray[1] as { forceInput?: boolean } | undefined
if (Array.isArray(rawType))
return rawType.length > 0 ? 'widget' : 'empty-combo'
if (typeof rawType !== 'string') return 'socket'
if (options?.forceInput) return 'socket'
return WIDGET_TYPES.has(rawType) ? 'widget' : 'socket'
}
export function classifyAutoRunnable(
key: string,
def: RawNodeDef & { output_node?: boolean }
): AutoRunVerdict {
for (const [name, spec] of Object.entries(def.input?.required ?? {})) {
const kind = classifyInput(name, spec)
if (kind === 'socket')
return {
key,
verdict: 'NEEDS_WIRES',
reason: `required input "${name}" is a socket`
}
if (kind === 'empty-combo')
return {
key,
verdict: 'NEEDS_MODELS',
reason: `required combo "${name}" has no options on this backend`
}
}
if (def.output_node === true)
return {
key,
verdict: 'AUTO_RUNNABLE',
needsPreviewSink: false,
reason: 'widgets satisfy all required inputs; node is its own terminus'
}
if ((def.output ?? []).length > 0)
return {
key,
verdict: 'AUTO_RUNNABLE',
needsPreviewSink: true,
reason: 'widgets satisfy all required inputs; output 0 -> PreviewAny'
}
return {
key,
verdict: 'NO_SINK',
reason: 'no outputs and not an OUTPUT_NODE - nothing observable to queue'
}
}
export function planAutoRuns(
defs: Record<string, RawNodeDef & { output_node?: boolean }>,
packNodeKeys: string[]
): AutoRunVerdict[] {
return packNodeKeys.map((key) => classifyAutoRunnable(key, defs[key]))
}
// Independent single-node chains per prompt; one bad node fails its batch,
// not the whole tier, and the executed-set check still attributes per node.
export function batchAutoRunnable(
verdicts: AutoRunVerdict[],
batchSize: number
): AutoRunVerdict[][] {
const runnable = verdicts.filter(
(verdict) => verdict.verdict === 'AUTO_RUNNABLE'
)
const batches: AutoRunVerdict[][] = []
for (let offset = 0; offset < runnable.length; offset += batchSize)
batches.push(runnable.slice(offset, offset + batchSize))
return batches
}

View File

@@ -0,0 +1,124 @@
import { readFileSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
const MANIFEST_PATH = fileURLToPath(
new URL('../data/customNodeManifest.json', import.meta.url)
)
const VALID_TIERS = ['load', 'run', 'connectivity', 'io'] as const
type CustomNodeTier = (typeof VALID_TIERS)[number]
export interface CustomNodeManifestEntry {
pack: string
repo: string
pin: string
tiers: CustomNodeTier[]
// Frontend-format workflow (path relative to browser_tests/) loaded and queued
// by the run/io tiers; empty or absent file = tier skips. Run the backend with
// --cache-none, or repeat runs classify PARTIAL when cached nodes skip executing.
workflow: string
// Runtime class_type / object_info keys, NOT Python class names (e.g. rgthree
// registers "Power Primitive (rgthree)", not RgthreePowerPrimitive).
expectedNodes: string[]
requiresGpu: boolean
requiresModels: string[]
timeoutMs: number
// Optional; absent means true. Set false ONLY with evidence that the pack's
// nodes fail to mount under Vue Nodes 2.0 (probe it - a README grumble is
// not evidence). When false, renderer-specific Vue assertions are not
// applied to this pack: its tests still run and pass their LiteGraph-canvas
// assertions, so the zero-skip gate is preserved.
vueNodesCompatible?: boolean
// Per-node Vue Nodes 2.0 incompatibility ledger: node key -> reason with
// evidence (author statement or reproduced mount failure; a run failure
// alone is NOT evidence - it may be our own fixture error). Ledgered nodes
// keep every canvas assertion; only their Vue mount assertion is withheld.
// A key that stops existing on the backend fails the suite, so entries
// cannot silently rot.
vueIncompatibleNodes?: Record<string, string>
// Auto-run baseline: nodes observed unable to execute standalone on a bare
// backend (validation reject or execution error on pure defaults - empty
// expressions, empty folders, no webcam). Asserted BOTH ways: a failing
// node missing from this list is a regression, and a listed node that now
// runs clean must be removed. Weak-signal territory by design - a wrong
// entry here means a fixture gap, never a skipped test.
cannotRunAlone?: string[]
}
function assertEntry(entry: CustomNodeManifestEntry, index: number): void {
const missing: string[] = []
if (typeof entry.pack !== 'string' || entry.pack.length === 0)
missing.push('pack')
// CI clones from repo, so an empty value must fail here, not mid-clone.
// pin stays optional ("" = default branch head).
if (typeof entry.repo !== 'string' || entry.repo.length === 0)
missing.push('repo')
// workflow may be an empty string until the pack gains a run-tier fixture.
if (typeof entry.workflow !== 'string') missing.push('workflow')
// A run-tier row with no workflow would otherwise skip locally, leaving
// only CI's skip gate to notice the lost coverage. Fail at load instead.
else if (
entry.workflow === '' &&
Array.isArray(entry.tiers) &&
entry.tiers.includes('run')
)
missing.push('workflow (required when tiers includes "run")')
if (!Array.isArray(entry.expectedNodes) || entry.expectedNodes.length === 0)
missing.push('expectedNodes')
if (!Array.isArray(entry.tiers) || entry.tiers.length === 0)
missing.push('tiers')
// A typo like "connectivty" would otherwise pass and silently drop that
// tier's coverage - the exact drift this manifest exists to catch.
else if (entry.tiers.some((tier) => !VALID_TIERS.includes(tier)))
missing.push(`tiers (unknown value; allowed: ${VALID_TIERS.join(', ')})`)
if (!Array.isArray(entry.requiresModels)) missing.push('requiresModels')
if (typeof entry.requiresGpu !== 'boolean') missing.push('requiresGpu')
if (!Number.isFinite(entry.timeoutMs) || entry.timeoutMs <= 0)
missing.push('timeoutMs')
if (
entry.vueNodesCompatible !== undefined &&
typeof entry.vueNodesCompatible !== 'boolean'
)
missing.push('vueNodesCompatible')
if (
entry.vueIncompatibleNodes !== undefined &&
(typeof entry.vueIncompatibleNodes !== 'object' ||
entry.vueIncompatibleNodes === null ||
Array.isArray(entry.vueIncompatibleNodes) ||
Object.values(entry.vueIncompatibleNodes).some(
(reason) => typeof reason !== 'string' || reason.length === 0
))
)
missing.push('vueIncompatibleNodes (node key -> non-empty reason string)')
if (
entry.cannotRunAlone !== undefined &&
(!Array.isArray(entry.cannotRunAlone) ||
entry.cannotRunAlone.some(
(key) => typeof key !== 'string' || key.length === 0
) ||
new Set(entry.cannotRunAlone).size !== entry.cannotRunAlone.length)
)
missing.push('cannotRunAlone (unique non-empty node keys)')
if (missing.length > 0)
throw new Error(
`custom-node manifest entry ${index} (${entry.pack ?? '?'}) missing: ${missing.join(', ')}`
)
}
// Renderer passes for the load tier: LiteGraph canvas always, Vue Nodes 2.0
// unless the pack declares itself incompatible. Conditional coverage, never a
// test.skip - the caller still runs and gates on the returned passes.
export function rendererPassesFor(
entry: Pick<CustomNodeManifestEntry, 'vueNodesCompatible'>
): boolean[] {
return entry.vueNodesCompatible === false ? [false] : [false, true]
}
export function loadManifest(): CustomNodeManifestEntry[] {
const entries = JSON.parse(
readFileSync(MANIFEST_PATH, 'utf-8')
) as CustomNodeManifestEntry[]
entries.forEach(assertEntry)
return entries
}

View File

@@ -0,0 +1,54 @@
import type { CustomNodeOutcome } from '@e2e/fixtures/customNode/runResult'
interface ObjectInfoNode {
input?: { required?: Record<string, unknown> }
}
export type ObjectInfo = Record<string, ObjectInfoNode>
export interface ApiPromptNode {
id: string
classType: string
inputs: Record<string, unknown>
}
export function expectedNodesPresent(
objectInfo: ObjectInfo,
expectedNodes: string[]
): { present: string[]; missing: string[] } {
const present: string[] = []
const missing: string[] = []
for (const name of expectedNodes) {
if (name in objectInfo) present.push(name)
else missing.push(name)
}
return { present, missing }
}
export interface PreValidationFailure {
outcome: Extract<CustomNodeOutcome, 'MISSING_NODE' | 'VALIDATION_FAIL'>
message: string
}
// Turns an opaque backend 400 into a precise infra error before submit (BE-401):
// every required input declared in object_info must be present in the fixture node.
export function preValidate(
objectInfo: ObjectInfo,
nodes: ApiPromptNode[]
): PreValidationFailure | null {
for (const node of nodes) {
const def = objectInfo[node.classType]
if (!def)
return {
outcome: 'MISSING_NODE',
message: `node ${node.id} ${node.classType} missing from object_info`
}
for (const name of Object.keys(def.input?.required ?? {})) {
if (!(name in node.inputs))
return {
outcome: 'VALIDATION_FAIL',
message: `node ${node.id} ${node.classType} missing required input "${name}"`
}
}
}
return null
}

View File

@@ -0,0 +1,72 @@
export type CustomNodeOutcome =
| 'NOT_INSTALLED'
| 'IMPORT_ERROR'
| 'MISSING_NODE'
| 'VALIDATION_FAIL'
| 'EXECUTION_ERROR'
| 'PARTIAL'
| 'TIMEOUT'
| 'PASS'
export interface ExecutionError {
exceptionType?: string
nodeId?: string
nodeType?: string
traceback?: string[]
}
export type PromptEvent =
| { type: 'execution_start' }
| { type: 'executing'; node: string | null }
| { type: 'execution_success' }
| { type: 'execution_error'; error: ExecutionError }
| { type: 'execution_interrupted'; error?: ExecutionError }
export interface RunResult {
outcome: CustomNodeOutcome
executedNodes: string[]
error?: ExecutionError
}
// `executing` with a non-null node is the only cache-safe "this node actually ran"
// signal: ComfyUI emits it solely for non-cached nodes (execution.py:493), while the
// `executed` message and /history outputs are replayed for cached nodes too.
function executedNodesFrom(events: PromptEvent[]): string[] {
const executed = new Set<string>()
for (const event of events) {
if (event.type === 'executing' && event.node !== null)
executed.add(event.node)
}
return [...executed]
}
export function classifyRun(input: {
events: PromptEvent[]
expectedNodeIds: string[]
timedOut?: boolean
}): RunResult {
const { events, expectedNodeIds, timedOut = false } = input
const executedNodes = executedNodesFrom(events)
if (timedOut) return { outcome: 'TIMEOUT', executedNodes }
const failure = events.find(
(
event
): event is Extract<
PromptEvent,
{ type: 'execution_error' | 'execution_interrupted' }
> =>
event.type === 'execution_error' || event.type === 'execution_interrupted'
)
if (failure)
return { outcome: 'EXECUTION_ERROR', executedNodes, error: failure.error }
if (!events.some((event) => event.type === 'execution_success'))
return { outcome: 'TIMEOUT', executedNodes }
const ranEveryExpected = expectedNodeIds.every((node) =>
executedNodes.includes(node)
)
return { outcome: ranEveryExpected ? 'PASS' : 'PARTIAL', executedNodes }
}

View File

@@ -0,0 +1,216 @@
// Type-driven pairing generator for the connectivity (contract) tier.
// Wildcard `*` slots are excluded from pairing: LiteGraph.isValidConnection
// short-circuits on `*` before the real type compare, so a wildcard link
// proves reachability, not type interop.
export interface RawNodeDef {
input?: {
required?: Record<string, unknown>
optional?: Record<string, unknown>
}
output?: unknown[]
output_name?: string[]
python_module?: string
}
interface NormalizedSlot {
name: string
type: string
}
export interface NormalizedNode {
type: string
pack: string
inputs: NormalizedSlot[]
outputs: NormalizedSlot[]
}
interface SlotRef {
nodeType: string
pack: string
slotName: string
slotType: string
}
export interface PlannedPair {
producer: SlotRef
consumer: SlotRef
}
export interface PairingPlan {
pairs: PlannedPair[]
// No compatible partner in the loaded corpus: a health signal, not a failure.
orphans: Array<SlotRef & { dir: 'in' | 'out' }>
// `*` / empty-typed slots, excluded by design (false confidence).
wildcards: Array<SlotRef & { dir: 'in' | 'out' }>
// COMBO-literal slots, excluded by design: isValidConnection only compares
// the string COMBO while each slot carries its own option set, so a
// type-level pairing proves nothing (a checkpoint dropdown would "connect"
// to a scheduler dropdown). Targeted fixtures cover combo behavior.
combos: Array<SlotRef & { dir: 'in' | 'out' }>
}
// Extends the shared outcome taxonomy (runResult.ts); ORPHAN_TYPE is a
// plan-time skip so it never reaches the executor.
// WIDGET_ONLY_ON_INSTANCE: the pack's own frontend JS rebuilt a declared
// input as a widget-only control, so there is no socket to wire - excluded
// like wildcards, never a failure and never a silent pass.
export type ConnectivityOutcome =
| 'PASS'
| 'CONNECT_REJECTED'
| 'ROUNDTRIP_LOST'
| 'SLOT_CONTRACT_MISMATCH'
| 'WIDGET_ONLY_ON_INSTANCE'
export function packOf(pythonModule: string | undefined): string {
if (pythonModule?.startsWith('custom_nodes.'))
return pythonModule.slice('custom_nodes.'.length)
return 'core'
}
export function isWildcard(type: string): boolean {
return type === '' || type === '*'
}
// COMBO list literals are arrays; their connectable socket type is COMBO.
function slotTypeOf(rawType: unknown): string | null {
if (Array.isArray(rawType)) return 'COMBO'
return typeof rawType === 'string' ? rawType : null
}
function inputSlots(
entries: Record<string, unknown> | undefined
): NormalizedSlot[] {
if (!entries) return []
const slots: NormalizedSlot[] = []
for (const [name, spec] of Object.entries(entries)) {
const specArray = Array.isArray(spec) ? spec : [spec]
const type = slotTypeOf(specArray[0])
if (type === null) continue
const opts = specArray[1] as { socketless?: boolean } | undefined
// socketless = widget only, no slot: not connectable, out of the matrix.
if (opts?.socketless) continue
slots.push({ name, type })
}
return slots
}
export function normalizeNodeDefs(
defs: Record<string, RawNodeDef>
): NormalizedNode[] {
return Object.entries(defs).map(([type, def]) => ({
type,
pack: packOf(def.python_module),
inputs: [
...inputSlots(def.input?.required),
...inputSlots(def.input?.optional)
],
outputs: (def.output ?? []).flatMap((rawType, index) => {
const slotType = slotTypeOf(rawType)
if (slotType === null) return []
// output_name entries can be non-strings (COMBO literals repeat the
// option array); the slot name must stay a string.
const rawName = def.output_name?.[index]
return [
{
name: typeof rawName === 'string' ? rawName : slotType,
type: slotType
}
]
})
}))
}
// Faithful mirror of LiteGraph.isValidConnection (LiteGraphGlobal.ts):
// wildcard/empty always match, comparison is case-insensitive, comma-unions
// match if any member pair matches. The live sweep still connects through the
// REAL validator, so any drift here surfaces as CONNECT_REJECTED, not a
// silent false green.
export function isTypeCompatible(a: string, b: string): boolean {
if (isWildcard(a) || isWildcard(b)) return true
const typeA = a.toLowerCase()
const typeB = b.toLowerCase()
if (typeA === typeB) return true
if (!typeA.includes(',') && !typeB.includes(',')) return false
return typeA
.split(',')
.some((memberA) =>
typeB.split(',').some((memberB) => isTypeCompatible(memberA, memberB))
)
}
function slotRef(node: NormalizedNode, slot: NormalizedSlot): SlotRef {
return {
nodeType: node.type,
pack: node.pack,
slotName: slot.name,
slotType: slot.type
}
}
// One representative compatible edge per slot, deterministically the first
// partner in (nodeType, slotName) order. This bounds cost to O(slots) but
// does NOT prove every pair; a full cross-product is an opt-in deep mode.
export function planPairs(
all: NormalizedNode[],
corpusTypes: string[]
): PairingPlan {
const sorted = [...all].sort((a, b) => a.type.localeCompare(b.type))
const pairable = (slot: NormalizedSlot) =>
!isWildcard(slot.type) && slot.type !== 'COMBO'
const producers: Array<SlotRef> = sorted.flatMap((node) =>
node.outputs.filter(pairable).map((slot) => slotRef(node, slot))
)
const consumers: Array<SlotRef> = sorted.flatMap((node) =>
node.inputs.filter(pairable).map((slot) => slotRef(node, slot))
)
const plan: PairingPlan = {
pairs: [],
orphans: [],
wildcards: [],
combos: []
}
const seen = new Set<string>()
const addPair = (producer: SlotRef, consumer: SlotRef) => {
const key = `${producer.nodeType}.${producer.slotName}->${consumer.nodeType}.${consumer.slotName}`
if (seen.has(key)) return
seen.add(key)
plan.pairs.push({ producer, consumer })
}
const corpus = all.filter((node) => corpusTypes.includes(node.type))
for (const node of corpus) {
for (const slot of node.inputs) {
if (isWildcard(slot.type)) {
plan.wildcards.push({ ...slotRef(node, slot), dir: 'in' })
continue
}
if (slot.type === 'COMBO') {
plan.combos.push({ ...slotRef(node, slot), dir: 'in' })
continue
}
const producer = producers.find((candidate) =>
isTypeCompatible(candidate.slotType, slot.type)
)
if (producer) addPair(producer, slotRef(node, slot))
else plan.orphans.push({ ...slotRef(node, slot), dir: 'in' })
}
for (const slot of node.outputs) {
if (isWildcard(slot.type)) {
plan.wildcards.push({ ...slotRef(node, slot), dir: 'out' })
continue
}
if (slot.type === 'COMBO') {
plan.combos.push({ ...slotRef(node, slot), dir: 'out' })
continue
}
const consumer = consumers.find((candidate) =>
isTypeCompatible(slot.type, candidate.slotType)
)
if (consumer) addPair(slotRef(node, slot), consumer)
else plan.orphans.push({ ...slotRef(node, slot), dir: 'out' })
}
}
return plan
}

View File

@@ -0,0 +1,132 @@
[
{
"pack": "ComfyUI-Impact-Pack",
"repo": "https://github.com/ltdrdata/ComfyUI-Impact-Pack",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/impact_primitives_run.json",
"expectedNodes": ["ImpactInt", "ImpactFloat"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": [
"CLIPSegDetectorProvider",
"ImpactMakeImageBatch",
"ImpactMakeMaskBatch",
"MasksToMaskList",
"NoiseInjectionDetailerHookProvider"
]
},
{
"pack": "ComfyUI-VideoHelperSuite",
"repo": "https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/vhs_video_pipeline_run.json",
"expectedNodes": ["VHS_LoadVideoPath", "VHS_VideoInfo"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 90000,
"cannotRunAlone": [
"VHS_LoadAudio",
"VHS_LoadImagePath",
"VHS_LoadImages",
"VHS_LoadImagesPath",
"VHS_LoadVideoFFmpegPath",
"VHS_LoadVideoPath",
"VHS_SelectLatest"
]
},
{
"pack": "rgthree-comfy",
"repo": "https://github.com/rgthree/rgthree-comfy",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/rgthree_seed_display_run.json",
"expectedNodes": ["Seed (rgthree)", "Display Any (rgthree)"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": ["Image or Latent Size (rgthree)"]
},
{
"pack": "ComfyUI_essentials",
"repo": "https://github.com/cubiq/ComfyUI_essentials",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/essentials_math_display_run.json",
"expectedNodes": ["SimpleMathInt+", "DisplayAny"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": [
"MaskFromList+",
"SimpleMath+",
"SimpleMathDual+",
"SimpleMathFloat+",
"SimpleMathInt+",
"SimpleMathPercent+",
"SimpleMathSlider+",
"SimpleMathSliderLowRes+"
]
},
{
"pack": "ComfyUI-KJNodes",
"repo": "https://github.com/kijai/ComfyUI-KJNodes",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/kjnodes_constants_run.json",
"expectedNodes": ["INTConstant", "FloatConstant"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": [
"CameraPoseVisualizer",
"CreateAudioMask",
"CreateMagicMask",
"CreateVoronoiMask",
"GenerateNoise",
"ImageAndMaskPreview",
"LoadImagesFromFolderKJ",
"LoadVideosFromFolder",
"MaskOrImageToWeight",
"VisualizeCUDAMemoryHistory",
"WebcamCaptureCV2",
"WidgetToString"
]
},
{
"pack": "ComfyUI-Custom-Scripts",
"repo": "https://github.com/pythongosssss/ComfyUI-Custom-Scripts",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/customscripts_string_show_run.json",
"expectedNodes": ["StringFunction|pysssss", "ShowText|pysssss"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": ["LoadText|pysssss", "MathExpression|pysssss"]
},
{
"pack": "was-node-suite-comfyui",
"repo": "https://github.com/WASasquatch/was-node-suite-comfyui",
"pin": "",
"tiers": ["load", "connectivity", "run"],
"workflow": "assets/customNodes/was_number_text_run.json",
"expectedNodes": ["Constant Number", "Number to Text", "Text to Console"],
"requiresGpu": false,
"requiresModels": [],
"timeoutMs": 30000,
"cannotRunAlone": [
"Bus Node",
"Diffusers Hub Model Down-Loader",
"Image Aspect Ratio",
"Image Batch",
"Latent Batch",
"Mask Batch",
"Mask Rect Area",
"Number Counter",
"Random Number"
]
}
]

View File

@@ -34,6 +34,10 @@ export class AppModeHelper {
public readonly outputPlaceholder: Locator
/** The linear-mode widget list container (visible in app mode). */
public readonly linearWidgets: Locator
/** The validation warning shown above the app mode run button. */
public readonly validationWarning: Locator
/** The action that opens graph mode errors from the validation warning. */
public readonly viewErrorsInGraphButton: Locator
/** The PrimeVue Popover for the image picker (renders with role="dialog"). */
public readonly imagePickerPopover: Locator
/** The Run button in the app mode footer. */
@@ -92,13 +96,19 @@ export class AppModeHelper {
this.outputPlaceholder = this.page.getByTestId(
TestIds.builder.outputPlaceholder
)
this.linearWidgets = this.page.getByTestId('linear-widgets')
this.linearWidgets = this.page.getByTestId(TestIds.linear.widgetContainer)
this.validationWarning = this.page.getByTestId(
TestIds.linear.validationWarning
)
this.viewErrorsInGraphButton = this.validationWarning.getByTestId(
TestIds.linear.viewErrorsInGraph
)
this.imagePickerPopover = this.page
.getByRole('dialog')
.filter({ has: this.page.getByRole('button', { name: 'All' }) })
.first()
this.runButton = this.page
.getByTestId('linear-run-button')
.getByTestId(TestIds.linear.runButton)
.getByRole('button', { name: /run/i })
this.welcome = this.page.getByTestId(TestIds.appMode.welcome)
this.emptyWorkflowText = this.page.getByTestId(

View File

@@ -172,6 +172,9 @@ export const TestIds = {
mobileNavigation: 'linear-mobile-navigation',
mobileWorkflows: 'linear-mobile-workflows',
outputInfo: 'linear-output-info',
runButton: 'linear-run-button',
validationWarning: 'linear-validation-warning',
viewErrorsInGraph: 'linear-view-errors',
widgetContainer: 'linear-widgets'
},
builder: {

View File

@@ -0,0 +1,17 @@
import type { ConsoleMessage, Page } from '@playwright/test'
export function collectConsoleErrors(page: Page): {
errors: string[]
stop: () => void
} {
const errors: string[] = []
const listener = (message: ConsoleMessage) => {
if (message.type() !== 'error') return
// Resource errors ("Failed to load resource: 404") are useless without
// the URL, which lives in the message location, not the text.
const url = message.location().url
errors.push(url ? `${message.text()} [${url}]` : message.text())
}
page.on('console', listener)
return { errors, stop: () => page.off('console', listener) }
}

View File

@@ -0,0 +1,27 @@
import type { ComfyPage } from '@e2e/fixtures/ComfyPage'
import { TestIds } from '@e2e/fixtures/selectors'
// Boot every session with a blank graph (loadBlankWorkflow) instead of the
// bundled default template, whose model references error on a model-less
// harness backend and would trip the zero-visible-errors invariant. The
// backend must run --multi-user (the repo-wide prerequisite for browser
// tests): the fixture then writes these settings to the same per-worker
// user the session reads, on CI and locally alike.
// The shared fixture disables the errors tab to hide missing-model
// indicators in unrelated suites; this suite exists to SEE errors, so every
// error surface stays live.
export const customNodeSuiteSettings = {
'Comfy.TutorialCompleted': false,
'Comfy.RightSidePanel.ShowErrorsTab': true
}
// The tutorial path auto-opens the templates browser over the blank graph.
// Dismiss it deterministically so no window ever shows unexpected UI.
export async function dismissTemplatesDialog(
comfyPage: ComfyPage
): Promise<void> {
const templates = comfyPage.page.getByTestId(TestIds.templates.content)
await templates.waitFor({ state: 'visible' })
await comfyPage.page.keyboard.press('Escape')
await templates.waitFor({ state: 'hidden' })
}

View File

@@ -0,0 +1,16 @@
import type { Locator, Page } from '@playwright/test'
import { TestIds } from '@e2e/fixtures/selectors'
// The app's user-visible error surfaces. A regression run is green only if a
// human looking at the screen would see zero errors - not merely a clean
// console. The harness self-check asserts the overlay IS visible after a
// forced execution error, so these selectors are permanently proven live.
export function errorSurfaces(page: Page): Record<string, Locator> {
return {
errorOverlay: page.getByTestId(TestIds.dialogs.errorOverlay),
errorDialog: page.getByTestId(TestIds.dialogs.errorDialog),
nodeRenderErrors: page.locator('.node-error'),
errorToasts: page.locator('.p-toast-message-error')
}
}

View File

@@ -119,6 +119,14 @@ class NodeSlotReference {
const rawPos = node.getConnectionPos(type === 'input', index)
const convertedPos =
window.app!.canvas.ds!.convertOffsetToCanvas(rawPos)
// convertOffsetToCanvas is canvas-relative; page.mouse needs page
// coordinates. Identical when the canvas sits at (0,0), but custom-node
// JS can inject page chrome above it (e.g. rgthree's progress bar
// shifts the canvas 16px down), which silently turned every slot drag
// into a title-bar node drag.
const rect = window.app!.canvas.canvas.getBoundingClientRect()
convertedPos[0] += rect.left
convertedPos[1] += rect.top
// Debug logging - convert Float64Arrays to regular arrays for visibility
console.warn(

View File

@@ -0,0 +1,106 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import type { NodeError, PromptResponse } from '@/schemas/apiSchema'
import { ExecutionHelper } from '@e2e/fixtures/helpers/ExecutionHelper'
import { enableErrorsOverlay } from '@e2e/fixtures/helpers/ErrorsTabHelper'
import { TestIds } from '@e2e/fixtures/selectors'
const SAVE_IMAGE_NODE_ID = '9'
function buildSaveImageRequiredInputError(): NodeError {
return {
class_type: 'SaveImage',
dependent_outputs: [],
errors: [
{
type: 'required_input_missing',
message: 'Required input is missing: images',
details: '',
extra_info: { input_name: 'images' }
}
]
}
}
test.describe(
'App mode validation warning',
{ tag: ['@ui', '@workflow'] },
() => {
test.beforeEach(async ({ comfyPage }) => {
await enableErrorsOverlay(comfyPage)
await comfyPage.workflow.loadWorkflow('linear-validation-warning')
await comfyPage.appMode.toggleAppMode()
await expect(comfyPage.appMode.linearWidgets).toBeVisible()
})
test('opens graph errors from the app mode validation warning', async ({
comfyPage
}) => {
await expect(comfyPage.appMode.validationWarning).toBeHidden()
const exec = new ExecutionHelper(comfyPage)
await exec.mockValidationFailure({
[SAVE_IMAGE_NODE_ID]: buildSaveImageRequiredInputError()
})
await comfyPage.appMode.runButton.click()
const appModeOverlay = comfyPage.appMode.centerPanel.getByTestId(
TestIds.dialogs.errorOverlay
)
await expect(appModeOverlay).toBeHidden()
await expect(comfyPage.appMode.validationWarning).toBeVisible()
await expect(comfyPage.appMode.validationWarning).toContainText(
/Required input missing/i
)
await expect(comfyPage.appMode.viewErrorsInGraphButton).toBeVisible()
await comfyPage.appMode.viewErrorsInGraphButton.click()
await expect(comfyPage.appMode.linearWidgets).toBeHidden()
await expect(
comfyPage.page.getByTestId(TestIds.propertiesPanel.root)
).toBeVisible()
await expect(
comfyPage.page.getByTestId(TestIds.propertiesPanel.errorsTab)
).toBeVisible()
})
test('keeps the app mode run button enabled when the warning is visible', async ({
comfyPage
}) => {
const exec = new ExecutionHelper(comfyPage)
await exec.mockValidationFailure({
[SAVE_IMAGE_NODE_ID]: buildSaveImageRequiredInputError()
})
await comfyPage.appMode.runButton.click()
await expect(comfyPage.appMode.validationWarning).toBeVisible()
await expect(comfyPage.appMode.runButton).toBeEnabled()
let promptQueued = false
const mockResponse: PromptResponse = {
prompt_id: 'test-id',
node_errors: {},
error: ''
}
await comfyPage.page.route(
'**/api/prompt',
async (route) => {
promptQueued = true
await route.fulfill({
status: 200,
body: JSON.stringify(mockResponse)
})
},
{ times: 1 }
)
await comfyPage.appMode.runButton.click()
await expect.poll(() => promptQueued).toBe(true)
})
}
)

View File

@@ -0,0 +1,332 @@
# Adding a custom-node pack to the regression suite
The authoritative, step-by-step process for onboarding a new pack. Written to
be followable by a human or an agent with no prior context. The suite itself
(what it asserts, how to run it) is documented in [README.md](README.md);
this file is only about adding coverage for a new pack.
The short version: install the pack on a local test backend, read the pack's
real node keys out of `/object_info`, author one small model-free workflow,
add one row to the manifest, prove it green locally, push. No new test code
is ever needed - the specs iterate the manifest.
## What a manifest row buys you (the tiers)
Adding the one row enrolls the pack in two kinds of coverage:
- **Every-node tiers (automatic, zero configuration).** The suite reads the
pack's FULL node list from the live backend and, for every registered
node: mounts it in both renderers, round-trips it through save/reload,
plans typed connections for all its concrete slots, and executes it for
real when it is self-sufficient (every required input is a widget with a
valid default; output wired to `PreviewAny` or the node is its own
terminus). Nodes that cannot run alone are classified and logged, never
silently dropped: `NEEDS_WIRES` (required socket inputs), `NEEDS_MODELS`
(empty model/file combo on the bare backend), `NO_SINK` (nothing
observable to queue), or "rejected at validation on defaults" (needs a
curated fixture).
- **Curated tiers (the row's fields).** `expectedNodes` + `workflow` drive
the hand-authored run-tier chain (Step 4) proving a real multi-node
wiring executes end to end, and serve as must-exist sentinels.
Every-node coverage means a pack update is tested the moment CI installs
it - including nodes you never listed.
## Step 0 - prerequisites
- A local test backend and dev server set up exactly per the
[README prerequisites](README.md#prerequisites). Do not skip `--multi-user`
or `--cache-none`.
- The pack's GitHub URL. The CI job clones and pip-installs it, so the repo
must be public and its `requirements.txt` must install on a CPU-only
runner. Packs that hard-require CUDA at import time cannot be onboarded
until they guard that import.
## Step 1 - install the pack on the test backend
```bash
cd <test-backend>/custom_nodes
git clone https://github.com/<owner>/<pack>
pip install -r <pack>/requirements.txt # if the pack has one
```
If you run a CPU-only backend, constrain pip so the pack cannot swap in a
different torch (CI does the same):
```bash
pip freeze | grep -iE '^(torch|torchvision|torchaudio)==' > /tmp/torch-constraints.txt
pip install -r <pack>/requirements.txt -c /tmp/torch-constraints.txt
```
Restart the backend and check its log: the `Import times for custom nodes`
block must list the pack with no `IMPORT FAILED` marker. An import failure is
a pack bug or a missing dependency - fix that first; nothing downstream can
work without a clean import.
While you are here, note whether the pack ships frontend JS:
```bash
curl -s http://127.0.0.1:8288/extensions | python3 -c '
import json, sys
print(sum(1 for p in json.load(sys.stdin) if p.startswith("/extensions/<pack-dir-name>/")))
'
```
Non-zero means the pack patches the frontend at runtime (restyled nodes,
rebuilt widgets, injected page chrome). Write that down - it decides whether
Step 6 needs the CI-parity run. Both "green locally, red on CI" failures in
the first 5-pack onboarding came from exactly this.
## Step 2 - read the pack's real node keys
The manifest's `expectedNodes` are the pack's `object_info` keys (the same
strings the API uses as `class_type`). They are NOT Python class names and
NOT display names. Get them from the running backend:
```bash
curl -s http://127.0.0.1:8288/object_info | python3 -c '
import json, sys
d = json.load(sys.stdin)
for key, node in sorted(d.items()):
if node.get("python_module") == "custom_nodes.<pack-dir-name>":
print(key)
'
```
Real traps this step catches (each one shipped in a real pack):
| Pack | Correct key | Wrong guesses that look right |
| ---------------------- | ------------------- | ------------------------------------------------------------------------------- |
| ComfyUI_essentials | `SimpleMathInt+` | `SimpleMathInt` (keys carry a trailing `+`, except `DisplayAny` which has none) |
| ComfyUI-KJNodes | `INTConstant` | `INT Constant` (that is the display name) |
| ComfyUI-Custom-Scripts | `ShowText\|pysssss` | `ShowText` (keys carry a `\|pysssss` suffix) |
| rgthree-comfy | `Seed (rgthree)` | `RgthreeSeed` (the Python class name) |
## Step 3 - pick the expected nodes
Choose 2-3 nodes that are:
- **Model-free**: no checkpoint / VAE / CLIP inputs, no file downloads. The
gate runs on CPU with no models installed. Constants, math, text, and
display nodes are ideal.
- **Wireable into a chain**: at least one producer (has a typed output) and
one terminal node. A terminal node either has `output_node: true` in
`/object_info` (it terminates a workflow by itself) or you end the chain in
the core `PreviewAny` node, which accepts any type.
Check a candidate's inputs, outputs, and `output_node` flag:
```bash
curl -s http://127.0.0.1:8288/object_info | python3 -c '
import json, sys
node = json.load(sys.stdin)["<exact key>"]
print(json.dumps({k: node[k] for k in ("input", "output", "output_name", "output_node")}, indent=1))
'
```
Every node you list in `expectedNodes` must appear in the run workflow: the
run tier asserts each one actually executes on the backend.
## Step 4 - author the run-tier workflow
Add one JSON file under `browser_tests/assets/customNodes/`, named
`<pack>_<what it does>_run.json`. Copy an existing asset as the template
(`rgthree_seed_display_run.json` is the simplest two-node example;
`was_number_text_run.json` shows a 3-node chain). It is the frontend
workflow format, hand-authorable:
- `nodes[].type` is the exact `object_info` key from Step 2.
- `widgets_values` is an array in the node's widget order: the `input`
entries from `/object_info` in declaration order (`required` first, then
`optional`), keeping only widget-type inputs (INT, FLOAT, STRING, BOOLEAN,
and combo lists) and skipping any input whose options say
`"forceInput": true` (those are sockets, never widgets). A required input
that is neither a widget type nor `forceInput` (a custom type like
`NUMBER`) is also a socket: wire a link into it or the run fails on a
missing required input.
- A link is one row in `links`: `[link_id, from_node_id, from_slot,
to_node_id, to_slot, "TYPE"]`, plus the matching `link`/`links` ids on the
two nodes' `inputs`/`outputs` entries.
- To wire INTO an input that would normally be a widget (no `forceInput`),
the input entry also needs a `"widget": { "name": "<input name>" }` key -
see `browser_tests/assets/vueNodes/linked-int-widget.json`.
- Keep it tiny. Two to four nodes proving "this pack executes" is the whole
job; feature-depth testing belongs to the pack's own repo.
- If the workflow needs a media file, reuse something already under
`browser_tests/assets/` (e.g. `plain_video.mp4`) - never commit new binary
assets. CI stages `plain_video.mp4` into the backend's `input/` dir; if
your workflow needs a different existing asset staged, extend the
`Stage run-tier assets` step in
`.github/workflows/ci-tests-custom-nodes.yaml`.
- A media path in the workflow (e.g. `input/plain_video.mp4`) resolves
against the backend process's working directory, not the repo. Locally,
copy the file into the `input/` dir of the directory you launched
`main.py` from, or the run tier fails validation with
`Invalid file path` and the test reports `TIMEOUT`.
## Step 5 - add the manifest row
Append one object to `browser_tests/fixtures/data/customNodeManifest.json`:
| Field | Meaning |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pack` | The pack's directory name under `custom_nodes/` (what `git clone` creates). |
| `repo` | The GitHub URL CI clones. Required non-empty. |
| `pin` | Commit SHA or tag CI checks out after cloning; `""` = default branch head. Pin when a pack breaks often; `""` also means new upstream regressions surface here first. |
| `tiers` | Which tiers run: `load` (registers + renders in both renderers), `connectivity` (typed links + slot drags), `run` (executes the workflow). Use all three unless a tier is impossible for the pack. |
| `workflow` | Path relative to `browser_tests/` of the Step 4 file. `""` only while the pack has no `run` tier. |
| `expectedNodes` | The Step 2/3 keys. The load tier mounts each in both renderers; the run tier asserts each executes. |
| `requiresGpu` | `true` only if execution genuinely needs CUDA. Such packs cannot use the `run` tier on the CPU gate. |
| `requiresModels` | Model files the workflow needs (`[]` for the packs onboarded so far - keep it that way whenever possible). |
| `timeoutMs` | Per-test budget. `30000` unless the workflow does real work (video decode uses `90000`). |
| `vueNodesCompatible` | Optional, default `true`. See the policy below. Only ever set `false`, and only with evidence. |
`loadManifest()` (`browser_tests/fixtures/customNode/manifest.ts`) validates
every row and fails loudly on a missing field, an empty `repo`, a misspelled
tier, or a `run` tier with an empty `workflow`.
## Step 6 - prove it green locally, in both environments
### 6a - fast loop (dev server)
```bash
pnpm test:custom-nodes
```
Green means: every tier for every pack passes, zero skips, and the suite's
zero-visible-errors invariant held (no error overlay, dialog, node error, or
error toast at any point). Iterate here - it is the fastest loop.
### 6b - CI-parity run (required if the pack ships frontend JS)
The dev server never loads pack frontend JS (its `/extensions` list is
core-only), so 6a exercises vanilla nodes. If Step 1 found frontend JS, a
6a green proves nothing about the pack's real runtime behavior. CI serves
the built frontend from the backend, so reproduce that exactly:
```bash
pnpm build
# relaunch the test backend with the same flags plus:
# --front-end-root <repo>/dist
# and make sure any run-tier media is in that process's input/ dir
PLAYWRIGHT_TEST_URL=http://127.0.0.1:8288 pnpm exec playwright test \
browser_tests/tests/customNodes/ --config playwright.chrome.config.ts --workers=1
```
Both real failures during the first 5-pack onboarding only existed here:
rgthree's progress bar shifted the canvas and broke slot-drag coordinates,
and rgthree's Seed rebuilt a declared input as widget-only. Skipping 6b
means discovering that class of problem one CI round at a time.
### Failure classes and what they mean
- **T0 fails only in the Vue Nodes pass** (the LiteGraph pass is green):
suspected Vue Nodes 2.0 incompatibility. Follow the policy below - do not
delete the pack, do not skip the test.
- **Run tier fails with `PARTIAL`** (some expected nodes never executed):
either the backend is missing `--cache-none` (cached nodes emit no
`executing` event) or an expected node is not actually in the workflow.
- **Run tier fails with an execution error**: the workflow JSON is wrong
(bad key, wrong `widgets_values` order, type-mismatched link) or the pack
cannot execute model-free. Fix the workflow or drop the node for a
simpler one.
- **Connectivity reports zero planned pairs**: the pack's slots are all
wildcard or combo typed (both are excluded from pairing by design because
they bypass the real type compare). The pack still gets load/run coverage.
- **Connectivity logs `widget-only on instance` exclusions**: the pack's own
frontend JS rebuilt a declared input as a widget-only control (rgthree's
Seed does this to `seed`), so there is no socket to wire. Recorded and
excluded, like wildcards - pack design, not a regression.
- **Auto-run reports a node "not in cannotRunAlone"**: the node failed to
execute on pure defaults (validation reject, or a real exception from
degenerate defaults - empty expression, empty folder, no webcam). If the
node USED to run clean this is a regression; otherwise add it to the
row's `cannotRunAlone` baseline with the run log in the PR. The check is
two-way: a listed node that starts running clean fails the suite until
the stale entry is removed.
- **Auto-run fails with `HUNG_BACKEND`**: a node blocked forever during
execution (the canonical case downloads a model at runtime and hangs
without network). The failure names the suspects and the remedy: add the
offender to `AUTO_RUN_EXCLUDE` in `allNodes.spec.ts` with its mechanism,
and restart the test backend (the hang is non-interruptible).
- **Mount test fails on console errors**: a pack's JS logged real errors
while its nodes mounted. If it is pack-attributed noise with no visible
error surface (KJNodes' loader previews fetching `filename=undefined`),
add a scoped `CONSOLE_ERROR_ALLOWLIST` entry with the mechanism;
otherwise it is a finding.
### The exception ledgers (all reasons on the record)
Every escape hatch is a reviewed list whose entries carry the mechanism, so
the gate stays honest and none can grow silently:
| Ledger | Lives in | Covers |
| ---------------------------- | ---------------------- | ------------------------------------------------------------------------------------------ |
| `vueIncompatibleNodes` | manifest row | node cannot mount under Vue Nodes 2.0 (evidence rule below) |
| `cannotRunAlone` | manifest row | node cannot execute standalone on a bare backend; asserted both ways so entries cannot rot |
| `AUTO_RUN_EXCLUDE` | `allNodes.spec.ts` | executing the node is unsafe on a bare backend (runtime downloads, hangs) |
| `CONSOLE_ERROR_ALLOWLIST` | `allNodes.spec.ts` | pack-attributed console noise with no visible error surface |
| `CONNECT_REJECTED_ALLOWLIST` | `connectivity.spec.ts` | pack JS legitimately vetoes a planned wiring |
| `ROUNDTRIP_LOST_ALLOWLIST` | `connectivity.spec.ts` | pack's own serialize/configure drops links it manages itself |
## Step 7 - push and watch CI
The `CI: Tests Custom Nodes` job (gating) re-does Steps 1-6 from scratch on
every PR: clones every manifest `repo` at its `pin`, pip-installs under CPU
torch constraints, boots the backend, runs the suite, and fails on any
install error, any test failure, or any skipped test. A new pack row is
automatically picked up; no workflow edit is needed unless you must stage an
extra asset (Step 4).
If CI goes red where local was green, reproduce under the Step 6b
environment before changing anything - the first such failure looked like
upstream drift but was actually pack frontend JS that never loads under
the dev server. Only after 6b reproduces it, decide: adjust the suite's
expectation honestly (the way widget-only instance slots became a recorded
exclusion) or, for genuine upstream drift (`pin: ""` tracks the pack's
default branch head), pin the pack to its last good commit. Never paper
over it with a skip.
## Vue Nodes 2.0 compatibility policy
Some packs only work under the LiteGraph canvas renderer and fail to mount
under Vue Nodes 2.0. The suite must state that fact without producing false
failures and without skipping tests:
1. **Default**: every pack is assumed compatible. New rows omit
`vueNodesCompatible`.
2. **Evidence rule**: set `"vueNodesCompatible": false` ONLY after the T0
Vue pass fails for the pack locally while the LiteGraph pass is green,
and the failure reproduces on a retry. A README grumble, a hunch, or an
old forum thread is not evidence. Record the evidence (the failing
assertion and the pack version) in the PR description of the change that
sets the flag. When only SOME of a pack's nodes fail to mount, use the
per-node `vueIncompatibleNodes` ledger in the manifest row instead of
flagging the whole pack - compatibility is per-node, not per-pack (all
823 nodes across the first 7 packs mount clean, so both mechanisms ship
unused; the every-node mount tier is what earns an entry).
3. **Effect of `false`**: the load tier runs its LiteGraph pass only, and
the connectivity drag test does not drag that pack's edges under Vue
Nodes. The tests still run and pass their canvas assertions - nothing is
`test.skip`ped, so the CI skip gate stays honest. The run tier and the
connectivity contract sweep are renderer-independent (they never toggle
the Vue Nodes setting) and run for the pack regardless of the flag - a
flagged pack must still execute and wire cleanly there.
4. **Un-flagging**: if a pack ships Vue Nodes support later, delete the flag
and prove T0 green in both passes locally.
## Checklist
- [ ] Pack installs clean on the test backend (no `IMPORT FAILED`)
- [ ] Checked whether the pack ships frontend JS (Step 1 `/extensions` probe)
- [ ] `expectedNodes` copied exactly from `/object_info` (Step 2 traps checked)
- [ ] All expected nodes are model-free and present in the run workflow
- [ ] Workflow JSON under `browser_tests/assets/customNodes/`, no new binaries
- [ ] Any media staged into the backend's own `input/` dir locally (Step 4)
- [ ] Manifest row appended with every field (Step 5 table)
- [ ] `vueNodesCompatible` omitted, or set `false` with recorded evidence
- [ ] 6a green: `pnpm test:custom-nodes` against the dev server, zero skips
- [ ] 6b green when the pack ships frontend JS: built dist + backend-served run
- [ ] Every-node tiers green: no unexplained mount/save-reload/auto-run
failures; any new ledger entry carries its mechanism
- [ ] Pushed; `CI: Tests Custom Nodes` green on the PR

View File

@@ -0,0 +1,115 @@
# Custom-node regression suite
Proves community custom-node packs work against this frontend across both
renderers: nodes register, render under LiteGraph (canvas) AND Vue Nodes 2.0
(DOM), and execute real workflows end to end. Manifest-driven: adding a pack
is one JSON row, no new test code.
## Prerequisites
1. A ComfyUI backend on `127.0.0.1:8288` with every manifest pack (the
`pack` entries in `browser_tests/fixtures/data/customNodeManifest.json`)
and ComfyUI_devtools
installed. Launch it with `--multi-user` (the repo-wide browser-test
prerequisite; the fixture writes per-worker user settings and the suite
depends on them landing), `--cache-none` (repeat runs must re-execute
every node or the executed-set check fails honestly with `PARTIAL`), and
with `browser_tests/assets/plain_video.mp4` copied into its `input/` dir.
2. The dev server proxying that backend:
`DEV_SERVER_COMFYUI_URL=http://127.0.0.1:8288 pnpm dev`
## Running
| Script | What it does |
| -------------------------------------- | ------------------------------------------------------------------------------------- |
| `pnpm test:custom-nodes` | whole suite headless - the pass/fail gate (every tier passes, zero skips) |
| `pnpm test:custom-nodes:watch` | headed slow-motion run of the browser tiers, hands-off watching |
| `pnpm test:custom-nodes:debug` | step through the browser tiers in the Playwright Inspector (F10 step, F8 resume) |
| `pnpm test:custom-nodes:impact-render` | Impact nodes render in both renderers (Inspector) |
| `pnpm test:custom-nodes:impact-run` | Impact group workflow executes on the backend (Inspector) |
| `pnpm test:custom-nodes:vhs-render` | VHS nodes render in both renderers (Inspector) |
| `pnpm test:custom-nodes:vhs-run` | VHS decodes a real video through its node chain (Inspector) |
| `pnpm test:custom-nodes:connectivity` | slot/type contract: type-paired links + real slot drags in both renderers (Inspector) |
| `pnpm test:custom-nodes:self-check` | watches the harness catch a deliberate execution error |
Example - watch the VHS video-decode run step by step:
```bash
pnpm test:custom-nodes:vhs-run
```
Two windows open: the app under test and the Playwright Inspector. Press F10
to execute one robot action at a time (workflow loads, queue fires, backend
decodes the video), F8 to run to the end. While paused, look but do not click
inside the app window - your clicks change the state the next assertion
checks.
Any `-g` pattern works against the generic scripts, e.g.
`pnpm test:custom-nodes:debug -g "Impact-Pack.*T0"`.
## What the tests assert
- **T0 load**: pack nodes are registered in `/object_info`, added to a
cleared graph, counted exactly, and each added node's own `[data-node-id]`
element mounts under Vue Nodes 2.0. Both renderer passes - unless the pack
declares `vueNodesCompatible: false` in the manifest (evidence required;
see [ADDING_CUSTOM_NODES.md](ADDING_CUSTOM_NODES.md)), in which case its tests run their
LiteGraph-canvas assertions only. Never a skip.
- **T1 run**: the manifest workflow is loaded and queued; the backend's
`executing` event stream must contain every expected node id, and the run
must end in `execution_success`.
- **Every-node tiers** (`allNodes.spec.ts`): the pack's FULL node list,
discovered live from `/object_info`, is exercised with zero
configuration - every registered node mounts in both renderers (chunked
at an empirically calibrated batch size), survives a serialize/configure
save-reload round-trip, and executes for real on the backend when
self-sufficient (all required inputs are widgets with valid defaults).
Nodes that cannot run alone are classified and logged
(`NEEDS_WIRES` / `NEEDS_MODELS` / `NO_SINK` / rejected-at-validation),
never silently dropped; the documented exception ledgers (see
[ADDING_CUSTOM_NODES.md](ADDING_CUSTOM_NODES.md)) carry a written mechanism for every
escape hatch.
- **connectivity (contract)**: wiring-only, no execution. A
type-pairing generator (`fixtures/customNode/typePairing.ts`) indexes
`/object_info` producers/consumers and plans one representative typed edge
per slot (wildcard `*` slots excluded - they bypass the real type compare
and prove nothing). Each planned edge must connect through the real
`isValidConnection` veto, then survive `serialize()` -> `configure()` and
appear in `graphToPrompt()` output. A curated subset is additionally
dragged for real - slot dot to slot dot - under both renderers. Orphan
types (no partner in the corpus) are reported, never fake-failed. One
representative edge per slot bounds cost; it does not prove all pairs.
- **Zero visible errors, always**: every browser test asserts the app's
error surfaces (error overlay, error dialog, node render errors, error
toasts) are absent at start and after every pass. A run is green only if a
human watching the screen sees no errors. The self-check inverts this: it
forces a real execution error and asserts the overlay IS visible, proving
the selectors stay live.
## Adding a pack
One manifest row plus one small workflow JSON - no new test code. The
authoritative step-by-step process (verifying the pack's real node keys,
authoring the run workflow, the `vueNodesCompatible` evidence rule, what CI
does with the row) lives in [ADDING_CUSTOM_NODES.md](ADDING_CUSTOM_NODES.md). Follow it
exactly; the traps it lists all shipped in real packs.
## Gotchas
- **Pack frontend JS does not load under the Vite dev server.** The dev
server's `/extensions` endpoint lists core extensions only, so nodes render
vanilla locally even when the backend has the packs installed. CI serves
the built frontend from the backend, where every pack's JS loads and can
restyle nodes, rebuild widgets, or inject page chrome. Before pushing
changes that could interact with pack JS, reproduce CI locally:
`pnpm build`, relaunch the backend with `--front-end-root <repo>/dist`,
and run the suite with `PLAYWRIGHT_TEST_URL` pointed at the backend.
- Do not run with `--trace on` against system Chrome
(`playwright.chrome.config.ts` pins trace off): the trace recorder crashes
pages under the branded Chrome channel and every test reports a bogus 15s
timeout.
- In a git worktree whose `node_modules` is symlinked from another checkout,
prefix scripts with `pnpm --config.verify-deps-before-run=false ...` to
skip pnpm's auto-install check.
- First run against a cold dev server can exceed the 15s per-test setup
budget while Vite compiles; just run again.

View File

@@ -0,0 +1,449 @@
/* oxlint-disable playwright/no-skipped-test -- tiers conditionally skip when the target backend lacks the required packs; environment gating, not a disabled test */
// Every-node coverage: the suite's core contract (mounts, survives
// save/reload, executes when self-sufficient) applied to ALL nodes a pack
// registers - not just the curated expectedNodes sentinels. Node lists come
// from the live backend, so a pack update is covered the moment it installs.
import type { Page } from '@playwright/test'
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
batchAutoRunnable,
planAutoRuns
} from '@e2e/fixtures/customNode/autoRun'
import { LocalDesktopTarget } from '@e2e/fixtures/customNode/ComfyTarget'
import { loadManifest } from '@e2e/fixtures/customNode/manifest'
import type { RawNodeDef } from '@e2e/fixtures/customNode/typePairing'
import { normalizeNodeDefs } from '@e2e/fixtures/customNode/typePairing'
import { collectConsoleErrors } from '@e2e/fixtures/utils/consoleErrorCollector'
import {
customNodeSuiteSettings,
dismissTemplatesDialog
} from '@e2e/fixtures/utils/customNodeSuite'
import { errorSurfaces } from '@e2e/fixtures/utils/errorSurfaces'
const target = new LocalDesktopTarget()
// Empirically calibrated (browser_tests/tools/batchCalibration.spec.ts):
// 24 was the largest chunk size with deterministic results across repeats
// and the best per-node cost (6.6ms/node; smaller chunks pay per-chunk
// overhead, larger ones pay fit-view and DOM density).
const BATCH_SIZE = 24
const AUTO_RUN_BATCH = 10
const GRID_SPACING = { x: 420, y: 360 }
// Nodes whose EXECUTION is unsafe on a bare sandboxed backend even though
// their inputs classify as auto-runnable. Every entry names the mechanism.
// The canonical case: a node that downloads a model at execution time hangs
// a network-restricted backend in a non-interruptible call, jamming the
// prompt queue for everything after it.
const AUTO_RUN_EXCLUDE: Record<string, Record<string, string>> = {
'rgthree-comfy': {
'Power Primitive (rgthree)':
'requires its pack JS to build the primitive value at queue time; raw defaults KeyError. Whether a page applies pack JS varies by serving setup, so excluded unconditionally - curated-workflow candidate',
'Power Puter (rgthree)':
'requires its pack JS to compile the expression at queue time; raw defaults KeyError. Excluded unconditionally - curated-workflow candidate'
},
'ComfyUI-KJNodes': {
PointsEditor:
'requires its pack JS to inject the points JSON at queue time; raw defaults JSONDecodeError. Excluded unconditionally - curated-workflow candidate',
SplineEditor:
'requires its pack JS to inject the spline JSON at queue time; raw defaults JSONDecodeError. Excluded unconditionally - curated-workflow candidate',
StringToFloatList:
'requires its pack JS to normalize the list string at queue time; raw defaults ValueError. Excluded unconditionally - curated-workflow candidate'
},
ComfyUI_essentials: {
'RemBGSession+':
'initializes a rembg session that downloads its ONNX model at execution; hangs (non-interruptibly) on a backend without network/model access',
'TransitionMask+':
'list-expanded execution emits no per-node executing event on some runs, so the executed-set signal flip-flops between PASS and PARTIAL; mount/save-reload/connectivity tiers still cover it',
'TransparentBGSession+':
'ML-session initializer like RemBGSession+; sets up/downloads a background-removal model at execution, unstable on a bare backend'
}
}
// Pack-attributed console noise with no visible error surface. Every entry
// names the mechanism; anything not matching stays a hard failure.
const CONSOLE_ERROR_ALLOWLIST: Record<
string,
Array<{ pattern: RegExp; reason: string }>
> = {
'ComfyUI-KJNodes': [
{
// Image/video loader previews fetch their combo value at creation;
// on a backend with an empty input dir the value is undefined and the
// preview 404s (and retries with a fresh rand). Console-only noise,
// no visible error; upstream-report candidate.
pattern:
/Failed to load resource.*\/api\/view\?type=input&filename=undefined/,
reason: 'loader preview fetches undefined filename on empty input dir'
}
]
}
test.use({ initialSettings: customNodeSuiteSettings })
test.beforeEach(async ({ comfyPage }) => {
await dismissTemplatesDialog(comfyPage)
})
async function expectNoVisibleErrors(
page: Page,
context: string
): Promise<void> {
for (const [surface, locator] of Object.entries(errorSurfaces(page)))
await expect(locator, `${context}: ${surface}`).toHaveCount(0)
}
// One in-page round-trip per chunk: batch-add on a grid, fit the viewport.
// Returns ids aligned with `types` (null = createNode failed).
function addChunk(page: Page, types: string[]): Promise<Array<string | null>> {
return page.evaluate(
([chunk, spacingX, spacingY]) => {
window.app!.graph.clear()
const cols = Math.ceil(Math.sqrt(chunk.length))
const ids: Array<string | null> = []
for (const [index, type] of chunk.entries()) {
const node = window.LiteGraph!.createNode(type)
if (!node) {
ids.push(null)
continue
}
node.pos = [
(index % cols) * (spacingX as number),
Math.floor(index / cols) * (spacingY as number)
]
window.app!.graph.add(node)
ids.push(String(node.id))
}
const canvas = window.app!.canvas
const rect = canvas.canvas.getBoundingClientRect()
const width = cols * (spacingX as number)
const height = Math.ceil(chunk.length / cols) * (spacingY as number)
const scale = Math.min(
(rect.width / Math.max(width, 1)) * 0.9,
(rect.height / Math.max(height, 1)) * 0.9,
1
)
canvas.ds.scale = scale
canvas.ds.offset = [60 / scale, 60 / scale]
canvas.setDirty(true, true)
return ids
},
[types, GRID_SPACING.x, GRID_SPACING.y] as const
)
}
async function packNodeKeys(
page: Page,
pack: string
): Promise<{ keys: string[]; defs: Record<string, RawNodeDef> }> {
const defs = (await page.evaluate(() =>
window.app!.api.getNodeDefs()
)) as unknown as Record<string, RawNodeDef>
const keys = normalizeNodeDefs(defs)
.filter((node) => node.pack === pack)
.map((node) => node.type)
.sort()
return { keys, defs }
}
for (const entry of loadManifest()) {
test.describe(`all nodes: ${entry.pack}`, () => {
test('every registered node mounts in both renderers', async ({
comfyPage
}) => {
test.setTimeout(240_000)
const { keys } = await packNodeKeys(comfyPage.page, entry.pack)
test.skip(
keys.length === 0,
`${entry.pack} not installed on this backend`
)
const ledger = entry.vueIncompatibleNodes ?? {}
for (const ledgered of Object.keys(ledger))
expect(
keys,
`stale ledger entry: ${ledgered} is not registered by ${entry.pack}`
).toContain(ledgered)
for (const vueNodesEnabled of [false, true]) {
const consoleErrors = collectConsoleErrors(comfyPage.page)
await comfyPage.settings.setSetting(
'Comfy.VueNodes.Enabled',
vueNodesEnabled
)
const failures: string[] = []
for (let offset = 0; offset < keys.length; offset += BATCH_SIZE) {
const chunk = keys.slice(offset, offset + BATCH_SIZE)
const ids = await addChunk(comfyPage.page, chunk)
await comfyPage.nextFrame()
const count = await comfyPage.nodeOps.getGraphNodesCount()
if (count !== chunk.length)
failures.push(
`chunk@${offset}: graph has ${count} of ${chunk.length} nodes`
)
for (const [index, id] of ids.entries()) {
const key = chunk[index]
if (id === null) {
failures.push(`${key}: createNode returned null`)
continue
}
if (!vueNodesEnabled) continue
if (key in ledger) continue
const visible = await comfyPage.page
.locator(`[data-node-id="${id}"]`)
.isVisible({ timeout: 2_000 })
.catch(() => false)
if (!visible) failures.push(`${key}: no Vue mount`)
}
}
if (vueNodesEnabled && Object.keys(ledger).length > 0)
console.log(
`${entry.pack}: ${Object.keys(ledger).length} node(s) ledgered Vue-incompatible; Vue mount not asserted for them`
)
consoleErrors.stop()
expect(
failures,
`VueNodes=${vueNodesEnabled}: ${JSON.stringify(failures, null, 1)}`
).toEqual([])
const allowlist = CONSOLE_ERROR_ALLOWLIST[entry.pack] ?? []
const allowed = consoleErrors.errors.filter((error) =>
allowlist.some((rule) => rule.pattern.test(error))
)
if (allowed.length > 0)
console.log(
`${entry.pack}: ${allowed.length} console error(s) matched the pack's allowlist (${allowlist.map((rule) => rule.reason).join('; ')})`
)
expect(
consoleErrors.errors.filter(
(error) => !allowlist.some((rule) => rule.pattern.test(error))
),
`console errors with VueNodes=${vueNodesEnabled}`
).toEqual([])
await expectNoVisibleErrors(
comfyPage.page,
`after all-nodes VueNodes=${vueNodesEnabled} pass`
)
}
})
test('every registered node survives save/reload', async ({
comfyPage
}) => {
test.setTimeout(240_000)
const { keys } = await packNodeKeys(comfyPage.page, entry.pack)
test.skip(
keys.length === 0,
`${entry.pack} not installed on this backend`
)
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
const mismatches: string[] = []
for (let offset = 0; offset < keys.length; offset += BATCH_SIZE) {
const chunk = keys.slice(offset, offset + BATCH_SIZE)
const chunkMismatches = await comfyPage.page.evaluate((types) => {
window.app!.graph.clear()
const before = new Map<
string,
{ type: string; widgetValues: number }
>()
for (const type of types) {
const node = window.LiteGraph!.createNode(type)
if (!node) continue
window.app!.graph.add(node)
before.set(String(node.id), {
type,
widgetValues: (node.widgets ?? []).length
})
}
const serialized = window.app!.graph.serialize()
window.app!.graph.configure(serialized)
const problems: string[] = []
for (const [id, expected] of before) {
const restored = window.app!.graph.getNodeById(Number(id))
if (!restored) {
problems.push(`${expected.type}: lost on reload`)
continue
}
if (restored.type !== expected.type)
problems.push(
`${expected.type}: type became ${String(restored.type)}`
)
const widgets = (restored.widgets ?? []).length
if (widgets !== expected.widgetValues)
problems.push(
`${expected.type}: widgets ${expected.widgetValues} -> ${widgets}`
)
}
window.app!.graph.clear()
return problems
}, chunk)
mismatches.push(...chunkMismatches)
}
expect(mismatches, JSON.stringify(mismatches, null, 1)).toEqual([])
await expectNoVisibleErrors(comfyPage.page, 'after save/reload sweep')
})
test('every auto-runnable node executes without error', async ({
comfyPage
}) => {
test.setTimeout(900_000)
const { keys, defs } = await packNodeKeys(comfyPage.page, entry.pack)
test.skip(
keys.length === 0,
`${entry.pack} not installed on this backend`
)
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', false)
// A hung execution from an earlier test would make every run below
// false-timeout; fail fast with the real cause instead.
const queueBusy = await comfyPage.page.evaluate(async () => {
const queue = (await window.app!.api.getQueue()) as {
Running?: unknown[]
}
return (queue.Running ?? []).length
})
expect(
queueBusy,
'backend queue already has a running prompt (earlier hung execution?) - restart the test backend'
).toBe(0)
const excluded = AUTO_RUN_EXCLUDE[entry.pack] ?? {}
for (const [key, reason] of Object.entries(excluded))
console.log(`${entry.pack}: ${key} excluded from auto-run (${reason})`)
const verdicts = planAutoRuns(
defs,
keys.filter((key) => !(key in excluded))
)
const counts = new Map<string, number>()
for (const verdict of verdicts)
counts.set(verdict.verdict, (counts.get(verdict.verdict) ?? 0) + 1)
console.log(
`${entry.pack} auto-run plan: ${[...counts.entries()]
.map(([verdict, count]) => `${verdict}=${count}`)
.join(' ')}`
)
const batches = batchAutoRunnable(verdicts, AUTO_RUN_BATCH)
const hardFailures: string[] = []
// Nodes observed unable to run standalone: backend validation rejects
// their defaults, or execution errors on them (empty expressions,
// empty folders, no webcam). Reconciled against the committed
// cannotRunAlone baseline below - never silently dropped.
const cannotRun = new Map<string, string>()
const ranClean = new Set<string>()
for (const batch of batches) {
const outcome = await runBatch(comfyPage.page, batch)
if (outcome === 'PASS') {
for (const verdict of batch) ranClean.add(verdict.key)
continue
}
// A jammed queue makes every further run a false timeout - stop and
// name the suspects instead of bisecting through poisoned results.
if (outcome.startsWith('HUNG_BACKEND')) {
hardFailures.push(
`[${batch.map((verdict) => verdict.key).join(', ')}]: ${outcome} - add the offender to AUTO_RUN_EXCLUDE with its mechanism`
)
break
}
// Isolate: rerun each node alone so one bad node names itself
// instead of implicating its nine batch-mates.
for (const verdict of batch) {
const single = await runBatch(comfyPage.page, [verdict])
if (single === 'PASS') ranClean.add(verdict.key)
else if (single.startsWith('HUNG_BACKEND')) {
hardFailures.push(
`${verdict.key}: ${single} - add to AUTO_RUN_EXCLUDE with its mechanism`
)
break
} else cannotRun.set(verdict.key, single)
}
}
// Two-way baseline reconciliation. A failure missing from the baseline
// is a regression (a node that used to run clean broke); a baseline
// entry that now runs clean, or names a node that is not even
// auto-runnable, is stale and must be removed - the list cannot rot.
const baseline = new Set(entry.cannotRunAlone ?? [])
const runnable = new Set(
batches.flatMap((batch) => batch.map((verdict) => verdict.key))
)
for (const [key, detail] of cannotRun)
if (!baseline.has(key))
hardFailures.push(
`${key}: ${detail} - not in cannotRunAlone; a regression, or a new baseline entry (attach the run log)`
)
for (const key of baseline) {
if (ranClean.has(key))
hardFailures.push(
`${key}: ran clean but is listed in cannotRunAlone - remove the stale entry`
)
else if (!runnable.has(key))
hardFailures.push(
`${key}: listed in cannotRunAlone but is not auto-runnable on this backend - remove the stale entry`
)
}
console.log(
`${entry.pack} auto-ran ${ranClean.size} node(s) clean; ${cannotRun.size} cannot run alone (baseline ${baseline.size})`
)
expect(hardFailures, JSON.stringify(hardFailures, null, 1)).toEqual([])
})
})
}
async function runBatch(
page: Page,
batch: Array<{ key: string; needsPreviewSink?: boolean }>
): Promise<string> {
const ids = await page.evaluate(
([nodes, spacingY]) => {
window.app!.graph.clear()
const ids: string[] = []
for (const [index, spec] of nodes.entries()) {
const node = window.LiteGraph!.createNode(spec.key)
if (!node) continue
node.pos = [0, index * (spacingY as number)]
window.app!.graph.add(node)
ids.push(String(node.id))
if (spec.needsPreviewSink) {
const sink = window.LiteGraph!.createNode('PreviewAny')!
sink.pos = [460, index * (spacingY as number)]
window.app!.graph.add(sink)
node.connect(0, sink, 0)
}
}
return ids
},
[batch, GRID_SPACING.y] as const
)
// Auto-runnable nodes are widget-only and CPU-trivial; anything that has
// not finished in 20s is hung, and validation rejects return instantly.
const result = await target.runWorkflow(page, {
expectedNodeIds: ids,
timeoutMs: 20_000
})
if (result.outcome === 'TIMEOUT') {
// Leave no execution behind: an abandoned run jams the single prompt
// queue and every later run false-timeouts behind it. Interrupt, then
// verify the queue actually drained - a non-interruptible hang (e.g. a
// node blocked in a network download) can only be cleared by a backend
// restart, so name it instead of letting it poison the rest.
const drained = await page.evaluate(async () => {
await window.app!.api.interrupt()
for (let attempt = 0; attempt < 10; attempt++) {
await new Promise((resolve) => setTimeout(resolve, 500))
const queue = (await window.app!.api.getQueue()) as {
Running?: unknown[]
}
if ((queue.Running ?? []).length === 0) return true
}
return false
})
if (!drained)
return 'HUNG_BACKEND (non-interruptible execution; backend restart required)'
}
return result.outcome === 'PASS'
? 'PASS'
: `${result.outcome}${result.error?.nodeType ? ` (${result.error.nodeType}: ${result.error.exceptionType ?? ''})` : ''}`
}

View File

@@ -0,0 +1,115 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
batchAutoRunnable,
classifyAutoRunnable,
planAutoRuns
} from '@e2e/fixtures/customNode/autoRun'
test.describe('autoRun classifier', () => {
test('widget-only node with outputs is runnable via a PreviewAny sink', () => {
const verdict = classifyAutoRunnable('IntConstant', {
input: { required: { value: ['INT', { default: 0 }] } },
output: ['INT'],
output_node: false
})
expect(verdict.verdict).toBe('AUTO_RUNNABLE')
expect(verdict.needsPreviewSink).toBe(true)
})
test('widget-only OUTPUT_NODE runs standalone', () => {
const verdict = classifyAutoRunnable('ShowValue', {
input: {
required: {
text: ['STRING', {}],
mode: [['raw value', 'tensor shape']]
}
},
output: [],
output_node: true
})
expect(verdict.verdict).toBe('AUTO_RUNNABLE')
expect(verdict.needsPreviewSink).toBe(false)
})
test('a required socket input means NEEDS_WIRES', () => {
const verdict = classifyAutoRunnable('VaeDecode', {
input: { required: { samples: ['LATENT'], vae: ['VAE'] } },
output: ['IMAGE'],
output_node: false
})
expect(verdict.verdict).toBe('NEEDS_WIRES')
expect(verdict.reason).toContain('samples')
})
test('forceInput STRING is a socket, not a widget', () => {
const verdict = classifyAutoRunnable('TextSink', {
input: { required: { text: ['STRING', { forceInput: true }] } },
output: ['STRING'],
output_node: true
})
expect(verdict.verdict).toBe('NEEDS_WIRES')
})
test('an empty required combo means NEEDS_MODELS', () => {
const verdict = classifyAutoRunnable('CheckpointLoader', {
input: { required: { ckpt_name: [[]] } },
output: ['MODEL'],
output_node: false
})
expect(verdict.verdict).toBe('NEEDS_MODELS')
expect(verdict.reason).toContain('ckpt_name')
})
test('no outputs and not an OUTPUT_NODE means NO_SINK', () => {
const verdict = classifyAutoRunnable('SideEffectOnly', {
input: { required: { value: ['INT', {}] } },
output: [],
output_node: false
})
expect(verdict.verdict).toBe('NO_SINK')
})
test('optional socket inputs do not block auto-running', () => {
const verdict = classifyAutoRunnable('MathWithOptionalAny', {
input: {
required: { expression: ['STRING', {}] },
optional: { a: ['*'] }
},
output: ['INT', 'FLOAT'],
output_node: true
})
expect(verdict.verdict).toBe('AUTO_RUNNABLE')
})
test('planAutoRuns maps keys and batchAutoRunnable chunks only runnables', () => {
const defs = {
A: {
input: { required: { v: ['INT', {}] } },
output: ['INT'],
output_node: false
},
B: {
input: { required: { x: ['LATENT'] } },
output: ['LATENT'],
output_node: false
},
C: {
input: { required: { v: ['FLOAT', {}] } },
output: ['FLOAT'],
output_node: false
}
}
const verdicts = planAutoRuns(defs, ['A', 'B', 'C'])
expect(verdicts.map((verdict) => verdict.verdict)).toEqual([
'AUTO_RUNNABLE',
'NEEDS_WIRES',
'AUTO_RUNNABLE'
])
const batches = batchAutoRunnable(verdicts, 1)
expect(batches).toHaveLength(2)
expect(batches[0][0].key).toBe('A')
})
})

View File

@@ -0,0 +1,478 @@
import type { Page } from '@playwright/test'
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
customNodeSuiteSettings,
dismissTemplatesDialog
} from '@e2e/fixtures/utils/customNodeSuite'
import { loadManifest } from '@e2e/fixtures/customNode/manifest'
import type {
ConnectivityOutcome,
PlannedPair,
RawNodeDef
} from '@e2e/fixtures/customNode/typePairing'
import {
isWildcard,
normalizeNodeDefs,
planPairs
} from '@e2e/fixtures/customNode/typePairing'
import { collectConsoleErrors } from '@e2e/fixtures/utils/consoleErrorCollector'
import { errorSurfaces } from '@e2e/fixtures/utils/errorSurfaces'
const CORE_PROOF_NODE_COUNT = 16
// A node may legitimately veto a wiring via onConnectInput; committed
// entries here must name the veto. Green means actual rejections are a
// subset of this list.
const CONNECT_REJECTED_ALLOWLIST: string[] = [
// pysssss MathExpression only accepts INT/FLOAT-producing links into its
// expression variables; its JS vetoes text-list producers.
'AddTextPrefix.texts -> MathExpression|pysssss.expression'
]
// A pack's own serialize/configure hooks may drop inbound links it manages
// itself; each committed entry must name the pack behavior. These reproduce
// in manual use of those packs (wire, save, reload: the link is gone), so
// they are pack behavior on record, not frontend regressions.
const ROUNDTRIP_LOST_ALLOWLIST: string[] = [
// rgthree SDXL Power Prompt rebuilds its dimension widget-inputs during
// configure and drops inbound links to them.
'BatchCount+.INT -> SDXL Power Prompt - Positive (rgthree).target_width',
'BatchCount+.INT -> SDXL Power Prompt - Positive (rgthree).target_height',
'BatchCount+.INT -> SDXL Power Prompt - Positive (rgthree).crop_width',
'BatchCount+.INT -> SDXL Power Prompt - Positive (rgthree).crop_height',
'BatchCount+.INT -> SDXL Power Prompt - Simple / Negative (rgthree).target_width',
'BatchCount+.INT -> SDXL Power Prompt - Simple / Negative (rgthree).target_height',
'BatchCount+.INT -> SDXL Power Prompt - Simple / Negative (rgthree).crop_width',
'BatchCount+.INT -> SDXL Power Prompt - Simple / Negative (rgthree).crop_height',
// VHS_SelectLatest rebuilds its dynamic slots on configure, detaching
// links on both its inputs and outputs.
'AddTextPrefix.texts -> VHS_SelectLatest.filename_prefix',
'AddTextPrefix.texts -> VHS_SelectLatest.filename_postfix',
'VHS_SelectLatest.Filename -> AddLabel.font_color'
]
test.use({ initialSettings: customNodeSuiteSettings })
test.beforeEach(async ({ comfyPage }) => {
await dismissTemplatesDialog(comfyPage)
})
async function expectNoVisibleErrors(
page: Page,
context: string
): Promise<void> {
for (const [surface, locator] of Object.entries(errorSurfaces(page)))
await expect(locator, `${context}: ${surface}`).toHaveCount(0)
}
function concrete(slot: { type: string }): boolean {
return !isWildcard(slot.type)
}
function isEntryInstalled(
nodeTypes: Set<string>,
entry: { expectedNodes: string[] }
): boolean {
return entry.expectedNodes.every((type) => nodeTypes.has(type))
}
const connectivityEntries = loadManifest().filter((entry) =>
entry.tiers.includes('connectivity')
)
test('connectivity: every type-paired link survives model, serialize, and prompt round-trips', async ({
comfyPage
}) => {
test.setTimeout(120_000)
const defs = (await comfyPage.page.evaluate(() =>
window.app!.api.getNodeDefs()
)) as unknown as Record<string, RawNodeDef>
const nodes = normalizeNodeDefs(defs)
// Pack-specific expectations apply only where the pack is installed; on a
// backend without it (e.g. a generic CI runner) the core sweep still runs
// and the absence is reported, never fake-failed or fake-passed.
const nodeTypes = new Set(nodes.map((node) => node.type))
const installedEntries = connectivityEntries.filter((entry) =>
isEntryInstalled(nodeTypes, entry)
)
for (const entry of connectivityEntries)
if (!installedEntries.includes(entry))
console.log(`connectivity: ${entry.pack} not installed on this backend`)
// Sweep EVERY node the installed packs register, not just the curated
// expectedNodes sentinels - the corpus comes from the live backend, so a
// pack update is covered the moment it installs.
const installedPacks = new Set(installedEntries.map((entry) => entry.pack))
const packTypes = nodes
.filter((node) => installedPacks.has(node.pack))
.map((node) => node.type)
const coreProof = nodes
.filter(
(node) =>
node.pack === 'core' &&
node.inputs.some(concrete) &&
node.outputs.some(concrete)
)
.map((node) => node.type)
.sort()
.slice(0, CORE_PROOF_NODE_COUNT)
const plan = planPairs(nodes, [...packTypes, ...coreProof])
expect(plan.pairs.length, 'pairing produced no edges').toBeGreaterThan(0)
console.log(
`connectivity plan: ${plan.pairs.length} pairs, ${plan.orphans.length} orphan slots, ${plan.wildcards.length} wildcard + ${plan.combos.length} combo slots (excluded by design)`
)
for (const entry of installedEntries) {
expect(
plan.pairs.some(
(pair) =>
pair.producer.pack === entry.pack || pair.consumer.pack === entry.pack
),
`${entry.pack} contributes no pairs - corpus or pack attribution broke`
).toBe(true)
}
const consoleErrors = collectConsoleErrors(comfyPage.page)
const results = await runPairsInPage(comfyPage.page, plan.pairs)
consoleErrors.stop()
expect(consoleErrors.errors, 'console errors during breadth sweep').toEqual(
[]
)
const widgetOnly = results.filter(
(result) =>
result.outcome ===
('WIDGET_ONLY_ON_INSTANCE' satisfies ConnectivityOutcome)
)
if (widgetOnly.length > 0)
console.log(
`connectivity sweep: ${widgetOnly.length} pair(s) excluded - pack JS made the declared input widget-only: ${widgetOnly.map((result) => result.key).join('; ')}`
)
const failures = results.filter(
(result) =>
result.outcome !== ('PASS' satisfies ConnectivityOutcome) &&
result.outcome !==
('WIDGET_ONLY_ON_INSTANCE' satisfies ConnectivityOutcome) &&
!(
result.outcome === ('CONNECT_REJECTED' satisfies ConnectivityOutcome) &&
CONNECT_REJECTED_ALLOWLIST.includes(result.key)
) &&
!(
result.outcome === ('ROUNDTRIP_LOST' satisfies ConnectivityOutcome) &&
ROUNDTRIP_LOST_ALLOWLIST.includes(result.key)
)
)
const passed = results.filter((result) => result.outcome === 'PASS').length
console.log(`connectivity sweep: ${passed}/${results.length} pairs PASS`)
expect(failures, JSON.stringify(failures, null, 1)).toEqual([])
expect(passed).toBeGreaterThan(0)
await expectNoVisibleErrors(comfyPage.page, 'after breadth sweep')
})
// Instance-level probe for the drag test: the first planned pair whose
// producer output AND consumer input both exist on freshly created node
// instances (pack JS can rebuild declared inputs as widget-only controls).
function firstMaterializedPair(
page: Page,
pairs: PlannedPair[]
): Promise<PlannedPair | null> {
return page.evaluate((pairsInPage) => {
for (const pair of pairsInPage) {
const producer = window.LiteGraph!.createNode(pair.producer.nodeType)
const consumer = window.LiteGraph!.createNode(pair.consumer.nodeType)
const outFound = producer?.outputs.some(
(slot) => slot.name === pair.producer.slotName
)
const inFound = consumer?.inputs.some(
(slot) => slot.name === pair.consumer.slotName
)
if (outFound && inFound) return pair
}
return null
}, pairs)
}
// The self-check below runs THIS SAME executor on poisoned pairs; if it stops
// being able to reject, every green sweep above is meaningless.
function runPairsInPage(
page: Page,
pairs: PlannedPair[]
): Promise<Array<{ key: string; outcome: string; detail?: string }>> {
return page.evaluate(async (pairsInPage) => {
const graph = window.app!.graph
const report: Array<{
key: string
outcome: string
detail?: string
}> = []
for (const pair of pairsInPage) {
const key = `${pair.producer.nodeType}.${pair.producer.slotName} -> ${pair.consumer.nodeType}.${pair.consumer.slotName}`
try {
graph.clear()
const producer = window.LiteGraph!.createNode(pair.producer.nodeType)
const consumer = window.LiteGraph!.createNode(pair.consumer.nodeType)
if (!producer || !consumer) {
report.push({
key,
outcome: 'SLOT_CONTRACT_MISMATCH',
detail: 'createNode returned null for a registered type'
})
continue
}
graph.add(producer)
graph.add(consumer)
const outIndex = producer.outputs.findIndex(
(slot) => slot.name === pair.producer.slotName
)
const inIndex = consumer.inputs.findIndex(
(slot) => slot.name === pair.consumer.slotName
)
if (outIndex < 0 || inIndex < 0) {
// A pack's own frontend JS may rebuild a declared input as a
// widget-only control (rgthree's Seed does this to `seed`). That is
// pack design, not a wiring regression - excluded like wildcards.
// A name that exists NEITHER as slot nor widget stays a hard fail.
const widgetOnly =
outIndex >= 0 &&
(consumer.widgets ?? []).some(
(widget) => widget.name === pair.consumer.slotName
)
report.push({
key,
outcome: widgetOnly
? 'WIDGET_ONLY_ON_INSTANCE'
: 'SLOT_CONTRACT_MISMATCH',
detail: `declared slot missing on instance (out=${outIndex}, in=${inIndex})`
})
continue
}
const link = producer.connect(outIndex, consumer, inIndex)
if (!link || consumer.inputs[inIndex]?.link == null) {
report.push({ key, outcome: 'CONNECT_REJECTED' })
continue
}
const serialized = graph.serialize()
graph.configure(serialized)
const restored = graph.getNodeById(consumer.id)
if (restored?.inputs?.[inIndex]?.link == null) {
report.push({
key,
outcome: 'ROUNDTRIP_LOST',
detail: 'serialize/configure dropped the link'
})
continue
}
const prompt = (await window.app!.graphToPrompt()) as {
output?: Record<string, { inputs?: Record<string, unknown> }>
}
const promptInput =
prompt.output?.[String(consumer.id)]?.inputs?.[pair.consumer.slotName]
if (!Array.isArray(promptInput)) {
report.push({
key,
outcome: 'ROUNDTRIP_LOST',
detail: 'link missing from graphToPrompt output'
})
continue
}
report.push({ key, outcome: 'PASS' })
} catch (error) {
report.push({
key,
outcome: 'SLOT_CONTRACT_MISMATCH',
detail: `threw: ${String(error)}`
})
}
}
graph.clear()
return report
}, pairs)
}
test('connectivity self-check: the executor rejects broken pairs', async ({
comfyPage
}) => {
const slot = (nodeType: string, slotName: string, slotType: string) => ({
nodeType,
pack: 'core',
slotName,
slotType
})
const results = await runPairsInPage(comfyPage.page, [
{
producer: slot('CheckpointLoaderSimple', 'MODEL', 'MODEL'),
consumer: slot('KSampler', 'latent_image', 'LATENT')
},
{
producer: slot('EmptyLatentImage', 'LATENT', 'LATENT'),
consumer: slot('KSampler', 'does_not_exist', 'LATENT')
}
])
expect(results.map((result) => result.outcome)).toEqual([
'CONNECT_REJECTED',
'SLOT_CONTRACT_MISMATCH'
])
})
test('connectivity drags: curated slot-to-slot wires connect under both renderers', async ({
comfyPage
}) => {
test.setTimeout(120_000)
const defs = (await comfyPage.page.evaluate(() =>
window.app!.api.getNodeDefs()
)) as unknown as Record<string, RawNodeDef>
const nodes = normalizeNodeDefs(defs)
// Native anchor pair plus one in-pack, link-typed pair per connectivity
// pack (derived from the same generator the breadth sweep uses).
const dragEdges: PlannedPair[] = [
{
producer: {
nodeType: 'EmptyLatentImage',
pack: 'core',
slotName: 'LATENT',
slotType: 'LATENT'
},
consumer: {
nodeType: 'KSampler',
pack: 'core',
slotName: 'latent_image',
slotType: 'LATENT'
}
}
]
const nodeTypes = new Set(nodes.map((node) => node.type))
for (const entry of connectivityEntries) {
if (!isEntryInstalled(nodeTypes, entry)) {
console.log(
`connectivity drag: ${entry.pack} not installed on this backend`
)
continue
}
// Restrict the partner pool to the pack itself so the drag proves an
// in-pack wiring; widget-backed primitive inputs render real slot dots
// in Vue (verified empirically), so no slot type is excluded at plan time.
const packPlan = planPairs(
nodes.filter((node) => node.pack === entry.pack),
entry.expectedNodes
)
expect(
packPlan.pairs.length,
`${entry.pack} has no in-pack draggable pair - drag coverage lost`
).toBeGreaterThan(0)
// The plan comes from object_info, but a pack's own JS can rebuild a
// declared input as widget-only on the instance (rgthree's Seed does).
// Drag the first pair whose slots actually materialize; a pack whose
// every planned pair is customized away has no socket contract to drag.
const inPack = await firstMaterializedPair(comfyPage.page, packPlan.pairs)
if (!inPack) {
console.log(
`connectivity drag: ${entry.pack} planned pairs are widget-only on instances; drag not applicable`
)
continue
}
dragEdges.push(inPack)
}
const vueIncompatiblePacks = new Set(
connectivityEntries
.filter((entry) => entry.vueNodesCompatible === false)
.map((entry) => entry.pack)
)
for (const vueNodesEnabled of [false, true]) {
const consoleErrors = collectConsoleErrors(comfyPage.page)
await comfyPage.settings.setSetting(
'Comfy.VueNodes.Enabled',
vueNodesEnabled
)
for (const edge of dragEdges) {
if (vueNodesEnabled && vueIncompatiblePacks.has(edge.producer.pack)) {
console.log(
`connectivity drag: ${edge.producer.pack} declares vueNodesCompatible=false; Vue drag not applicable`
)
continue
}
await comfyPage.nodeOps.clearGraph()
const producer = await comfyPage.nodeOps.addNode(
edge.producer.nodeType,
undefined,
{ x: 150, y: 200 }
)
const consumer = await comfyPage.nodeOps.addNode(
edge.consumer.nodeType,
undefined,
{ x: 700, y: 200 }
)
await comfyPage.nextFrame()
const [outIndex, inIndex] = await comfyPage.page.evaluate(
([producerId, consumerId, outName, inName]) => {
const byId = (id: string) =>
window.app!.graph.nodes.find((node) => String(node.id) === id)!
const src = byId(producerId)
const dst = byId(consumerId)
return [
src.outputs.findIndex((slot) => slot.name === outName),
dst.inputs.findIndex((slot) => slot.name === inName)
]
},
[
String(producer.id),
String(consumer.id),
edge.producer.slotName,
edge.consumer.slotName
] as const
)
const key = `${edge.producer.nodeType}.${edge.producer.slotName} -> ${edge.consumer.nodeType}.${edge.consumer.slotName}`
expect(outIndex, `${key}: producer slot on instance`).toBeGreaterThan(-1)
expect(inIndex, `${key}: consumer slot on instance`).toBeGreaterThan(-1)
if (vueNodesEnabled) {
await comfyPage.vueNodes.waitForNodes(2)
// Output-side mirror of getInputSlotConnectionDot, addressed by
// data-slot-key so shared-label ambiguity cannot misfire the drag.
const outDot = comfyPage.page
.locator(`[data-node-id="${String(producer.id)}"]`)
.locator('.lg-slot--output')
.filter({
has: comfyPage.page.locator(
`[data-slot-key="${String(producer.id)}-out-${outIndex}"]`
)
})
.getByTestId('slot-connection-dot')
const inDot = comfyPage.vueNodes.getInputSlotConnectionDot(
String(consumer.id),
inIndex
)
await outDot.dragTo(inDot)
} else {
await producer.connectOutput(outIndex, consumer, inIndex)
}
const linked = await comfyPage.page.evaluate(
([consumerId, index]) => {
const node = window.app!.graph.nodes.find(
(candidate) => String(candidate.id) === consumerId
)
return node?.inputs?.[Number(index)]?.link != null
},
[String(consumer.id), String(inIndex)] as const
)
expect(linked, `${key} with VueNodes=${vueNodesEnabled}`).toBe(true)
}
consoleErrors.stop()
expect(
consoleErrors.errors,
`console errors with VueNodes=${vueNodesEnabled}`
).toEqual([])
await expectNoVisibleErrors(
comfyPage.page,
`after drag pass VueNodes=${vueNodesEnabled}`
)
}
})

View File

@@ -0,0 +1,58 @@
import { readFileSync } from 'node:fs'
import { resolve } from 'node:path'
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
customNodeSuiteSettings,
dismissTemplatesDialog
} from '@e2e/fixtures/utils/customNodeSuite'
import { collectConsoleErrors } from '@e2e/fixtures/utils/consoleErrorCollector'
import { errorSurfaces } from '@e2e/fixtures/utils/errorSurfaces'
import { assetPath } from '@e2e/fixtures/utils/paths'
// Core-only, model-free workflow: the bundled default template references
// model files a scoped test backend does not have, which rightly trips the
// error surfaces this suite asserts are clean.
const smokeWorkflow = JSON.parse(
readFileSync(resolve(assetPath('customNodes/core_smoke.json')), 'utf-8')
) as ComfyWorkflowJSON
test.use({ initialSettings: customNodeSuiteSettings })
test.beforeEach(async ({ comfyPage }) => {
await dismissTemplatesDialog(comfyPage)
})
test.describe('smoke: core workflow', () => {
test('loads without console errors in both renderers', async ({
comfyPage
}) => {
for (const vueNodesEnabled of [false, true]) {
const consoleErrors = collectConsoleErrors(comfyPage.page)
await comfyPage.settings.setSetting(
'Comfy.VueNodes.Enabled',
vueNodesEnabled
)
await comfyPage.workflow.loadGraphData(smokeWorkflow)
await comfyPage.nextFrame()
consoleErrors.stop()
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBeGreaterThan(0)
expect(
consoleErrors.errors,
`console errors (VueNodes=${vueNodesEnabled})`
).toEqual([])
for (const [surface, locator] of Object.entries(
errorSurfaces(comfyPage.page)
))
await expect(
locator,
`${surface} (VueNodes=${vueNodesEnabled})`
).toHaveCount(0)
}
})
})

View File

@@ -0,0 +1,191 @@
/* oxlint-disable playwright/no-skipped-test -- tiers conditionally skip when the target backend lacks the required packs (installed custom nodes or devtools); this is the framework's designed environment gating, not a disabled test */
import { existsSync, readFileSync } from 'node:fs'
import { resolve } from 'node:path'
import type { Page } from '@playwright/test'
import type { ComfyWorkflowJSON } from '@/platform/workflow/validation/schemas/workflowSchema'
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
customNodeSuiteSettings,
dismissTemplatesDialog
} from '@e2e/fixtures/utils/customNodeSuite'
import { LocalDesktopTarget } from '@e2e/fixtures/customNode/ComfyTarget'
import {
loadManifest,
rendererPassesFor
} from '@e2e/fixtures/customNode/manifest'
import { expectedNodesPresent } from '@e2e/fixtures/customNode/objectInfoValidator'
import { collectConsoleErrors } from '@e2e/fixtures/utils/consoleErrorCollector'
import { errorSurfaces } from '@e2e/fixtures/utils/errorSurfaces'
import { assetPath } from '@e2e/fixtures/utils/paths'
const target = new LocalDesktopTarget()
const OBJECT_INFO_SANITY_FLOOR = 50
test.use({ initialSettings: customNodeSuiteSettings })
test.beforeEach(async ({ comfyPage }) => {
await dismissTemplatesDialog(comfyPage)
})
async function expectNoVisibleErrors(
page: Page,
context: string
): Promise<void> {
for (const [surface, locator] of Object.entries(errorSurfaces(page)))
await expect(locator, `${context}: ${surface}`).toHaveCount(0)
}
function readWorkflow(relativePath: string): ComfyWorkflowJSON {
return JSON.parse(
readFileSync(resolve(relativePath), 'utf-8')
) as ComfyWorkflowJSON
}
async function nodeIdsByType(
page: Page,
classTypes: string[]
): Promise<string[]> {
return await page.evaluate((types) => {
const nodes = window.app!.graph.nodes ?? []
return nodes
.filter((node) => {
const n = node as { comfyClass?: string; type?: string }
return types.includes(n.comfyClass ?? n.type ?? '')
})
.map((node) => String(node.id))
}, classTypes)
}
for (const entry of loadManifest()) {
const workflowRelative = `browser_tests/${entry.workflow}`
test.describe(`custom node: ${entry.pack}`, () => {
test('T0 load: expected nodes register and render in both renderers', async ({
comfyPage
}) => {
test.setTimeout(entry.timeoutMs)
const objectInfo = await target.getObjectInfo(comfyPage.page)
expect(
Object.keys(objectInfo).length,
'object_info sanity floor'
).toBeGreaterThan(OBJECT_INFO_SANITY_FLOOR)
const { missing } = expectedNodesPresent(objectInfo, entry.expectedNodes)
test.skip(
missing.length > 0,
`${entry.pack} not installed on this backend (missing: ${missing.join(', ')})`
)
await expectNoVisibleErrors(comfyPage.page, 'at startup')
// A pack that declares vueNodesCompatible: false is exercised under the
// LiteGraph canvas only - rendering its nodes under Vue Nodes 2.0 would
// fail for a known pack limitation, not a frontend regression. This is
// conditional coverage, not a test skip: the test still runs and gates.
const rendererPasses = rendererPassesFor(entry)
if (entry.vueNodesCompatible === false)
console.log(
`${entry.pack} declares vueNodesCompatible=false; Vue Nodes pass not applicable`
)
for (const vueNodesEnabled of rendererPasses) {
const consoleErrors = collectConsoleErrors(comfyPage.page)
await comfyPage.settings.setSetting(
'Comfy.VueNodes.Enabled',
vueNodesEnabled
)
await comfyPage.nodeOps.clearGraph()
const addedIds: string[] = []
for (const classType of entry.expectedNodes) {
const node = await comfyPage.nodeOps.addNode(classType)
addedIds.push(String(node.id))
}
await comfyPage.nextFrame()
expect(await comfyPage.nodeOps.getGraphNodesCount()).toBe(
entry.expectedNodes.length
)
// Vue Nodes 2.0 mounts each node as a [data-node-id] element; assert
// the pack's own nodes rendered, not just any node count.
if (vueNodesEnabled)
for (const id of addedIds)
await expect(comfyPage.vueNodes.getNodeLocator(id)).toBeVisible()
consoleErrors.stop()
expect(
consoleErrors.errors,
`console errors with VueNodes=${vueNodesEnabled}`
).toEqual([])
await expectNoVisibleErrors(
comfyPage.page,
`after VueNodes=${vueNodesEnabled} pass`
)
}
})
test('T1 run: workflow executes without error', async ({ comfyPage }) => {
test.setTimeout(entry.timeoutMs + 15_000)
const objectInfo = await target.getObjectInfo(comfyPage.page)
const { missing } = expectedNodesPresent(objectInfo, entry.expectedNodes)
test.skip(
!entry.tiers.includes('run') ||
missing.length > 0 ||
entry.requiresGpu ||
entry.requiresModels.length > 0 ||
!entry.workflow ||
!existsSync(resolve(workflowRelative)),
`run tier unavailable for ${entry.pack}`
)
await expectNoVisibleErrors(comfyPage.page, 'at startup')
await comfyPage.workflow.loadGraphData(readWorkflow(workflowRelative))
const result = await target.runWorkflow(comfyPage.page, {
expectedNodeIds: await nodeIdsByType(
comfyPage.page,
entry.expectedNodes
),
timeoutMs: entry.timeoutMs
})
expect(result.outcome, JSON.stringify(result.error ?? {})).toBe('PASS')
await expectNoVisibleErrors(comfyPage.page, 'after run')
})
})
}
test('harness self-check: captures a real execution error', async ({
comfyPage
}) => {
test.setTimeout(30_000)
const objectInfo = await target.getObjectInfo(comfyPage.page)
expect(
Object.keys(objectInfo).length,
'object_info sanity floor'
).toBeGreaterThan(OBJECT_INFO_SANITY_FLOOR)
test.skip(
!('DevToolsErrorRaiseNode' in objectInfo),
'ComfyUI_devtools not installed on this backend'
)
await comfyPage.workflow.loadGraphData(
readWorkflow(assetPath('nodes/execution_error.json'))
)
const result = await target.runWorkflow(comfyPage.page, {
expectedNodeIds: [],
timeoutMs: 15000
})
expect(result.outcome).toBe('EXECUTION_ERROR')
expect(result.error?.exceptionType).toBeTruthy()
// Proves the event tap captures node ids from the live `executing` stream
// (its detail is a bare string): the failing node starts before it raises.
expect(result.executedNodes.length).toBeGreaterThan(0)
// Positive control for the zero-visible-errors invariant: a real execution
// error MUST surface in the app's error overlay. If this fails, the
// expectNoVisibleErrors selectors have rotted and every clean assertion in
// this suite is meaningless.
await expect(errorSurfaces(comfyPage.page).errorOverlay).toBeVisible()
})

View File

@@ -0,0 +1,29 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import {
loadManifest,
rendererPassesFor
} from '@e2e/fixtures/customNode/manifest'
test.describe('customNode manifest', () => {
test('loads entries with the shape the regression spec depends on', () => {
const entries = loadManifest()
expect(entries.length).toBeGreaterThan(0)
for (const entry of entries) {
expect(entry.pack).toBeTruthy()
expect(entry.expectedNodes.length).toBeGreaterThan(0)
expect(entry.tiers.length).toBeGreaterThan(0)
}
})
test('rendererPassesFor drops only the Vue pass, only on an explicit false', () => {
expect(rendererPassesFor({})).toEqual([false, true])
expect(rendererPassesFor({ vueNodesCompatible: true })).toEqual([
false,
true
])
expect(rendererPassesFor({ vueNodesCompatible: false })).toEqual([false])
})
})

View File

@@ -0,0 +1,47 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import type { ObjectInfo } from '@e2e/fixtures/customNode/objectInfoValidator'
import {
expectedNodesPresent,
preValidate
} from '@e2e/fixtures/customNode/objectInfoValidator'
const objectInfo: ObjectInfo = {
KSampler: { input: { required: { model: {}, seed: {} } } }
}
test.describe('objectInfoValidator', () => {
test('expectedNodesPresent splits present from missing', () => {
const { present, missing } = expectedNodesPresent(objectInfo, [
'KSampler',
'Missing (rgthree)'
])
expect(present).toEqual(['KSampler'])
expect(missing).toEqual(['Missing (rgthree)'])
})
test('preValidate returns MISSING_NODE for an unregistered class', () => {
const failure = preValidate(objectInfo, [
{ id: '1', classType: 'Ghost', inputs: {} }
])
expect(failure?.outcome).toBe('MISSING_NODE')
})
test('preValidate returns VALIDATION_FAIL naming the missing required input', () => {
const failure = preValidate(objectInfo, [
{ id: '3', classType: 'KSampler', inputs: { model: 0 } }
])
expect(failure?.outcome).toBe('VALIDATION_FAIL')
expect(failure?.message).toContain('missing required input "seed"')
})
test('preValidate passes when every required input is present', () => {
expect(
preValidate(objectInfo, [
{ id: '3', classType: 'KSampler', inputs: { model: 0, seed: 1 } }
])
).toBeNull()
})
})

View File

@@ -0,0 +1,71 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import { classifyRun } from '@e2e/fixtures/customNode/runResult'
test.describe('classifyRun', () => {
test('PASS when every expected node appears in the executing stream', () => {
const result = classifyRun({
events: [
{ type: 'execution_start' },
{ type: 'executing', node: '1' },
{ type: 'executing', node: '2' },
{ type: 'executing', node: null },
{ type: 'execution_success' }
],
expectedNodeIds: ['1', '2']
})
expect(result.outcome).toBe('PASS')
expect(result.executedNodes).toEqual(['1', '2'])
})
test('PARTIAL when a succeeding run replays a cached node that never emitted executing', () => {
const result = classifyRun({
events: [{ type: 'executing', node: '1' }, { type: 'execution_success' }],
expectedNodeIds: ['1', '2']
})
expect(result.outcome).toBe('PARTIAL')
expect(result.executedNodes).toEqual(['1'])
})
test('EXECUTION_ERROR captures the failing node details', () => {
const result = classifyRun({
events: [
{ type: 'executing', node: '1' },
{
type: 'execution_error',
error: { exceptionType: 'ValueError', nodeId: '1' }
}
],
expectedNodeIds: ['1']
})
expect(result.outcome).toBe('EXECUTION_ERROR')
expect(result.error?.exceptionType).toBe('ValueError')
})
test('EXECUTION_ERROR when the run is interrupted', () => {
const result = classifyRun({
events: [
{ type: 'executing', node: '1' },
{ type: 'execution_interrupted' }
],
expectedNodeIds: ['1']
})
expect(result.outcome).toBe('EXECUTION_ERROR')
})
test('TIMEOUT when flagged or when no terminal event arrived', () => {
const flagged = classifyRun({
events: [{ type: 'executing', node: '1' }],
expectedNodeIds: ['1'],
timedOut: true
})
const noTerminal = classifyRun({
events: [{ type: 'executing', node: '1' }],
expectedNodeIds: ['1']
})
expect(flagged.outcome).toBe('TIMEOUT')
expect(noTerminal.outcome).toBe('TIMEOUT')
})
})

View File

@@ -0,0 +1,139 @@
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import type { RawNodeDef } from '@e2e/fixtures/customNode/typePairing'
import {
isTypeCompatible,
normalizeNodeDefs,
packOf,
planPairs
} from '@e2e/fixtures/customNode/typePairing'
const DEFS: Record<string, RawNodeDef> = {
LatentSource: {
input: { required: {} },
output: ['LATENT'],
output_name: ['LATENT'],
python_module: 'nodes'
},
LatentSink: {
input: { required: { latent: ['LATENT', {}] } },
output: [],
python_module: 'custom_nodes.SomePack'
},
UnionSource: {
input: { required: {} },
output: ['STRING,INT'],
output_name: ['value'],
python_module: 'nodes'
},
IntSink: {
input: { required: { value: ['int', {}] } },
output: [],
python_module: 'nodes'
},
ComboNode: {
input: { required: { choice: [['a', 'b'], {}] } },
output: [],
python_module: 'nodes'
},
SocketlessNode: {
input: { required: { hidden: ['STRING', { socketless: true }] } },
output: [],
python_module: 'nodes'
},
WildcardNode: {
input: { required: { anything: ['*', {}] } },
output: ['*'],
output_name: ['out'],
python_module: 'nodes'
},
OrphanNode: {
input: { required: {} },
output: ['NOBODY_CONSUMES_THIS'],
output_name: ['orphan'],
python_module: 'custom_nodes.OrphanPack'
}
}
test.describe('typePairing', () => {
test('isTypeCompatible mirrors the real validator semantics', () => {
expect(isTypeCompatible('LATENT', 'LATENT')).toBe(true)
expect(isTypeCompatible('latent', 'LATENT')).toBe(true)
expect(isTypeCompatible('LATENT', 'IMAGE')).toBe(false)
expect(isTypeCompatible('STRING,INT', 'INT')).toBe(true)
expect(isTypeCompatible('STRING,INT', 'FLOAT')).toBe(false)
expect(isTypeCompatible('*', 'ANYTHING')).toBe(true)
expect(isTypeCompatible('', 'ANYTHING')).toBe(true)
})
test('packOf attributes core vs custom pack', () => {
expect(packOf('nodes')).toBe('core')
expect(packOf('comfy_extras.nodes_x')).toBe('core')
expect(packOf('custom_nodes.ComfyUI-Impact-Pack')).toBe(
'ComfyUI-Impact-Pack'
)
expect(packOf(undefined)).toBe('core')
})
test('normalize maps COMBO literals and drops socketless inputs', () => {
const nodes = normalizeNodeDefs(DEFS)
const combo = nodes.find((n) => n.type === 'ComboNode')!
expect(combo.inputs).toEqual([{ name: 'choice', type: 'COMBO' }])
const socketless = nodes.find((n) => n.type === 'SocketlessNode')!
expect(socketless.inputs).toEqual([])
})
test('planPairs pairs exact and union types, deterministically', () => {
const nodes = normalizeNodeDefs(DEFS)
const plan = planPairs(nodes, ['LatentSink', 'IntSink'])
const keys = plan.pairs.map(
(p) =>
`${p.producer.nodeType}.${p.producer.slotName}->${p.consumer.nodeType}.${p.consumer.slotName}`
)
expect(keys).toContain('LatentSource.LATENT->LatentSink.latent')
expect(keys).toContain('UnionSource.value->IntSink.value')
const again = planPairs(nodes, ['LatentSink', 'IntSink'])
expect(again.pairs).toEqual(plan.pairs)
})
test('COMBO literals are excluded from pairing with names coerced to strings', () => {
const nodes = normalizeNodeDefs({
ComboSource: {
input: { required: {} },
output: [['A', 'B', 'C']],
output_name: [['A', 'B', 'C'] as unknown as string],
python_module: 'nodes'
},
...DEFS
})
const source = nodes.find((n) => n.type === 'ComboSource')!
expect(source.outputs).toEqual([{ name: 'COMBO', type: 'COMBO' }])
const plan = planPairs(nodes, ['ComboSource', 'ComboNode'])
expect(plan.pairs).toEqual([])
expect(plan.combos.map((s) => `${s.nodeType}.${s.slotName}`)).toEqual([
'ComboSource.COMBO',
'ComboNode.choice'
])
})
test('wildcard slots are excluded, orphan types recorded not failed', () => {
const nodes = normalizeNodeDefs(DEFS)
const plan = planPairs(nodes, ['WildcardNode', 'OrphanNode'])
expect(plan.wildcards.map((w) => w.nodeType)).toEqual([
'WildcardNode',
'WildcardNode'
])
expect(plan.orphans).toEqual([
{
nodeType: 'OrphanNode',
pack: 'OrphanPack',
slotName: 'orphan',
slotType: 'NOBODY_CONSUMES_THIS',
dir: 'out'
}
])
expect(plan.pairs).toEqual([])
})
})

View File

@@ -1,5 +1,6 @@
import { expect } from '@playwright/test'
import { toLinkId } from '@/types/linkId'
import { toNodeId } from '@/types/nodeId'
import { comfyPageFixture as test } from '@e2e/fixtures/ComfyPage'
@@ -15,9 +16,10 @@ test.describe('Graph', { tag: ['@smoke', '@canvas'] }, () => {
await comfyPage.workflow.loadWorkflow('inputs/input_order_swap')
await expect
.poll(() =>
comfyPage.page.evaluate(() => {
return window.app!.graph!.links.get(1)?.target_slot
})
comfyPage.page.evaluate(
(linkId) => window.app!.graph!.links.get(linkId)?.target_slot,
toLinkId(1)
)
)
.toBe(1)
})

View File

@@ -3,6 +3,7 @@ import {
comfyPageFixture as test,
comfyExpect as expect
} from '@e2e/fixtures/ComfyPage'
import { TestIds } from '@e2e/fixtures/selectors'
test.describe('Linear Mode', { tag: '@ui' }, () => {
test('Displays linear controls when app mode active', async ({
@@ -16,7 +17,9 @@ test.describe('Linear Mode', { tag: '@ui' }, () => {
test('Run button visible in linear mode', async ({ comfyPage }) => {
await comfyPage.appMode.enterAppModeWithInputs([])
await expect(comfyPage.page.getByTestId('linear-run-button')).toBeVisible()
await expect(
comfyPage.page.getByTestId(TestIds.linear.runButton)
).toBeVisible()
})
test('Workflow info section visible', async ({ comfyPage }) => {

View File

@@ -0,0 +1,171 @@
// Measurement tool, not a gate: finds the largest mount-check batch size that
// is deterministic (identical results across repeats) and fast, for the
// all-nodes load tier. Run explicitly against a backend with the manifest
// packs installed and pack JS loaded (dist-serving; see ADDING_CUSTOM_NODES 6b):
// CN_CALIBRATE=1 PLAYWRIGHT_TEST_URL=http://127.0.0.1:8288 \
// pnpm exec playwright test browser_tests/tools/batchCalibration.spec.ts \
// --config playwright.chrome.config.ts --workers=1
// Skipped everywhere else; the gating custom-nodes CI job is path-scoped to
// tests/customNodes/ and never sees this file.
/* oxlint-disable playwright/no-skipped-test -- opt-in measurement tool */
import {
comfyExpect as expect,
comfyPageFixture as test
} from '@e2e/fixtures/ComfyPage'
import type { RawNodeDef } from '@e2e/fixtures/customNode/typePairing'
import { normalizeNodeDefs } from '@e2e/fixtures/customNode/typePairing'
import {
customNodeSuiteSettings,
dismissTemplatesDialog
} from '@e2e/fixtures/utils/customNodeSuite'
const CALIBRATION_PACKS = [
'ComfyUI-Impact-Pack',
'ComfyUI-VideoHelperSuite',
'rgthree-comfy',
'ComfyUI_essentials',
'ComfyUI-KJNodes',
'ComfyUI-Custom-Scripts',
'was-node-suite-comfyui'
]
const SAMPLE_SIZE = 120
// Divisors of SAMPLE_SIZE so every size covers the identical node set.
const CHUNK_SIZES = [4, 8, 12, 24, 40, 60]
const REPEATS = 3
const GRID_SPACING = { x: 420, y: 360 }
const VISIBLE_TIMEOUT_MS = 500
test.use({ initialSettings: customNodeSuiteSettings })
test.beforeEach(async ({ comfyPage }) => {
await dismissTemplatesDialog(comfyPage)
})
test('batch-size calibration for all-nodes mount checks', async ({
comfyPage
}) => {
test.skip(
process.env.CN_CALIBRATE !== '1',
'calibration tool - set CN_CALIBRATE=1 and run against a pack-loaded backend'
)
test.setTimeout(1_800_000)
const defs = (await comfyPage.page.evaluate(() =>
window.app!.api.getNodeDefs()
)) as unknown as Record<string, RawNodeDef>
const packNodes = normalizeNodeDefs(defs)
.filter((node) => CALIBRATION_PACKS.includes(node.pack))
.sort((a, b) => a.type.localeCompare(b.type))
// Round-robin across packs so the sample mixes heavy-JS nodes (rgthree,
// pysssss) with bulk packs instead of alphabetically front-loading one pack.
const byPack = new Map<string, string[]>()
for (const node of packNodes) {
const list = byPack.get(node.pack) ?? []
list.push(node.type)
byPack.set(node.pack, list)
}
const sample: string[] = []
for (let i = 0; sample.length < SAMPLE_SIZE; i++) {
let took = false
for (const pack of CALIBRATION_PACKS) {
const list = byPack.get(pack) ?? []
if (i < list.length && sample.length < SAMPLE_SIZE) {
sample.push(list[i])
took = true
}
}
if (!took) break
}
expect(sample, 'sample size').toHaveLength(SAMPLE_SIZE)
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
const results: Array<{
size: number
repeat: number
msPerNode: number
missing: string[]
}> = []
for (const size of CHUNK_SIZES) {
for (let repeat = 0; repeat < REPEATS; repeat++) {
const started = Date.now()
const missing: string[] = []
for (let offset = 0; offset < sample.length; offset += size) {
const chunk = sample.slice(offset, offset + size)
const ids = await comfyPage.page.evaluate(
([types, spacingX, spacingY]) => {
window.app!.graph.clear()
const cols = Math.ceil(Math.sqrt(types.length))
const ids: Array<string | null> = []
for (const [index, type] of types.entries()) {
const node = window.LiteGraph!.createNode(type)
if (!node) {
ids.push(null)
continue
}
node.pos = [
(index % cols) * (spacingX as number),
Math.floor(index / cols) * (spacingY as number)
]
window.app!.graph.add(node)
ids.push(String(node.id))
}
// Fit the whole grid into the viewport so an off-screen node is a
// culling result we measure, not a guaranteed miss.
const canvas = window.app!.canvas
const rect = canvas.canvas.getBoundingClientRect()
const width = cols * (spacingX as number)
const height = Math.ceil(types.length / cols) * (spacingY as number)
const scale = Math.min(
(rect.width / Math.max(width, 1)) * 0.9,
(rect.height / Math.max(height, 1)) * 0.9,
1
)
canvas.ds.scale = scale
canvas.ds.offset = [60 / scale, 60 / scale]
canvas.setDirty(true, true)
return ids
},
[chunk, GRID_SPACING.x, GRID_SPACING.y] as const
)
await comfyPage.nextFrame()
for (const [index, id] of ids.entries()) {
if (id === null) {
missing.push(`${chunk[index]} (createNode null)`)
continue
}
const visible = await comfyPage.page
.locator(`[data-node-id="${id}"]`)
.isVisible({ timeout: VISIBLE_TIMEOUT_MS })
.catch(() => false)
if (!visible) missing.push(chunk[index])
}
}
const msPerNode = (Date.now() - started) / sample.length
results.push({
size,
repeat,
msPerNode: Math.round(msPerNode * 10) / 10,
missing: missing.sort()
})
console.log(
`CALIB size=${size} repeat=${repeat} msPerNode=${msPerNode.toFixed(1)} missing=${missing.length}`
)
}
}
// Determinism verdict per size: the missing-set must be identical across
// repeats. Report everything; selection happens in the PR, from this data.
for (const size of CHUNK_SIZES) {
const runs = results.filter((result) => result.size === size)
const signatures = new Set(runs.map((run) => run.missing.join('|')))
const avg = runs.reduce((sum, run) => sum + run.msPerNode, 0) / runs.length
console.log(
`CALIB-SUMMARY size=${size} deterministic=${signatures.size === 1} avgMsPerNode=${avg.toFixed(1)} missing=${runs[0].missing.length}`
)
}
console.log('CALIB-MISSING ' + JSON.stringify(results[0].missing))
expect(results).toHaveLength(CHUNK_SIZES.length * REPEATS)
})

View File

@@ -52,6 +52,15 @@
"test:browser": "pnpm exec playwright test",
"test:browser:coverage": "cross-env COLLECT_COVERAGE=true pnpm test:browser",
"test:browser:local": "cross-env PLAYWRIGHT_LOCAL=1 PLAYWRIGHT_TEST_URL=http://localhost:5173 pnpm test:browser",
"test:custom-nodes": "cross-env PLAYWRIGHT_TEST_URL=http://localhost:5173 pnpm exec playwright test browser_tests/tests/customNodes/ --config playwright.chrome.config.ts --workers=1",
"test:custom-nodes:watch": "cross-env PLAYWRIGHT_TEST_URL=http://localhost:5173 PLAYWRIGHT_LOCAL=1 SLOW_MO=300 pnpm exec playwright test browser_tests/tests/customNodes/customNode.regression.spec.ts browser_tests/tests/customNodes/connectivity.spec.ts --config playwright.chrome.config.ts --workers=1 --headed",
"test:custom-nodes:debug": "cross-env PLAYWRIGHT_TEST_URL=http://localhost:5173 pnpm exec playwright test browser_tests/tests/customNodes/customNode.regression.spec.ts browser_tests/tests/customNodes/connectivity.spec.ts --config playwright.chrome.config.ts --workers=1 --debug",
"test:custom-nodes:impact-render": "pnpm test:custom-nodes:debug -g \"ComfyUI-Impact-Pack.*T0\"",
"test:custom-nodes:impact-run": "pnpm test:custom-nodes:debug -g \"ComfyUI-Impact-Pack.*T1\"",
"test:custom-nodes:vhs-render": "pnpm test:custom-nodes:debug -g \"VideoHelperSuite.*T0\"",
"test:custom-nodes:vhs-run": "pnpm test:custom-nodes:debug -g \"VideoHelperSuite.*T1\"",
"test:custom-nodes:connectivity": "pnpm test:custom-nodes:debug -g \"connectivity\"",
"test:custom-nodes:self-check": "pnpm test:custom-nodes:watch -g \"self-check\"",
"test:coverage": "vitest run --coverage",
"test:coverage:critical": "cross-env COVERAGE_CRITICAL=true vitest run --coverage",
"test:unit": "vitest run",

View File

@@ -0,0 +1,10 @@
import { defineConfig } from '@playwright/test'
import base from './playwright.config'
// Run against the system-installed Google Chrome (no bundled-chromium download).
// trace stays off: Playwright's trace recorder crashes pages under the branded
// Chrome channel on this machine (instant browser close, reported as timeout).
export default defineConfig(base, {
use: { channel: 'chrome', video: 'off', trace: 'off' }
})

View File

@@ -37,7 +37,7 @@
size="unset"
class="min-h-8 rounded-lg px-3 py-2 text-xs font-normal"
data-testid="error-overlay-see-errors"
@click="seeErrors"
@click="viewErrorsInGraph"
>
{{
appMode
@@ -67,31 +67,18 @@ import { useI18n } from 'vue-i18n'
import Button from '@/components/ui/button/Button.vue'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useRightSidePanelStore } from '@/stores/workspace/rightSidePanelStore'
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import { useErrorOverlayState } from '@/components/error/useErrorOverlayState'
import { useViewErrorsInGraph } from '@/composables/useViewErrorsInGraph'
const { appMode = false } = defineProps<{ appMode?: boolean }>()
const { t } = useI18n()
const executionErrorStore = useExecutionErrorStore()
const rightSidePanelStore = useRightSidePanelStore()
const canvasStore = useCanvasStore()
const { viewErrorsInGraph } = useViewErrorsInGraph()
const { isVisible, overlayMessage, overlayTitle } = useErrorOverlayState()
function dismiss() {
executionErrorStore.dismissErrorOverlay()
}
function seeErrors() {
canvasStore.linearMode = false
if (canvasStore.canvas) {
canvasStore.canvas.deselectAll()
canvasStore.updateSelectedItems()
}
rightSidePanelStore.openPanel('errors')
executionErrorStore.dismissErrorOverlay()
}
</script>

View File

@@ -224,7 +224,7 @@ const handleOpenUserSettings = () => {
}
const handleOpenPlansAndPricing = () => {
subscriptionDialog.showPricingTable()
subscriptionDialog.showPricingTable({ reason: 'avatar_menu_plans' })
emit('close')
}
@@ -239,8 +239,7 @@ const handleOpenPlanAndCreditsSettings = () => {
}
const handleTopUp = () => {
// Track purchase credits entry from avatar popover
useTelemetry()?.trackAddApiCreditButtonClicked()
useTelemetry()?.trackAddApiCreditButtonClicked({ source: 'avatar_menu' })
dialogService.showTopUpCreditsDialog()
emit('close')
}
@@ -254,7 +253,7 @@ const handleOpenPartnerNodesInfo = () => {
}
const handleUpgradeToAddCredits = () => {
subscriptionDialog.showPricingTable()
subscriptionDialog.showPricingTable({ reason: 'upgrade_to_add_credits' })
emit('close')
}

View File

@@ -21,6 +21,6 @@ const { isFreeTier } = useBillingContext()
const subscriptionDialog = useSubscriptionDialog()
function handleClick() {
subscriptionDialog.showPricingTable()
subscriptionDialog.showPricingTable({ reason: 'subscribe_now_button' })
}
</script>

View File

@@ -1,5 +1,6 @@
import type { ComputedRef, Ref } from 'vue'
import type { SubscriptionDialogOptions } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import type {
BillingStatus,
@@ -75,9 +76,10 @@ export interface BillingActions {
*/
requireActiveSubscription: () => Promise<void>
/**
* Shows the subscription dialog.
* Shows the subscription dialog. Pass a reason so the paywall open and any
* downstream checkout stay attributed to the triggering product moment.
*/
showSubscriptionDialog: () => void
showSubscriptionDialog: (options?: SubscriptionDialogOptions) => void
}
export interface BillingState {

View File

@@ -7,6 +7,7 @@ import {
getTierFeatures
} from '@/platform/cloud/subscription/constants/tierPricing'
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import type { SubscriptionDialogOptions } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type {
PreviewSubscribeOptions,
SubscribeOptions
@@ -281,8 +282,8 @@ function useBillingContextInternal(): BillingContext {
return activeContext.value.requireActiveSubscription()
}
function showSubscriptionDialog() {
return activeContext.value.showSubscriptionDialog()
function showSubscriptionDialog(options?: SubscriptionDialogOptions) {
return activeContext.value.showSubscriptionDialog(options)
}
return {

View File

@@ -2,6 +2,7 @@ import { computed, ref } from 'vue'
import { useAuthActions } from '@/composables/auth/useAuthActions'
import { useSubscription } from '@/platform/cloud/subscription/composables/useSubscription'
import type { SubscriptionDialogOptions } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type {
BillingStatus,
BillingSubscriptionStatus,
@@ -189,12 +190,12 @@ export function useLegacyBilling(): BillingState & BillingActions {
async function requireActiveSubscription(): Promise<void> {
await fetchStatus()
if (!isActiveSubscription.value) {
legacyShowSubscriptionDialog()
legacyShowSubscriptionDialog({ reason: 'subscription_required' })
}
}
function showSubscriptionDialog(): void {
legacyShowSubscriptionDialog()
function showSubscriptionDialog(options?: SubscriptionDialogOptions): void {
legacyShowSubscriptionDialog(options)
}
return {

View File

@@ -503,7 +503,7 @@ export function useCoreCommands(): ComfyCommand[] {
}) => {
trackRunButton(metadata)
if (!isActiveSubscription.value) {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscribe_to_run' })
return
}
@@ -526,7 +526,7 @@ export function useCoreCommands(): ComfyCommand[] {
}) => {
trackRunButton(metadata)
if (!isActiveSubscription.value) {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscribe_to_run' })
return
}
@@ -548,7 +548,7 @@ export function useCoreCommands(): ComfyCommand[] {
}) => {
trackRunButton(metadata)
if (!isActiveSubscription.value) {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscribe_to_run' })
return
}

View File

@@ -0,0 +1,105 @@
import { createPinia, setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useRightSidePanelStore } from '@/stores/workspace/rightSidePanelStore'
import { LGraph, LGraphCanvas, LGraphNode } from '@/lib/litegraph/src/litegraph'
import { createMockCanvasRenderingContext2D } from '@/utils/__tests__/litegraphTestUtils'
import { useWorkflowStore } from '@/platform/workflow/management/stores/workflowStore'
import { useViewErrorsInGraph } from './useViewErrorsInGraph'
const apiMock = vi.hoisted(() => ({
getSettings: vi.fn(),
storeSetting: vi.fn(),
storeSettings: vi.fn()
}))
vi.mock('@/scripts/api', () => ({
api: apiMock
}))
const appMock = vi.hoisted(() => ({
ui: {
settings: {
dispatchChange: vi.fn()
}
},
rootGraph: {
events: new EventTarget(),
nodes: []
}
}))
vi.mock('@/scripts/app', () => ({
app: appMock
}))
function createSelectedCanvas() {
const graph = new LGraph()
const canvasElement = document.createElement('canvas')
canvasElement.width = 800
canvasElement.height = 600
canvasElement.getContext = vi
.fn()
.mockReturnValue(createMockCanvasRenderingContext2D())
const canvas = new LGraphCanvas(canvasElement, graph, {
skip_events: true,
skip_render: true
})
const node = new LGraphNode('Selected Node')
graph.add(node)
canvas.selectedItems.add(node)
node.selected = true
return { canvas, node }
}
describe('useViewErrorsInGraph', () => {
beforeEach(() => {
vi.clearAllMocks()
setActivePinia(createPinia())
apiMock.getSettings.mockResolvedValue({})
apiMock.storeSetting.mockResolvedValue(undefined)
apiMock.storeSettings.mockResolvedValue(undefined)
})
it('opens graph errors and clears app-mode error UI state', () => {
const canvasStore = useCanvasStore()
const executionErrorStore = useExecutionErrorStore()
const rightSidePanelStore = useRightSidePanelStore()
const workflowStore = useWorkflowStore()
const { canvas, node } = createSelectedCanvas()
workflowStore.activeWorkflow = {
activeMode: 'app'
} as typeof workflowStore.activeWorkflow
canvasStore.canvas = canvas
canvasStore.selectedItems = [node]
executionErrorStore.showErrorOverlay()
useViewErrorsInGraph().viewErrorsInGraph()
expect(node.selected).toBe(false)
expect(canvasStore.linearMode).toBe(false)
expect(canvasStore.selectedItems).toEqual([])
expect(rightSidePanelStore.activeTab).toBe('errors')
expect(rightSidePanelStore.isOpen).toBe(true)
expect(executionErrorStore.isErrorOverlayOpen).toBe(false)
})
it('opens graph errors when the canvas is not initialized', () => {
const canvasStore = useCanvasStore()
const executionErrorStore = useExecutionErrorStore()
const rightSidePanelStore = useRightSidePanelStore()
canvasStore.canvas = null
executionErrorStore.showErrorOverlay()
expect(() => useViewErrorsInGraph().viewErrorsInGraph()).not.toThrow()
expect(rightSidePanelStore.activeTab).toBe('errors')
expect(rightSidePanelStore.isOpen).toBe(true)
expect(executionErrorStore.isErrorOverlayOpen).toBe(false)
})
})

View File

@@ -0,0 +1,22 @@
import { useCanvasStore } from '@/renderer/core/canvas/canvasStore'
import { useExecutionErrorStore } from '@/stores/executionErrorStore'
import { useRightSidePanelStore } from '@/stores/workspace/rightSidePanelStore'
export function useViewErrorsInGraph() {
const canvasStore = useCanvasStore()
const executionErrorStore = useExecutionErrorStore()
const rightSidePanelStore = useRightSidePanelStore()
function viewErrorsInGraph() {
canvasStore.linearMode = false
if (canvasStore.canvas) {
canvasStore.canvas.deselectAll()
canvasStore.updateSelectedItems()
}
rightSidePanelStore.openPanel('errors')
executionErrorStore.dismissErrorOverlay()
}
return { viewErrorsInGraph }
}

View File

@@ -25,6 +25,6 @@ function handleClose() {
}
function handleSubscribe() {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'upload_model_upgrade' })
}
</script>

View File

@@ -140,7 +140,10 @@ describe('CloudSubscriptionRedirectView', () => {
expect(mockPerformSubscriptionCheckout).toHaveBeenCalledWith(
'creator',
'monthly',
false
{
openInNewTab: false,
paymentIntentSource: 'deep_link'
}
)
// Shows loading affordances
@@ -169,7 +172,10 @@ describe('CloudSubscriptionRedirectView', () => {
expect(mockPerformSubscriptionCheckout).toHaveBeenCalledWith(
'creator',
'monthly',
false
{
openInNewTab: false,
paymentIntentSource: 'deep_link'
}
)
})
@@ -180,7 +186,8 @@ describe('CloudSubscriptionRedirectView', () => {
expect(screen.getByText('Subscribe to Team Plan')).toBeInTheDocument()
expect(mockPerformTeamSubscriptionCheckout).toHaveBeenCalledWith(
'team_700',
'yearly'
'yearly',
{ paymentIntentSource: 'deep_link' }
)
// Team never goes through the personal checkout path
expect(mockPerformSubscriptionCheckout).not.toHaveBeenCalled()

View File

@@ -94,7 +94,9 @@ const runRedirect = wrapWithErrorHandlingAsync(async () => {
return
}
isTeamCheckout.value = true
await performTeamSubscriptionCheckout(stopId, billingCycle)
await performTeamSubscriptionCheckout(stopId, billingCycle, {
paymentIntentSource: 'deep_link'
})
return
}
@@ -112,7 +114,10 @@ const runRedirect = wrapWithErrorHandlingAsync(async () => {
if (isActiveSubscription.value) {
await accessBillingPortal(undefined, false)
} else {
await performSubscriptionCheckout(tierKeyParam, billingCycle, false)
await performSubscriptionCheckout(tierKeyParam, billingCycle, {
openInNewTab: false,
paymentIntentSource: 'deep_link'
})
}
}, reportError)

View File

@@ -351,12 +351,12 @@ const handleRefresh = wrapWithErrorHandlingAsync(async () => {
})
function handleAddCredits() {
telemetry?.trackAddApiCreditButtonClicked()
telemetry?.trackAddApiCreditButtonClicked({ source: 'credits_panel' })
void dialogService.showTopUpCreditsDialog()
}
function handleUpgradeToAddCredits() {
showPricingTable()
showPricingTable({ reason: 'upgrade_to_add_credits' })
}
async function handleWindowFocus() {

View File

@@ -5,6 +5,8 @@ import { render, screen } from '@testing-library/vue'
import enMessages from '@/locales/en/main.json' with { type: 'json' }
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import FreeTierDialogContent from './FreeTierDialogContent.vue'
const mockRenewalDate = vi.hoisted(() => ({ value: null as string | null }))
@@ -15,7 +17,7 @@ vi.mock('@/composables/billing/useBillingContext', () => ({
}))
}))
function renderComponent() {
function renderComponent(props?: { reason?: PaymentIntentSource }) {
const i18n = createI18n({
legacy: false,
locale: 'en',
@@ -23,6 +25,7 @@ function renderComponent() {
})
return render(FreeTierDialogContent, {
props,
global: {
plugins: [i18n]
}
@@ -43,4 +46,18 @@ describe('FreeTierDialogContent', () => {
renderComponent()
expect(screen.queryByText(/credits refresh on/)).not.toBeInTheDocument()
})
it('keeps the generic copy for intent reasons outside the credits variants', () => {
mockRenewalDate.value = '2026-07-15T10:00:00Z'
renderComponent({ reason: 'subscribe_to_run' })
expect(
screen.getByText('Your credits refresh on Jul 15, 2026.')
).toBeInTheDocument()
})
it('swaps to the out-of-credits copy without the refresh line', () => {
mockRenewalDate.value = '2026-07-15T10:00:00Z'
renderComponent({ reason: 'out_of_credits' })
expect(screen.queryByText(/credits refresh on/)).not.toBeInTheDocument()
})
})

View File

@@ -52,7 +52,7 @@
</p>
<p
v-if="!reason || reason === 'subscription_required'"
v-if="!isCreditsBlockedVariant"
class="m-0 text-sm text-text-secondary"
>
{{
@@ -65,10 +65,7 @@
</p>
<p
v-if="
(!reason || reason === 'subscription_required') &&
formattedRenewalDate
"
v-if="!isCreditsBlockedVariant && formattedRenewalDate"
class="m-0 text-sm text-text-secondary"
>
{{
@@ -88,7 +85,7 @@
@click="$emit('upgrade')"
>
{{
reason === 'out_of_credits' || reason === 'top_up_blocked'
isCreditsBlockedVariant
? $t('subscription.freeTier.upgradeCta')
: $t('subscription.freeTier.subscribeCta')
}}
@@ -103,12 +100,12 @@ import { computed } from 'vue'
import Button from '@/components/ui/button/Button.vue'
import { useBillingContext } from '@/composables/billing/useBillingContext'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import SubscriptionBenefits from '@/platform/cloud/subscription/components/SubscriptionBenefits.vue'
import { getTierCredits } from '@/platform/cloud/subscription/constants/tierPricing'
defineProps<{
reason?: SubscriptionDialogReason
const { reason } = defineProps<{
reason?: PaymentIntentSource
}>()
defineEmits<{
@@ -129,4 +126,10 @@ const formattedRenewalDate = computed(() => {
})
const freeTierCredits = computed(() => getTierCredits('free'))
// Only these two variants replace the generic free-tier copy; any other
// intent reason (subscribe_to_run, deep_link, ...) keeps the default pitch.
const isCreditsBlockedVariant = computed(
() => reason === 'out_of_credits' || reason === 'top_up_blocked'
)
</script>

View File

@@ -261,6 +261,7 @@ describe('PricingTable', () => {
tier: 'creator',
cycle: 'yearly',
checkout_type: 'change',
checkout_attempt_id: expect.any(String),
previous_tier: 'standard'
})
expect(mockAccessBillingPortal).toHaveBeenCalledWith('creator-yearly')
@@ -341,6 +342,7 @@ describe('PricingTable', () => {
expect(
window.localStorage.getItem(PENDING_SUBSCRIPTION_CHECKOUT_STORAGE_KEY)
).toBeNull()
expect(mockTrackBeginCheckout).not.toHaveBeenCalled()
})
it('should use the latest userId value when it changes after mount', async () => {
@@ -366,6 +368,7 @@ describe('PricingTable', () => {
tier: 'creator',
cycle: 'yearly',
checkout_type: 'change',
checkout_attempt_id: expect.any(String),
previous_tier: 'standard'
})
})

View File

@@ -277,13 +277,19 @@ import type {
TierKey,
TierPricing
} from '@/platform/cloud/subscription/constants/tierPricing'
import { recordPendingSubscriptionCheckoutAttempt } from '@/platform/cloud/subscription/utils/subscriptionCheckoutTracker'
import {
recordPendingSubscriptionCheckoutAttempt,
withPendingCheckoutAttemptId
} from '@/platform/cloud/subscription/utils/subscriptionCheckoutTracker'
import { performSubscriptionCheckout } from '@/platform/cloud/subscription/utils/subscriptionCheckoutUtil'
import { isPlanDowngrade } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import type { BillingCycle } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import { isCloud } from '@/platform/distribution/types'
import { useTelemetry } from '@/platform/telemetry'
import type { CheckoutAttributionMetadata } from '@/platform/telemetry/types'
import type {
CheckoutAttributionMetadata,
PaymentIntentSource
} from '@/platform/telemetry/types'
import { useAuthStore } from '@/stores/authStore'
type CheckoutTierKey = Exclude<TierKey, 'free' | 'founder'>
@@ -321,6 +327,10 @@ interface PricingTierConfig {
isPopular?: boolean
}
const { reason } = defineProps<{
reason?: PaymentIntentSource
}>()
const emit = defineEmits<{
chooseTeamWorkspace: []
}>()
@@ -463,16 +473,17 @@ const handleSubscribe = wrapWithErrorHandlingAsync(
} as const
const previousPlan = currentPlanDescriptor.value
const checkoutAttribution = await getCheckoutAttributionForCloud()
if (userId.value) {
telemetry?.trackBeginCheckout({
user_id: userId.value,
tier: targetPlan.tierKey,
cycle: targetPlan.billingCycle,
checkout_type: 'change',
...checkoutAttribution,
...(previousPlan ? { previous_tier: previousPlan.tierKey } : {})
})
}
const beginCheckoutMetadata = userId.value
? {
user_id: userId.value,
tier: targetPlan.tierKey,
cycle: targetPlan.billingCycle,
checkout_type: 'change' as const,
...(reason ? { payment_intent_source: reason } : {}),
...checkoutAttribution,
...(previousPlan ? { previous_tier: previousPlan.tierKey } : {})
}
: null
// Pass the target tier to create a deep link to subscription update confirmation
const checkoutTier = getCheckoutTier(
targetPlan.tierKey,
@@ -487,29 +498,39 @@ const handleSubscribe = wrapWithErrorHandlingAsync(
if (downgrade) {
// TODO(COMFY-StripeProration): Remove once backend checkout creation mirrors portal proration ("change at billing end")
await accessBillingPortal()
const didOpenPortal = await accessBillingPortal()
if (didOpenPortal && beginCheckoutMetadata) {
telemetry?.trackBeginCheckout(beginCheckoutMetadata)
}
} else {
const didOpenPortal = await accessBillingPortal(checkoutTier)
if (!didOpenPortal) {
return
}
recordPendingSubscriptionCheckoutAttempt({
const pendingAttempt = recordPendingSubscriptionCheckoutAttempt({
tier: targetPlan.tierKey,
cycle: targetPlan.billingCycle,
checkout_type: 'change',
payment_intent_source: reason,
...(previousPlan ? { previous_tier: previousPlan.tierKey } : {}),
...(previousPlan
? { previous_cycle: previousPlan.billingCycle }
: {})
})
if (beginCheckoutMetadata) {
telemetry?.trackBeginCheckout(
withPendingCheckoutAttemptId(
beginCheckoutMetadata,
pendingAttempt
)
)
}
}
} else {
await performSubscriptionCheckout(
tierKey,
currentBillingCycle.value,
true
)
await performSubscriptionCheckout(tierKey, currentBillingCycle.value, {
paymentIntentSource: reason
})
}
} finally {
isLoading.value = false

View File

@@ -56,7 +56,7 @@ const handleSubscribe = () => {
current_tier: tier.value?.toLowerCase()
})
isAwaitingStripeSubscription.value = true
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscribe_now_button' })
}
onBeforeUnmount(() => {

View File

@@ -54,6 +54,6 @@ function handleSubscribeToRun() {
trackRunButton({ subscribe_to_run: true })
}
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscribe_to_run' })
}
</script>

View File

@@ -48,7 +48,9 @@
v-if="isActiveSubscription"
variant="primary"
class="rounded-lg px-4 py-2 text-sm font-normal text-text-primary"
@click="showSubscriptionDialog"
@click="
showSubscriptionDialog({ reason: 'settings_billing_panel' })
"
>
{{ $t('subscription.upgradePlan') }}
</Button>

View File

@@ -33,7 +33,11 @@
</i18n-t>
</div>
<PricingTable class="flex-1" @choose-team-workspace="handleChooseTeam" />
<PricingTable
:reason
class="flex-1"
@choose-team-workspace="handleChooseTeam"
/>
<!-- Contact and Enterprise Links -->
<div class="flex flex-col items-center gap-2">
@@ -157,11 +161,11 @@ import { useBillingContext } from '@/composables/billing/useBillingContext'
import { isCloud } from '@/platform/distribution/types'
import { useTelemetry } from '@/platform/telemetry'
import { useCommandStore } from '@/stores/commandStore'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
const { onClose, reason, onChooseTeam } = defineProps<{
onClose: () => void
reason?: SubscriptionDialogReason
reason?: PaymentIntentSource
onChooseTeam?: () => void
}>()

View File

@@ -24,7 +24,9 @@ export function useAccountPreconditionDialog() {
)
return
case 'subscription':
void dialogService.showSubscriptionRequiredDialog()
void dialogService.showSubscriptionRequiredDialog({
reason: 'subscription_required'
})
return
case 'credits':
void dialogService.showTopUpCreditsDialog({

View File

@@ -55,12 +55,6 @@ vi.mock('@/platform/workspace/stores/teamWorkspaceStore', () => ({
})
}))
const mockTrackSubscription = vi.hoisted(() => vi.fn())
vi.mock('@/platform/telemetry', () => ({
useTelemetry: () => ({ trackSubscription: mockTrackSubscription })
}))
describe('usePricingTableUrlLoader', () => {
beforeEach(() => {
vi.clearAllMocks()
@@ -96,9 +90,6 @@ describe('usePricingTableUrlLoader', () => {
reason: 'deep_link',
planMode: undefined
})
expect(mockTrackSubscription).toHaveBeenCalledWith('modal_opened', {
reason: 'deep_link'
})
expect(mockRouterReplace).toHaveBeenCalledWith({ query: {} })
})
@@ -150,7 +141,6 @@ describe('usePricingTableUrlLoader', () => {
await loadPricingTableFromUrl()
expect(mockShowPricingTable).not.toHaveBeenCalled()
expect(mockTrackSubscription).not.toHaveBeenCalled()
})
it('denies, strips, and clears together when the user is not eligible', async () => {
@@ -161,7 +151,6 @@ describe('usePricingTableUrlLoader', () => {
await loadPricingTableFromUrl()
expect(mockShowPricingTable).not.toHaveBeenCalled()
expect(mockTrackSubscription).not.toHaveBeenCalled()
expect(mockRouterReplace).toHaveBeenCalledWith({
query: { other: 'param' }
})
@@ -230,7 +219,6 @@ describe('usePricingTableUrlLoader', () => {
)
expect(mockShowPricingTable).not.toHaveBeenCalled()
expect(mockTrackSubscription).not.toHaveBeenCalled()
expect(mockRouterReplace).toHaveBeenCalledWith({ query: {} })
expect(preservedQueryMocks.clearPreservedQuery).toHaveBeenCalledWith(
'pricing'

View File

@@ -7,7 +7,6 @@ import {
mergePreservedQueryIntoQuery
} from '@/platform/navigation/preservedQueryManager'
import { PRESERVED_QUERY_NAMESPACES } from '@/platform/navigation/preservedQueryNamespaces'
import { useTelemetry } from '@/platform/telemetry'
import { useWorkspaceUI } from '@/platform/workspace/composables/useWorkspaceUI'
import { useTeamWorkspaceStore } from '@/platform/workspace/stores/teamWorkspaceStore'
@@ -62,7 +61,6 @@ export function usePricingTableUrlLoader() {
const planMode =
param === 'team' || param === 'personal' ? param : undefined
useTelemetry()?.trackSubscription('modal_opened', { reason: 'deep_link' })
subscriptionDialog.showPricingTable({ reason: 'deep_link', planMode })
}

View File

@@ -15,7 +15,7 @@ import { t } from '@/i18n'
import { fetchWithUnifiedRemint } from '@/platform/auth/unified/remintRetry'
import { isCloud } from '@/platform/distribution/types'
import { useTelemetry } from '@/platform/telemetry'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { SubscriptionDialogOptions } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { CheckoutAttributionMetadata } from '@/platform/telemetry/types'
import { AuthStoreError, useAuthStore } from '@/stores/authStore'
import { useDialogService } from '@/services/dialogService'
@@ -237,14 +237,7 @@ function useSubscriptionInternal() {
})
}, reportError)
const showSubscriptionDialog = (options?: {
reason?: SubscriptionDialogReason
}) => {
useTelemetry()?.trackSubscription('modal_opened', {
current_tier: subscriptionTier.value?.toLowerCase(),
reason: options?.reason
})
const showSubscriptionDialog = (options?: SubscriptionDialogOptions) => {
void showSubscriptionRequiredDialog(options)
}
@@ -277,7 +270,7 @@ function useSubscriptionInternal() {
await fetchSubscriptionStatus()
if (!isSubscribedOrIsNotCloud.value) {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'subscription_required' })
}
}

View File

@@ -39,15 +39,23 @@ vi.mock('@/stores/commandStore', () => ({
}))
// useTelemetry() returns null in OSS, a dispatcher in cloud — toggle via mockIsCloud.
const { mockIsCloud, mockTrackHelpResourceClicked } = vi.hoisted(() => ({
const {
mockIsCloud,
mockTrackHelpResourceClicked,
mockTrackAddApiCreditButtonClicked
} = vi.hoisted(() => ({
mockIsCloud: { value: true },
mockTrackHelpResourceClicked: vi.fn()
mockTrackHelpResourceClicked: vi.fn(),
mockTrackAddApiCreditButtonClicked: vi.fn()
}))
vi.mock('@/platform/telemetry', () => ({
useTelemetry: () =>
mockIsCloud.value
? { trackHelpResourceClicked: mockTrackHelpResourceClicked }
? {
trackHelpResourceClicked: mockTrackHelpResourceClicked,
trackAddApiCreditButtonClicked: mockTrackAddApiCreditButtonClicked
}
: null
}))
@@ -69,6 +77,9 @@ describe('useSubscriptionActions', () => {
const { handleAddApiCredits } = useSubscriptionActions()
handleAddApiCredits()
expect(mockShowTopUpCreditsDialog).toHaveBeenCalledOnce()
expect(mockTrackAddApiCreditButtonClicked).toHaveBeenCalledWith({
source: 'settings_billing_panel'
})
})
})

View File

@@ -21,6 +21,9 @@ export function useSubscriptionActions() {
})
const handleAddApiCredits = () => {
telemetry?.trackAddApiCreditButtonClicked({
source: 'settings_billing_panel'
})
void dialogService.showTopUpCreditsDialog()
}

View File

@@ -5,8 +5,10 @@ import { useSubscriptionDialog } from './useSubscriptionDialog'
const mockCloseDialog = vi.fn()
const mockShowLayoutDialog = vi.fn()
const mockShowTeamWorkspacesDialog = vi.fn()
const mockTrackSubscription = vi.hoisted(() => vi.fn())
const mockIsInPersonalWorkspace = vi.hoisted(() => ({ value: true }))
const mockIsFreeTier = vi.hoisted(() => ({ value: false }))
const mockTier = vi.hoisted(() => ({ value: 'FREE' as string | null }))
const mockTeamWorkspacesEnabled = vi.hoisted(() => ({ value: false }))
const mockIsCloud = vi.hoisted(() => ({ value: true }))
const mockIsLegacyTeamPlan = vi.hoisted(() => ({ value: false }))
@@ -60,10 +62,15 @@ vi.mock('@/platform/workspace/stores/teamWorkspaceStore', () => ({
vi.mock('@/composables/billing/useBillingContext', () => ({
useBillingContext: () => ({
isFreeTier: mockIsFreeTier,
isLegacyTeamPlan: mockIsLegacyTeamPlan
isLegacyTeamPlan: mockIsLegacyTeamPlan,
tier: mockTier
})
}))
vi.mock('@/platform/telemetry', () => ({
useTelemetry: () => ({ trackSubscription: mockTrackSubscription })
}))
vi.mock('@/platform/workspace/composables/useWorkspaceUI', () => ({
useWorkspaceUI: () => ({
permissions: {
@@ -80,6 +87,7 @@ describe('useSubscriptionDialog', () => {
mockIsCloud.value = true
mockIsInPersonalWorkspace.value = true
mockIsFreeTier.value = false
mockTier.value = 'FREE'
mockTeamWorkspacesEnabled.value = false
mockIsLegacyTeamPlan.value = false
mockCanManageSubscription.value = true
@@ -198,6 +206,51 @@ describe('useSubscriptionDialog', () => {
const props = mockShowLayoutDialog.mock.calls[0][0].props
expect(props.initialPlanMode).toBe('team')
})
it('tracks modal_opened with the caller reason and current tier', () => {
mockTier.value = 'STANDARD'
const { showPricingTable } = useSubscriptionDialog()
showPricingTable({ reason: 'upgrade_to_add_credits' })
expect(mockTrackSubscription).toHaveBeenCalledWith('modal_opened', {
current_tier: 'standard',
reason: 'upgrade_to_add_credits'
})
})
it('tracks modal_opened on the workspace (unified) path too', () => {
mockTeamWorkspacesEnabled.value = true
const { showPricingTable } = useSubscriptionDialog()
showPricingTable({ reason: 'subscribe_to_run' })
expect(mockTrackSubscription).toHaveBeenCalledWith(
'modal_opened',
expect.objectContaining({ reason: 'subscribe_to_run' })
)
})
it('does not track modal_opened for the inactive member dialog', () => {
mockTeamWorkspacesEnabled.value = true
mockIsInPersonalWorkspace.value = false
mockCanManageSubscription.value = false
const { showPricingTable } = useSubscriptionDialog()
showPricingTable({ reason: 'subscribe_to_run' })
expect(mockShowLayoutDialog).toHaveBeenCalledTimes(1)
expect(mockTrackSubscription).not.toHaveBeenCalled()
})
it('does not track on non-cloud', () => {
mockIsCloud.value = false
const { showPricingTable } = useSubscriptionDialog()
showPricingTable({ reason: 'subscribe_to_run' })
expect(mockTrackSubscription).not.toHaveBeenCalled()
})
})
describe('show', () => {
@@ -235,6 +288,20 @@ describe('useSubscriptionDialog', () => {
expect.objectContaining({ key: 'subscription-required' })
)
})
it('tracks modal_opened with the reason for the free-tier dialog', () => {
mockIsFreeTier.value = true
mockIsInPersonalWorkspace.value = true
const { show } = useSubscriptionDialog()
show({ reason: 'out_of_credits' })
expect(mockTrackSubscription).toHaveBeenCalledTimes(1)
expect(mockTrackSubscription).toHaveBeenCalledWith(
'modal_opened',
expect.objectContaining({ reason: 'out_of_credits' })
)
})
})
describe('startTeamWorkspaceUpgradeFlow', () => {

View File

@@ -4,6 +4,8 @@ import { useDialogStore } from '@/stores/dialogStore'
import { useBillingContext } from '@/composables/billing/useBillingContext'
import { useFeatureFlags } from '@/composables/useFeatureFlags'
import { isCloud } from '@/platform/distribution/types'
import { useTelemetry } from '@/platform/telemetry'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import { useWorkspaceUI } from '@/platform/workspace/composables/useWorkspaceUI'
import { useTeamWorkspaceStore } from '@/platform/workspace/stores/teamWorkspaceStore'
@@ -11,14 +13,8 @@ const DIALOG_KEY = 'subscription-required'
const FREE_TIER_DIALOG_KEY = 'free-tier-info'
const RESUME_PRICING_KEY = 'comfy:resume-team-pricing'
export type SubscriptionDialogReason =
| 'subscription_required'
| 'out_of_credits'
| 'top_up_blocked'
| 'deep_link'
interface SubscriptionDialogOptions {
reason?: SubscriptionDialogReason
export interface SubscriptionDialogOptions {
reason?: PaymentIntentSource
/**
* Forces the unified pricing dialog to open on a specific plan tab,
* overriding the workspace-derived default (e.g. an "Upgrade to Team" CTA
@@ -38,6 +34,17 @@ export const useSubscriptionDialog = () => {
dialogStore.closeDialog({ key: FREE_TIER_DIALOG_KEY })
}
// Fired here — the choke point every paywall/pricing dialog variant passes
// through — so both the legacy and workspace billing paths emit it.
function trackModalOpened(reason?: PaymentIntentSource) {
// Resolved lazily to avoid the useBillingContext import cycle (see below).
const { tier } = useBillingContext()
useTelemetry()?.trackSubscription('modal_opened', {
current_tier: tier.value?.toLowerCase(),
reason
})
}
function showPricingTable(options?: SubscriptionDialogOptions) {
if (!isCloud) return
@@ -71,6 +78,8 @@ export const useSubscriptionDialog = () => {
return
}
trackModalOpened(options?.reason)
// Shared dialog shell styling for both variants.
const dialogComponentProps = {
style: 'width: min(1328px, 95vw); max-height: 958px;',
@@ -167,6 +176,8 @@ export const useSubscriptionDialog = () => {
// (not at composable setup) to avoid the useBillingContext import cycle.
const { isFreeTier } = useBillingContext()
if (isFreeTier.value && workspaceStore.isInPersonalWorkspace) {
trackModalOpened(options?.reason)
const component = defineAsyncComponent(
() =>
import('@/platform/cloud/subscription/components/FreeTierDialogContent.vue')
@@ -236,7 +247,7 @@ export const useSubscriptionDialog = () => {
sessionStorage.removeItem(RESUME_PRICING_KEY)
if (!workspaceStore.isInPersonalWorkspace) {
showPricingTable()
showPricingTable({ reason: 'team_upgrade_resume' })
}
} catch {
// sessionStorage may be unavailable

View File

@@ -0,0 +1,49 @@
import { beforeEach, describe, expect, it } from 'vitest'
import {
clearPendingSubscriptionCheckoutAttempt,
consumePendingSubscriptionCheckoutSuccess,
recordPendingSubscriptionCheckoutAttempt
} from './subscriptionCheckoutTracker'
const activeProStatus = {
is_active: true,
subscription_tier: 'PRO',
subscription_duration: 'MONTHLY'
} as const
describe('subscriptionCheckoutTracker', () => {
beforeEach(() => {
clearPendingSubscriptionCheckoutAttempt()
})
it('round-trips payment_intent_source from attempt to success metadata', () => {
recordPendingSubscriptionCheckoutAttempt({
tier: 'pro',
cycle: 'monthly',
checkout_type: 'new',
payment_intent_source: 'subscribe_to_run'
})
const metadata = consumePendingSubscriptionCheckoutSuccess(activeProStatus)
expect(metadata).toMatchObject({
tier: 'pro',
checkout_type: 'new',
payment_intent_source: 'subscribe_to_run'
})
})
it('omits payment_intent_source when the attempt had none', () => {
recordPendingSubscriptionCheckoutAttempt({
tier: 'pro',
cycle: 'monthly',
checkout_type: 'new'
})
const metadata = consumePendingSubscriptionCheckoutSuccess(activeProStatus)
expect(metadata).not.toBeNull()
expect(metadata).not.toHaveProperty('payment_intent_source')
})
})

View File

@@ -7,7 +7,12 @@ import type {
TierKey
} from '@/platform/cloud/subscription/constants/tierPricing'
import type { BillingCycle } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import type { SubscriptionSuccessMetadata } from '@/platform/telemetry/types'
import type {
BeginCheckoutMetadata,
PaymentIntentSource,
SubscriptionCheckoutType,
SubscriptionSuccessMetadata
} from '@/platform/telemetry/types'
const PENDING_SUBSCRIPTION_CHECKOUT_MAX_AGE_MS = 6 * 60 * 60 * 1000
const VALID_TIER_KEYS = new Set<TierKey>([
@@ -23,7 +28,6 @@ export const PENDING_SUBSCRIPTION_CHECKOUT_STORAGE_KEY =
export const PENDING_SUBSCRIPTION_CHECKOUT_EVENT =
'comfy:subscription-checkout-attempt-changed'
type CheckoutType = 'new' | 'change'
type SubscriptionDuration = 'MONTHLY' | 'ANNUAL'
interface SubscriptionStatusSnapshot {
@@ -32,22 +36,24 @@ interface SubscriptionStatusSnapshot {
subscription_duration?: SubscriptionDuration | null
}
interface PendingSubscriptionCheckoutAttempt {
export interface PendingSubscriptionCheckoutAttempt {
attempt_id: string
started_at_ms: number
tier: TierKey
cycle: BillingCycle
checkout_type: CheckoutType
checkout_type: SubscriptionCheckoutType
previous_tier?: TierKey
previous_cycle?: BillingCycle
payment_intent_source?: PaymentIntentSource
}
interface RecordPendingSubscriptionCheckoutAttemptInput {
interface PendingSubscriptionCheckoutAttemptInput {
tier: TierKey
cycle: BillingCycle
checkout_type: CheckoutType
checkout_type: SubscriptionCheckoutType
previous_tier?: TierKey
previous_cycle?: BillingCycle
payment_intent_source?: PaymentIntentSource
}
const dispatchPendingCheckoutChangeEvent = () => {
@@ -168,6 +174,9 @@ const normalizeAttempt = (
...(candidate.previous_cycle === 'monthly' ||
candidate.previous_cycle === 'yearly'
? { previous_cycle: candidate.previous_cycle }
: {}),
...(typeof candidate.payment_intent_source === 'string'
? { payment_intent_source: candidate.payment_intent_source }
: {})
}
}
@@ -224,20 +233,27 @@ const getPendingSubscriptionCheckoutAttempt =
export const hasPendingSubscriptionCheckoutAttempt = (): boolean =>
getPendingSubscriptionCheckoutAttempt() !== null
export const recordPendingSubscriptionCheckoutAttempt = (
input: RecordPendingSubscriptionCheckoutAttemptInput
export const createPendingSubscriptionCheckoutAttempt = (
input: PendingSubscriptionCheckoutAttemptInput
): PendingSubscriptionCheckoutAttempt => {
const storage = getStorage()
const attempt: PendingSubscriptionCheckoutAttempt = {
return {
attempt_id: createAttemptId(),
started_at_ms: Date.now(),
tier: input.tier,
cycle: input.cycle,
checkout_type: input.checkout_type,
...(input.previous_tier ? { previous_tier: input.previous_tier } : {}),
...(input.previous_cycle ? { previous_cycle: input.previous_cycle } : {})
...(input.previous_cycle ? { previous_cycle: input.previous_cycle } : {}),
...(input.payment_intent_source
? { payment_intent_source: input.payment_intent_source }
: {})
}
}
export const persistPendingSubscriptionCheckoutAttempt = (
attempt: PendingSubscriptionCheckoutAttempt
): PendingSubscriptionCheckoutAttempt => {
const storage = getStorage()
if (!storage) {
return attempt
}
@@ -255,6 +271,21 @@ export const recordPendingSubscriptionCheckoutAttempt = (
return attempt
}
export const recordPendingSubscriptionCheckoutAttempt = (
input: PendingSubscriptionCheckoutAttemptInput
): PendingSubscriptionCheckoutAttempt =>
persistPendingSubscriptionCheckoutAttempt(
createPendingSubscriptionCheckoutAttempt(input)
)
export const withPendingCheckoutAttemptId = (
metadata: BeginCheckoutMetadata,
attempt: PendingSubscriptionCheckoutAttempt
): BeginCheckoutMetadata => ({
...metadata,
checkout_attempt_id: attempt.attempt_id
})
const didAttemptSucceed = (
attempt: PendingSubscriptionCheckoutAttempt,
status: SubscriptionStatusSnapshot
@@ -287,6 +318,9 @@ export const consumePendingSubscriptionCheckoutSuccess = (
cycle: attempt.cycle,
checkout_type: attempt.checkout_type,
...(attempt.previous_tier ? { previous_tier: attempt.previous_tier } : {}),
...(attempt.payment_intent_source
? { payment_intent_source: attempt.payment_intent_source }
: {}),
value,
currency: 'USD',
ecommerce: {

View File

@@ -132,13 +132,14 @@ describe('performSubscriptionCheckout', () => {
json: async () => ({ checkout_url: checkoutUrl })
} as Response)
await performSubscriptionCheckout('pro', 'yearly', true)
await performSubscriptionCheckout('pro', 'yearly')
expect(mockTelemetry.trackBeginCheckout).toHaveBeenCalledWith({
user_id: 'user-123',
tier: 'pro',
cycle: 'yearly',
checkout_type: 'new',
checkout_attempt_id: expect.any(String),
ga_client_id: 'ga-client-id',
ga_session_id: 'ga-session-id',
ga_session_number: 'ga-session-number',
@@ -150,6 +151,12 @@ describe('performSubscriptionCheckout', () => {
gbraid: 'gbraid-456',
wbraid: 'wbraid-789'
})
const beginCheckoutMetadata =
mockTelemetry.trackBeginCheckout.mock.calls[0][0]
const [, storedAttempt] = mockLocalStorage.setItem.mock.calls[0]
expect(beginCheckoutMetadata.checkout_attempt_id).toBe(
JSON.parse(storedAttempt).attempt_id
)
expect(global.fetch).toHaveBeenCalledWith(
expect.stringContaining(
'/customers/cloud-subscription-checkout/pro-yearly'
@@ -186,7 +193,7 @@ describe('performSubscriptionCheckout', () => {
json: async () => ({ checkout_url: checkoutUrl })
} as Response)
await performSubscriptionCheckout('pro', 'monthly', true)
await performSubscriptionCheckout('pro', 'monthly')
expect(warnSpy).toHaveBeenCalledWith(
'[SubscriptionCheckout] Failed to collect checkout attribution',
@@ -203,11 +210,43 @@ describe('performSubscriptionCheckout', () => {
user_id: 'user-123',
tier: 'pro',
cycle: 'monthly',
checkout_type: 'new'
checkout_type: 'new',
checkout_attempt_id: expect.any(String)
})
expect(openSpy).toHaveBeenCalledWith(checkoutUrl, '_blank')
})
it('carries the payment intent source into begin_checkout and the pending attempt', async () => {
const checkoutUrl = 'https://checkout.stripe.com/test'
const openSpy = vi
.spyOn(window, 'open')
.mockImplementation(() => window as unknown as Window)
vi.mocked(global.fetch).mockResolvedValue({
ok: true,
json: async () => ({ checkout_url: checkoutUrl })
} as Response)
await performSubscriptionCheckout('pro', 'monthly', {
paymentIntentSource: 'out_of_credits'
})
expect(mockTelemetry.trackBeginCheckout).toHaveBeenCalledWith(
expect.objectContaining({ payment_intent_source: 'out_of_credits' })
)
const beginCheckoutMetadata =
mockTelemetry.trackBeginCheckout.mock.calls[0][0]
const [, storedAttempt] = mockLocalStorage.setItem.mock.calls[0]
const pendingAttempt = JSON.parse(storedAttempt)
expect(pendingAttempt).toMatchObject({
payment_intent_source: 'out_of_credits'
})
expect(beginCheckoutMetadata.checkout_attempt_id).toBe(
pendingAttempt.attempt_id
)
openSpy.mockRestore()
})
it('uses the latest userId when it changes after checkout starts', async () => {
const checkoutUrl = 'https://checkout.stripe.com/test'
const openSpy = vi
@@ -222,7 +261,7 @@ describe('performSubscriptionCheckout', () => {
json: async () => ({ checkout_url: checkoutUrl })
} as Response)
const checkoutPromise = performSubscriptionCheckout('pro', 'yearly', true)
const checkoutPromise = performSubscriptionCheckout('pro', 'yearly')
mockUserId.value = 'user-late'
authHeader.resolve({ Authorization: 'Bearer test-token' })
@@ -235,13 +274,14 @@ describe('performSubscriptionCheckout', () => {
user_id: 'user-late',
tier: 'pro',
cycle: 'yearly',
checkout_type: 'new'
checkout_type: 'new',
checkout_attempt_id: expect.any(String)
})
)
expect(openSpy).toHaveBeenCalledWith(checkoutUrl, '_blank')
})
it('does not persist a pending attempt when the checkout popup is blocked', async () => {
it('does not persist the pending attempt when the checkout popup is blocked', async () => {
const checkoutUrl = 'https://checkout.stripe.com/test'
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => null)
@@ -250,11 +290,18 @@ describe('performSubscriptionCheckout', () => {
json: async () => ({ checkout_url: checkoutUrl })
} as Response)
await performSubscriptionCheckout('pro', 'monthly', true)
await performSubscriptionCheckout('pro', 'monthly')
expect(openSpy).toHaveBeenCalledWith(checkoutUrl, '_blank')
expect(
window.localStorage.getItem(PENDING_SUBSCRIPTION_CHECKOUT_STORAGE_KEY)
).toBeNull()
const storedAttempt = window.localStorage.getItem(
PENDING_SUBSCRIPTION_CHECKOUT_STORAGE_KEY
)
expect(storedAttempt).toBeNull()
expect(mockLocalStorage.setItem).not.toHaveBeenCalled()
expect(mockTelemetry.trackBeginCheckout).toHaveBeenCalledWith(
expect.objectContaining({
checkout_attempt_id: expect.any(String)
})
)
})
})

View File

@@ -4,12 +4,19 @@ import { useFeatureFlags } from '@/composables/useFeatureFlags'
import { getComfyApiBaseUrl } from '@/config/comfyApi'
import { t } from '@/i18n'
import { fetchWithUnifiedRemint } from '@/platform/auth/unified/remintRetry'
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import {
createPendingSubscriptionCheckoutAttempt,
persistPendingSubscriptionCheckoutAttempt,
withPendingCheckoutAttemptId
} from '@/platform/cloud/subscription/utils/subscriptionCheckoutTracker'
import { isCloud } from '@/platform/distribution/types'
import { useTelemetry } from '@/platform/telemetry'
import type {
CheckoutAttributionMetadata,
PaymentIntentSource
} from '@/platform/telemetry/types'
import { AuthStoreError, useAuthStore } from '@/stores/authStore'
import type { CheckoutAttributionMetadata } from '@/platform/telemetry/types'
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import { recordPendingSubscriptionCheckoutAttempt } from '@/platform/cloud/subscription/utils/subscriptionCheckoutTracker'
import type { BillingCycle } from './subscriptionTierRank'
type CheckoutTier = TierKey | `${TierKey}-yearly`
@@ -31,6 +38,11 @@ const getCheckoutAttributionForCloud =
return getCheckoutAttribution()
}
interface PerformSubscriptionCheckoutOptions {
openInNewTab?: boolean
paymentIntentSource?: PaymentIntentSource
}
/**
* Core subscription checkout logic shared between PricingTable and
* SubscriptionRedirectView. Handles:
@@ -47,10 +59,12 @@ const getCheckoutAttributionForCloud =
export async function performSubscriptionCheckout(
tierKey: TierKey,
currentBillingCycle: BillingCycle,
openInNewTab: boolean = true
options: PerformSubscriptionCheckoutOptions = {}
): Promise<void> {
if (!isCloud) return
const { openInNewTab = true, paymentIntentSource } = options
const authStore = useAuthStore()
const { userId } = storeToRefs(authStore)
const telemetry = useTelemetry()
@@ -108,14 +122,29 @@ export async function performSubscriptionCheckout(
const data = await response.json()
if (data.checkout_url) {
const pendingAttempt = createPendingSubscriptionCheckoutAttempt({
tier: tierKey,
cycle: currentBillingCycle,
checkout_type: 'new',
payment_intent_source: paymentIntentSource
})
if (userId.value) {
telemetry?.trackBeginCheckout({
user_id: userId.value,
tier: tierKey,
cycle: currentBillingCycle,
checkout_type: 'new',
...checkoutAttribution
})
telemetry?.trackBeginCheckout(
withPendingCheckoutAttemptId(
{
user_id: userId.value,
tier: tierKey,
cycle: currentBillingCycle,
checkout_type: 'new',
...(paymentIntentSource
? { payment_intent_source: paymentIntentSource }
: {}),
...checkoutAttribution
},
pendingAttempt
)
)
}
if (openInNewTab) {
@@ -123,18 +152,9 @@ export async function performSubscriptionCheckout(
if (!checkoutWindow) {
return
}
recordPendingSubscriptionCheckoutAttempt({
tier: tierKey,
cycle: currentBillingCycle,
checkout_type: 'new'
})
persistPendingSubscriptionCheckoutAttempt(pendingAttempt)
} else {
recordPendingSubscriptionCheckoutAttempt({
tier: tierKey,
cycle: currentBillingCycle,
checkout_type: 'new'
})
persistPendingSubscriptionCheckoutAttempt(pendingAttempt)
globalThis.location.href = data.checkout_url
}
}

View File

@@ -1,9 +1,13 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { computed, reactive } from 'vue'
const { mockIsCloud, mockSubscribe } = vi.hoisted(() => ({
mockIsCloud: { value: true },
mockSubscribe: vi.fn()
}))
const { mockIsCloud, mockSubscribe, mockTrackBeginCheckout, mockUserId } =
vi.hoisted(() => ({
mockIsCloud: { value: true },
mockSubscribe: vi.fn(),
mockTrackBeginCheckout: vi.fn(),
mockUserId: { value: 'user-1' as string | null }
}))
vi.mock('@/platform/distribution/types', () => ({
get isCloud() {
@@ -16,6 +20,12 @@ vi.mock('@/config/comfyApi', () => ({
vi.mock('@/platform/workspace/api/workspaceApi', () => ({
workspaceApi: { subscribe: mockSubscribe }
}))
vi.mock('@/platform/telemetry', () => ({
useTelemetry: () => ({ trackBeginCheckout: mockTrackBeginCheckout })
}))
vi.mock('@/stores/authStore', () => ({
useAuthStore: () => reactive({ userId: computed(() => mockUserId.value) })
}))
import { performTeamSubscriptionCheckout } from './teamSubscriptionCheckoutUtil'
@@ -43,7 +53,9 @@ describe('performTeamSubscriptionCheckout', () => {
billing_op_id: 'op_1'
})
await performTeamSubscriptionCheckout('team_700', 'yearly')
await performTeamSubscriptionCheckout('team_700', 'yearly', {
paymentIntentSource: 'deep_link'
})
expect(mockSubscribe).toHaveBeenCalledWith('team_per_credit_annual', {
returnUrl: 'https://app.test/payment/success',
@@ -51,6 +63,14 @@ describe('performTeamSubscriptionCheckout', () => {
teamCreditStopId: 'team_700'
})
expect(assignedHref).toBe('https://stripe.test/pay')
expect(mockTrackBeginCheckout).toHaveBeenCalledWith({
user_id: 'user-1',
tier: 'team',
cycle: 'yearly',
checkout_type: 'new',
billing_op_id: 'op_1',
payment_intent_source: 'deep_link'
})
})
it('uses the monthly slug and lands in the app when no Stripe step is needed', async () => {
@@ -82,6 +102,16 @@ describe('performTeamSubscriptionCheckout', () => {
expect(assignedHref).toBeUndefined()
})
it('does not track begin_checkout when subscribe fails', async () => {
mockSubscribe.mockRejectedValueOnce(new Error('subscribe failed'))
await expect(
performTeamSubscriptionCheckout('team_700', 'yearly')
).rejects.toThrow('subscribe failed')
expect(mockTrackBeginCheckout).not.toHaveBeenCalled()
})
it('does nothing off cloud', async () => {
mockIsCloud.value = false

View File

@@ -1,10 +1,16 @@
import { getComfyPlatformBaseUrl } from '@/config/comfyApi'
import { getTeamPlanSlug } from '@/platform/cloud/subscription/constants/teamPlanCreditStops'
import { isCloud } from '@/platform/distribution/types'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import { workspaceApi } from '@/platform/workspace/api/workspaceApi'
import { trackWorkspaceCheckoutStarted } from '@/platform/workspace/utils/workspaceCheckoutTelemetry'
import type { BillingCycle } from './subscriptionTierRank'
interface PerformTeamSubscriptionCheckoutOptions {
paymentIntentSource?: PaymentIntentSource
}
/**
* Direct team-plan checkout for the marketing `/cloud/subscribe?tier=team` deep
* link: subscribes to the per-credit Team plan at the chosen slider stop and
@@ -22,7 +28,8 @@ import type { BillingCycle } from './subscriptionTierRank'
*/
export async function performTeamSubscriptionCheckout(
teamCreditStopId: string,
billingCycle: BillingCycle
billingCycle: BillingCycle,
options: PerformTeamSubscriptionCheckoutOptions = {}
): Promise<void> {
if (!isCloud) return
@@ -33,6 +40,14 @@ export async function performTeamSubscriptionCheckout(
teamCreditStopId
})
trackWorkspaceCheckoutStarted({
tier: 'team',
cycle: billingCycle,
checkoutType: 'new',
billingOpId: response.billing_op_id,
paymentIntentSource: options.paymentIntentSource
})
if (response.status === 'needs_payment_method') {
// A needs_payment_method response without a URL is unusable: surface it to
// the caller's error handling rather than silently dropping the user home

View File

@@ -30,6 +30,39 @@ describe('TelemetryRegistry', () => {
expect(b.trackSearchQuery).toHaveBeenCalledExactlyOnceWith(payload)
})
it('dispatches trackBeginCheckout with intent metadata to every provider', () => {
const a: TelemetryProvider = { trackBeginCheckout: vi.fn() }
const b: TelemetryProvider = {}
const registry = new TelemetryRegistry()
registry.registerProvider(a)
registry.registerProvider(b)
const metadata = {
user_id: 'user-1',
tier: 'pro' as const,
cycle: 'monthly' as const,
checkout_type: 'new' as const,
payment_intent_source: 'subscribe_to_run' as const
}
registry.trackBeginCheckout(metadata)
expect(a.trackBeginCheckout).toHaveBeenCalledExactlyOnceWith(metadata)
})
it('dispatches trackAddApiCreditButtonClicked with its source', () => {
const provider: TelemetryProvider = {
trackAddApiCreditButtonClicked: vi.fn()
}
const registry = new TelemetryRegistry()
registry.registerProvider(provider)
registry.trackAddApiCreditButtonClicked({ source: 'credits_panel' })
expect(
provider.trackAddApiCreditButtonClicked
).toHaveBeenCalledExactlyOnceWith({ source: 'credits_panel' })
})
it('skips providers that do not implement trackSearchQuery', () => {
const empty: TelemetryProvider = {}
const registry = new TelemetryRegistry()

View File

@@ -1,6 +1,7 @@
import type { AuditLog } from '@/services/customerEventsService'
import type {
AddCreditsClickMetadata,
AuthMetadata,
BeginCheckoutMetadata,
DefaultViewSetMetadata,
@@ -99,8 +100,10 @@ export class TelemetryRegistry implements TelemetryDispatcher {
this.dispatch((provider) => provider.trackMonthlySubscriptionCancelled?.())
}
trackAddApiCreditButtonClicked(): void {
this.dispatch((provider) => provider.trackAddApiCreditButtonClicked?.())
trackAddApiCreditButtonClicked(metadata?: AddCreditsClickMetadata): void {
this.dispatch((provider) =>
provider.trackAddApiCreditButtonClicked?.(metadata)
)
}
trackApiCreditTopupButtonPurchaseClicked(amount: number): void {

View File

@@ -313,6 +313,42 @@ describe('PostHogTelemetryProvider', () => {
)
})
it('captures begin_checkout with intent metadata', async () => {
const provider = createProvider()
await vi.dynamicImportSettled()
provider.trackBeginCheckout({
user_id: 'user-1',
tier: 'pro',
cycle: 'monthly',
checkout_type: 'new',
payment_intent_source: 'subscribe_to_run'
})
expect(hoisted.mockCapture).toHaveBeenCalledWith(
TelemetryEvents.BEGIN_CHECKOUT,
{
user_id: 'user-1',
tier: 'pro',
cycle: 'monthly',
checkout_type: 'new',
payment_intent_source: 'subscribe_to_run'
}
)
})
it('captures add-credit clicks with their source', async () => {
const provider = createProvider()
await vi.dynamicImportSettled()
provider.trackAddApiCreditButtonClicked({ source: 'credits_panel' })
expect(hoisted.mockCapture).toHaveBeenCalledWith(
TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED,
{ source: 'credits_panel' }
)
})
it('captures share attribution events', async () => {
const provider = createProvider()
await vi.dynamicImportSettled()

View File

@@ -10,7 +10,9 @@ import { remoteConfig } from '@/platform/remoteConfig/remoteConfig'
import type { RemoteConfig } from '@/platform/remoteConfig/types'
import type {
AddCreditsClickMetadata,
AuthMetadata,
BeginCheckoutMetadata,
DefaultViewSetMetadata,
EnterLinearMetadata,
ShareFlowMetadata,
@@ -350,8 +352,12 @@ export class PostHogTelemetryProvider implements TelemetryProvider {
this.trackEvent(eventName, metadata)
}
trackAddApiCreditButtonClicked(): void {
this.trackEvent(TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED)
trackAddApiCreditButtonClicked(metadata?: AddCreditsClickMetadata): void {
this.trackEvent(TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED, metadata)
}
trackBeginCheckout(metadata: BeginCheckoutMetadata): void {
this.trackEvent(TelemetryEvents.BEGIN_CHECKOUT, metadata)
}
trackMonthlySubscriptionSucceeded(

View File

@@ -115,6 +115,17 @@ describe('HostTelemetrySink', () => {
)
})
it('forwards add-credit clicks with their source', () => {
new HostTelemetrySink().trackAddApiCreditButtonClicked({
source: 'avatar_menu'
})
expect(state.capture).toHaveBeenCalledExactlyOnceWith(
TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED,
{ source: 'avatar_menu' }
)
})
it('does nothing when the host bridge is absent', () => {
delete window.__comfyDesktop2

View File

@@ -10,6 +10,7 @@ import {
import type { AuditLog } from '@/services/customerEventsService'
import type {
AddCreditsClickMetadata,
AuthMetadata,
BeginCheckoutMetadata,
DefaultViewSetMetadata,
@@ -126,8 +127,8 @@ export class HostTelemetrySink implements TelemetryProvider {
this.capture(TelemetryEvents.MONTHLY_SUBSCRIPTION_CANCELLED)
}
trackAddApiCreditButtonClicked(): void {
this.capture(TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED)
trackAddApiCreditButtonClicked(metadata?: AddCreditsClickMetadata): void {
this.capture(TelemetryEvents.ADD_API_CREDIT_BUTTON_CLICKED, metadata)
}
trackApiCreditTopupButtonPurchaseClicked(amount: number): void {

View File

@@ -12,12 +12,29 @@
* 3. Check dist/assets/*.js files contain no tracking code
*/
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import type { BillingCycle } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import type { AuditLog } from '@/services/customerEventsService'
import type { AppMode } from '@/utils/appMode'
export type PaymentIntentSource =
| 'subscription_required'
| 'out_of_credits'
| 'top_up_blocked'
| 'deep_link'
| 'subscribe_to_run'
| 'subscribe_now_button'
| 'upgrade_to_add_credits'
| 'settings_billing_panel'
| 'avatar_menu_plans'
| 'team_members_panel'
| 'invite_member_upsell'
| 'upload_model_upgrade'
| 'team_upgrade_resume'
export type SubscriptionCheckoutType = 'new' | 'change'
export type SubscriptionCheckoutTier = TierKey | 'team'
/**
* Authentication metadata for sign-up tracking
*/
@@ -426,16 +443,23 @@ export interface CheckoutAttributionMetadata {
export interface SubscriptionMetadata {
current_tier?: string
reason?: SubscriptionDialogReason
reason?: PaymentIntentSource
}
export interface AddCreditsClickMetadata {
source: 'credits_panel' | 'avatar_menu' | 'settings_billing_panel'
}
export interface BeginCheckoutMetadata
extends Record<string, unknown>, CheckoutAttributionMetadata {
user_id: string
tier: TierKey
tier: SubscriptionCheckoutTier
cycle: BillingCycle
checkout_type: 'new' | 'change'
checkout_type: SubscriptionCheckoutType
checkout_attempt_id?: string
billing_op_id?: string
previous_tier?: TierKey
payment_intent_source?: PaymentIntentSource
}
interface EcommerceItemMetadata {
@@ -457,8 +481,9 @@ export interface SubscriptionSuccessMetadata extends Record<string, unknown> {
checkout_attempt_id: string
tier: TierKey
cycle: BillingCycle
checkout_type: 'new' | 'change'
checkout_type: SubscriptionCheckoutType
previous_tier?: TierKey
payment_intent_source?: PaymentIntentSource
value: number
currency: string
ecommerce: EcommerceMetadata
@@ -489,7 +514,7 @@ export interface TelemetryProvider {
metadata?: SubscriptionSuccessMetadata
): void
trackMonthlySubscriptionCancelled?(): void
trackAddApiCreditButtonClicked?(): void
trackAddApiCreditButtonClicked?(metadata?: AddCreditsClickMetadata): void
trackApiCreditTopupButtonPurchaseClicked?(amount: number): void
trackApiCreditTopupSucceeded?(): void
trackWorkspaceInviteSent?(metadata: WorkspaceInviteMetadata): void

View File

@@ -321,7 +321,7 @@ const handleOpenWorkspaceSettings = () => {
}
const handleOpenPlansAndPricing = () => {
subscriptionDialog.showPricingTable()
subscriptionDialog.showPricingTable({ reason: 'avatar_menu_plans' })
emit('close')
}
@@ -336,13 +336,12 @@ const handleOpenPlanAndCreditsSettings = () => {
}
const handleUpgradeToAddCredits = () => {
subscriptionDialog.showPricingTable()
subscriptionDialog.showPricingTable({ reason: 'upgrade_to_add_credits' })
emit('close')
}
const handleTopUp = () => {
// Track purchase credits entry from avatar popover
useTelemetry()?.trackAddApiCreditButtonClicked()
useTelemetry()?.trackAddApiCreditButtonClicked({ source: 'avatar_menu' })
dialogService.showTopUpCreditsDialog()
emit('close')
}

View File

@@ -391,12 +391,13 @@ const showZeroState = computed(
)
function handleSubscribeWorkspace() {
showSubscriptionDialog()
showSubscriptionDialog({ reason: 'settings_billing_panel' })
}
function handleUpgrade() {
if (isFreeTierPlan.value) showPricingTable()
else showSubscriptionDialog()
if (isFreeTierPlan.value)
showPricingTable({ reason: 'settings_billing_panel' })
else showSubscriptionDialog({ reason: 'settings_billing_panel' })
}
function handleViewMoreDetails() {

View File

@@ -113,7 +113,7 @@ import { cn } from '@comfyorg/tailwind-utils'
import { useEventListener } from '@vueuse/core'
import Button from '@/components/ui/button/Button.vue'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import { useSubscriptionCheckout } from '@/platform/workspace/composables/useSubscriptionCheckout'
import SubscriptionAddPaymentPreviewWorkspace from './SubscriptionAddPaymentPreviewWorkspace.vue'
@@ -123,7 +123,7 @@ import UnifiedPricingTable from './UnifiedPricingTable.vue'
const { onClose, reason, initialPlanMode } = defineProps<{
onClose: () => void
reason?: SubscriptionDialogReason
reason?: PaymentIntentSource
initialPlanMode?: 'personal' | 'team'
}>()
@@ -152,7 +152,7 @@ const {
handleConfirmTransition,
handleTeamSubscribe,
handleResubscribe
} = useSubscriptionCheckout(emit)
} = useSubscriptionCheckout(emit, reason)
// Backspace mirrors the back arrow on the confirm step, but never while an
// editable element is focused (let it delete text there).

View File

@@ -5,7 +5,7 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'
import { ref } from 'vue'
import { createI18n } from 'vue-i18n'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import SubscriptionRequiredDialogContentWorkspace from './SubscriptionRequiredDialogContentWorkspace.vue'
@@ -17,25 +17,10 @@ const mockHandleResubscribe = vi.fn()
const mockHandleSuccessClose = vi.fn()
const mockCheckoutStep = ref<'pricing' | 'preview' | 'success'>('pricing')
const mockPreviewData = ref<{ transition_type: string } | null>(null)
const mockUseSubscriptionCheckout = vi.hoisted(() => vi.fn())
vi.mock('@/platform/workspace/composables/useSubscriptionCheckout', () => ({
useSubscriptionCheckout: () => ({
checkoutStep: mockCheckoutStep,
isLoadingPreview: ref(false),
loadingTier: ref(null),
isSubscribing: ref(false),
isResubscribing: ref(false),
previewData: mockPreviewData,
selectedTierKey: ref('standard'),
selectedBillingCycle: ref('yearly'),
isPolling: ref(false),
handleSubscribeClick: mockHandleSubscribeClick,
handleBackToPricing: mockHandleBackToPricing,
handleAddCreditCard: mockHandleAddCreditCard,
handleConfirmTransition: mockHandleConfirmTransition,
handleResubscribe: mockHandleResubscribe,
handleSuccessClose: mockHandleSuccessClose
})
useSubscriptionCheckout: mockUseSubscriptionCheckout
}))
const i18n = createI18n({
@@ -91,7 +76,7 @@ const SuccessStub = {
function renderComponent(
props: {
onClose?: () => void
reason?: SubscriptionDialogReason
reason?: PaymentIntentSource
isPersonal?: boolean
} = {}
) {
@@ -121,6 +106,23 @@ function renderComponent(
describe('SubscriptionRequiredDialogContentWorkspace', () => {
beforeEach(() => {
vi.clearAllMocks()
mockUseSubscriptionCheckout.mockReturnValue({
checkoutStep: mockCheckoutStep,
isLoadingPreview: ref(false),
loadingTier: ref(null),
isSubscribing: ref(false),
isResubscribing: ref(false),
previewData: mockPreviewData,
selectedTierKey: ref('standard'),
selectedBillingCycle: ref('yearly'),
isPolling: ref(false),
handleSubscribeClick: mockHandleSubscribeClick,
handleBackToPricing: mockHandleBackToPricing,
handleAddCreditCard: mockHandleAddCreditCard,
handleConfirmTransition: mockHandleConfirmTransition,
handleResubscribe: mockHandleResubscribe,
handleSuccessClose: mockHandleSuccessClose
})
mockCheckoutStep.value = 'pricing'
mockPreviewData.value = null
})
@@ -132,6 +134,15 @@ describe('SubscriptionRequiredDialogContentWorkspace', () => {
expect(screen.queryByTestId('transition-preview')).not.toBeInTheDocument()
})
it('passes the reason into subscription checkout', () => {
renderComponent({ reason: 'out_of_credits' })
expect(mockUseSubscriptionCheckout).toHaveBeenCalledWith(
expect.any(Function),
'out_of_credits'
)
})
it('shows the team workspace header by default', () => {
renderComponent()
expect(screen.getByText('Team Workspace')).toBeInTheDocument()

View File

@@ -116,7 +116,7 @@
import { cn } from '@comfyorg/tailwind-utils'
import Button from '@/components/ui/button/Button.vue'
import type { SubscriptionDialogReason } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import { useSubscriptionCheckout } from '@/platform/workspace/composables/useSubscriptionCheckout'
import PricingTableWorkspace from './PricingTableWorkspace.vue'
@@ -130,7 +130,7 @@ const {
isPersonal = false
} = defineProps<{
onClose: () => void
reason?: SubscriptionDialogReason
reason?: PaymentIntentSource
isPersonal?: boolean
}>()
@@ -154,7 +154,7 @@ const {
handleConfirmTransition,
handleResubscribe,
handleSuccessClose
} = useSubscriptionCheckout(emit)
} = useSubscriptionCheckout(emit, reason)
</script>
<style scoped>

View File

@@ -61,6 +61,9 @@ function onDismiss() {
function onUpgrade() {
dialogStore.closeDialog({ key: 'invite-member-upsell' })
subscriptionDialog.show({ planMode: 'team' })
subscriptionDialog.show({
planMode: 'team',
reason: 'invite_member_upsell'
})
}
</script>

View File

@@ -277,7 +277,7 @@ export function useMembersPanel() {
}
function showTeamPlans() {
subscriptionDialog.show({ planMode: 'team' })
subscriptionDialog.show({ planMode: 'team', reason: 'team_members_panel' })
}
return {

View File

@@ -1,8 +1,9 @@
import { createTestingPinia } from '@pinia/testing'
import { setActivePinia } from 'pinia'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import { computed } from 'vue'
import { computed, reactive } from 'vue'
import type { PaymentIntentSource } from '@/platform/telemetry/types'
import type { Plan } from '@/platform/workspace/api/workspaceApi'
import { findPlanSlug } from './useSubscriptionCheckout'
@@ -75,7 +76,9 @@ const {
mockPlans,
mockResubscribe,
mockToastAdd,
mockStartOperation
mockStartOperation,
mockTrackBeginCheckout,
mockUserId
} = vi.hoisted(() => ({
mockSubscribe: vi.fn(),
mockPreviewSubscribe: vi.fn(),
@@ -84,7 +87,9 @@ const {
mockPlans: { value: [] as Plan[] },
mockResubscribe: vi.fn(),
mockToastAdd: vi.fn(),
mockStartOperation: vi.fn()
mockStartOperation: vi.fn(),
mockTrackBeginCheckout: vi.fn(),
mockUserId: { value: 'user-1' as string | null }
}))
vi.mock('@/composables/billing/useBillingContext', () => ({
@@ -119,7 +124,14 @@ vi.mock('primevue/usetoast', () => ({
}))
vi.mock('@/platform/telemetry', () => ({
useTelemetry: () => ({ trackMonthlySubscriptionSucceeded: vi.fn() })
useTelemetry: () => ({
trackMonthlySubscriptionSucceeded: vi.fn(),
trackBeginCheckout: mockTrackBeginCheckout
})
}))
vi.mock('@/stores/authStore', () => ({
useAuthStore: () => reactive({ userId: computed(() => mockUserId.value) })
}))
vi.mock('vue-i18n', async (importOriginal) => {
@@ -135,10 +147,10 @@ vi.mock('vue-i18n', async (importOriginal) => {
describe('useSubscriptionCheckout', () => {
let emit: ReturnType<typeof vi.fn>
async function setup() {
async function setup(paymentIntentSource?: PaymentIntentSource) {
const { useSubscriptionCheckout } =
await import('./useSubscriptionCheckout')
return useSubscriptionCheckout(emit as never)
return useSubscriptionCheckout(emit as never, paymentIntentSource)
}
beforeEach(() => {
@@ -146,6 +158,7 @@ describe('useSubscriptionCheckout', () => {
vi.clearAllMocks()
mockPlans.value = allPlans()
mockStartOperation.mockResolvedValue({ status: 'succeeded' })
mockUserId.value = 'user-1'
emit = vi.fn()
})
@@ -459,6 +472,13 @@ describe('useSubscriptionCheckout', () => {
cancelUrl: 'https://platform.comfy.org/payment/failed'
})
expect(checkout.checkoutStep.value).toBe('success')
expect(mockTrackBeginCheckout).toHaveBeenCalledWith(
expect.objectContaining({
tier: 'team',
checkout_type: 'new',
billing_op_id: 'op-team-1'
})
)
})
it('uses the annual plan slug for the yearly cycle', async () => {
@@ -553,6 +573,39 @@ describe('useSubscriptionCheckout', () => {
detail: 'Team payment failed'
})
)
expect(mockTrackBeginCheckout).not.toHaveBeenCalled()
})
it('keeps team checkout_type as change when the preview request fails', async () => {
const checkout = await setup()
mockPreviewSubscribe.mockRejectedValueOnce(new Error('not supported'))
await checkout.handleSubscribeTeamClick({
stop: {
id: 'team_1400',
usd: 1400,
credits: 295_400,
discountedUsd: 1295
},
billingCycle: 'monthly',
isChange: true
})
mockSubscribe.mockResolvedValueOnce({
status: 'subscribed',
billing_op_id: 'op-team-change'
})
mockFetchStatus.mockResolvedValueOnce(undefined)
mockFetchBalance.mockResolvedValueOnce(undefined)
await checkout.handleTeamSubscribe()
expect(mockTrackBeginCheckout).toHaveBeenCalledWith(
expect.objectContaining({
tier: 'team',
cycle: 'monthly',
checkout_type: 'change',
billing_op_id: 'op-team-change'
})
)
})
})
@@ -603,6 +656,47 @@ describe('useSubscriptionCheckout', () => {
expect(checkout.checkoutStep.value).toBe('success')
})
it('skips begin_checkout when no user id is available', async () => {
mockUserId.value = null
const checkout = await setup('subscribe_to_run')
checkout.selectedTierKey.value = 'standard'
checkout.selectedBillingCycle.value = 'yearly'
mockSubscribe.mockResolvedValueOnce({
status: 'subscribed',
billing_op_id: 'op-1'
})
mockFetchStatus.mockResolvedValueOnce(undefined)
mockFetchBalance.mockResolvedValueOnce(undefined)
await checkout.handleAddCreditCard()
expect(mockTrackBeginCheckout).not.toHaveBeenCalled()
mockUserId.value = 'user-1'
})
it('fires begin_checkout carrying the payment intent source', async () => {
const checkout = await setup('subscribe_to_run')
checkout.selectedTierKey.value = 'standard'
checkout.selectedBillingCycle.value = 'yearly'
mockSubscribe.mockResolvedValueOnce({
status: 'subscribed',
billing_op_id: 'op-1'
})
mockFetchStatus.mockResolvedValueOnce(undefined)
mockFetchBalance.mockResolvedValueOnce(undefined)
await checkout.handleAddCreditCard()
expect(mockTrackBeginCheckout).toHaveBeenCalledWith({
user_id: 'user-1',
tier: 'standard',
cycle: 'yearly',
checkout_type: 'new',
billing_op_id: 'op-1',
payment_intent_source: 'subscribe_to_run'
})
})
it('opens payment URL when needs_payment_method', async () => {
const checkout = await setup()
checkout.selectedTierKey.value = 'standard'
@@ -720,6 +814,7 @@ describe('useSubscriptionCheckout', () => {
detail: 'Payment failed'
})
)
expect(mockTrackBeginCheckout).not.toHaveBeenCalled()
})
})

View File

@@ -9,16 +9,26 @@ import type { TeamPlanSelection } from '@/platform/cloud/subscription/constants/
import type { TierKey } from '@/platform/cloud/subscription/constants/tierPricing'
import type { BillingCycle } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import { useTelemetry } from '@/platform/telemetry'
import type {
PaymentIntentSource,
SubscriptionCheckoutType
} from '@/platform/telemetry/types'
import type {
Plan,
PreviewSubscribeResponse,
SubscribeResponse
} from '@/platform/workspace/api/workspaceApi'
import { useBillingOperationStore } from '@/platform/workspace/stores/billingOperationStore'
import { trackWorkspaceCheckoutStarted } from '@/platform/workspace/utils/workspaceCheckoutTelemetry'
type CheckoutStep = 'pricing' | 'preview' | 'success'
type CheckoutTierKey = Exclude<TierKey, 'free' | 'founder'>
interface SelectedTeamCheckout {
stop: TeamPlanSelection
checkoutType: SubscriptionCheckoutType
}
/**
* Which screen the `preview` step shows. Only a change prorates: a team change
* carries `previewData` (handleSubscribeTeamClick sets it solely for an immediate
@@ -45,9 +55,12 @@ export function findPlanSlug(
return plan?.slug ?? null
}
export function useSubscriptionCheckout(emit: {
(e: 'close', subscribed: boolean): void
}) {
export function useSubscriptionCheckout(
emit: {
(e: 'close', subscribed: boolean): void
},
paymentIntentSource?: PaymentIntentSource
) {
const { t } = useI18n()
const toast = useToast()
const {
@@ -68,13 +81,16 @@ export function useSubscriptionCheckout(emit: {
const isResubscribing = ref(false)
const previewData = ref<PreviewSubscribeResponse | null>(null)
const selectedTierKey = ref<CheckoutTierKey | null>(null)
const selectedTeamStop = ref<TeamPlanSelection | null>(null)
const selectedTeamCheckout = ref<SelectedTeamCheckout | null>(null)
const selectedBillingCycle = ref<BillingCycle>('yearly')
const isPolling = computed(() => billingOperationStore.hasPendingOperations)
const isTeamCheckout = computed(() => selectedTeamStop.value !== null)
const selectedTeamStop = computed(
() => selectedTeamCheckout.value?.stop ?? null
)
const isTeamCheckout = computed(() => selectedTeamCheckout.value !== null)
const previewVariant = computed<PreviewVariant>(() => {
if (selectedTeamStop.value) {
if (selectedTeamCheckout.value) {
return previewData.value ? 'team-change' : 'team-new'
}
if (previewData.value) {
@@ -154,7 +170,10 @@ export function useSubscriptionCheckout(emit: {
billingCycle: BillingCycle
isChange?: boolean
}) {
selectedTeamStop.value = payload.stop
selectedTeamCheckout.value = {
stop: payload.stop,
checkoutType: payload.isChange ? 'change' : 'new'
}
selectedBillingCycle.value = payload.billingCycle
selectedTierKey.value = null
previewData.value = null
@@ -182,7 +201,7 @@ export function useSubscriptionCheckout(emit: {
function handleBackToPricing() {
checkoutStep.value = 'pricing'
previewData.value = null
selectedTeamStop.value = null
selectedTeamCheckout.value = null
}
function handleSuccessClose() {
@@ -190,20 +209,34 @@ export function useSubscriptionCheckout(emit: {
}
async function handleSubscription() {
if (!selectedTierKey.value) return
const tierKey = selectedTierKey.value
if (!tierKey) return
const billingCycle = selectedBillingCycle.value
const checkoutType =
previewData.value &&
previewData.value.transition_type !== 'new_subscription'
? 'change'
: 'new'
isSubscribing.value = true
try {
const planSlug = getApiPlanSlug(
selectedTierKey.value,
selectedBillingCycle.value
)
const planSlug = getApiPlanSlug(tierKey, billingCycle)
if (!planSlug) return
const response = await subscribe(planSlug, {
returnUrl: `${getComfyPlatformBaseUrl()}/payment/success`,
cancelUrl: `${getComfyPlatformBaseUrl()}/payment/failed`
})
if (response) {
trackWorkspaceCheckoutStarted({
tier: tierKey,
cycle: billingCycle,
checkoutType,
billingOpId: response.billing_op_id,
paymentIntentSource
})
}
await handleSubscribeResponse(response)
} catch (error) {
showSubscribeError(error)
@@ -269,8 +302,8 @@ export function useSubscriptionCheckout(emit: {
}
async function handleTeamSubscription() {
const stop = selectedTeamStop.value
if (!stop?.id) {
const teamCheckout = selectedTeamCheckout.value
if (!teamCheckout?.stop.id) {
toast.add({
severity: 'error',
summary: t('subscription.teamPlan.name'),
@@ -279,16 +312,28 @@ export function useSubscriptionCheckout(emit: {
return
}
const { stop, checkoutType } = teamCheckout
const billingCycle = selectedBillingCycle.value
isSubscribing.value = true
try {
const planSlug = getTeamPlanSlug(selectedBillingCycle.value)
const planSlug = getTeamPlanSlug(billingCycle)
const response = await subscribe(planSlug, {
teamCreditStopId: stop.id,
billingCycle: selectedBillingCycle.value,
billingCycle,
returnUrl: `${getComfyPlatformBaseUrl()}/payment/success`,
cancelUrl: `${getComfyPlatformBaseUrl()}/payment/failed`
})
if (response) {
trackWorkspaceCheckoutStarted({
tier: 'team',
cycle: billingCycle,
checkoutType,
billingOpId: response.billing_op_id,
paymentIntentSource
})
}
await handleSubscribeResponse(response)
} catch (error) {
showSubscribeError(error)

View File

@@ -2,6 +2,7 @@ import { computed, ref, shallowRef } from 'vue'
import { useBillingPlans } from '@/platform/cloud/subscription/composables/useBillingPlans'
import { useSubscriptionDialog } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type { SubscriptionDialogOptions } from '@/platform/cloud/subscription/composables/useSubscriptionDialog'
import type {
BillingBalanceResponse,
BillingStatusResponse,
@@ -275,12 +276,12 @@ export function useWorkspaceBilling(): BillingState & BillingActions {
async function requireActiveSubscription(): Promise<void> {
await fetchStatus()
if (!isActiveSubscription.value) {
subscriptionDialog.show()
subscriptionDialog.show({ reason: 'subscription_required' })
}
}
function showSubscriptionDialog(): void {
subscriptionDialog.show()
function showSubscriptionDialog(options?: SubscriptionDialogOptions): void {
subscriptionDialog.show(options)
}
return {

View File

@@ -0,0 +1,38 @@
import { useTelemetry } from '@/platform/telemetry'
import type {
PaymentIntentSource,
SubscriptionCheckoutTier,
SubscriptionCheckoutType
} from '@/platform/telemetry/types'
import type { BillingCycle } from '@/platform/cloud/subscription/utils/subscriptionTierRank'
import { useAuthStore } from '@/stores/authStore'
interface TrackWorkspaceCheckoutStartedOptions {
tier: SubscriptionCheckoutTier
cycle: BillingCycle
checkoutType: SubscriptionCheckoutType
billingOpId: string
paymentIntentSource?: PaymentIntentSource
}
export function trackWorkspaceCheckoutStarted({
tier,
cycle,
checkoutType,
billingOpId,
paymentIntentSource
}: TrackWorkspaceCheckoutStartedOptions) {
const { userId } = useAuthStore()
if (!userId) return
useTelemetry()?.trackBeginCheckout({
user_id: userId,
tier,
cycle,
checkout_type: checkoutType,
billing_op_id: billingOpId,
...(paymentIntentSource
? { payment_intent_source: paymentIntentSource }
: {})
})
}

Some files were not shown because too many files have changed in this diff Show More