Commit Graph

21 Commits

Author SHA1 Message Date
Christian Byrne
ba9f3481fb test(infra): cloud Playwright project with @cloud/@oss tagging (#10546)
## What

Adds a `cloud` Playwright project so E2E tests can run against
`DISTRIBUTION=cloud` builds, with `@cloud` / `@oss` test tagging.

## Why

100+ usages of `isCloud` / `DISTRIBUTION` across 9 categories (API
routing, UI visibility, settings, auth). Zero cloud test infrastructure
existed — cloud-specific UI components (LoginButton, SubscribeButton,
etc.) had no E2E coverage path.

## Investigation: Runtime Toggle

Investigated whether `isCloud` could be made runtime-toggleable in
dev/test mode (via `window.__FORCE_CLOUD__`). **Not feasible** —
`__DISTRIBUTION__` is a Vite `define` compile-time constant used for
dead-code elimination. Runtime override would break tree-shaking in
production.

Full investigation:
`research/architecture/cloud-runtime-toggle-investigation.md`

## What's included

### Playwright Config
- New `cloud` project alongside existing `chromium`
- Cloud project: `grep: /@cloud/` — only runs `@cloud` tagged tests
- Chromium project: `grepInvert: /@cloud/` — excludes cloud tests

### Build Script
- `npm run build:cloud` → `DISTRIBUTION=cloud vite build`

### Test Tagging Convention
```typescript
test('works in both', async () => { ... });
test('subscription button visible @cloud', async () => { ... });
test('install manager prompt @oss', async () => { ... });
```

### Example Tests
- 2 cloud-only tests validating cloud UI visibility

## NOT included (future work)
- CI workflow job for cloud tests (separate PR)
- Cloud project is opt-in — not run by default locally

## Unblocks
- Cloud-specific E2E tests for entire team
- TB-03 LoginButton, TB-04 SubscribeButton (@Kaili Yang)
- DLG-04 SignIn, DLG-06 CancelSubscription

Part of: Test Coverage Q2 Overhaul

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10546-test-infra-cloud-Playwright-project-with-cloud-oss-tagging-32f6d73d3650810ebb59dea8ce4891e9)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-03-28 22:34:37 -07:00
Dante
98a9facc7d tool: add CSS containment audit skill and Playwright diagnostic (#10026)
## Summary

- Add a Playwright-based diagnostic tool (`@audit` tagged) that
automatically detects DOM elements where CSS `contain: layout style`
would improve rendering performance
- Extend `ComfyPage` fixture and `playwright.config.ts` to support
`@audit` tag (excluded from CI, perf infra enabled)
- Add `/contain-audit` skill definition documenting the workflow

## How it works

1. Loads the 245-node workflow in a real browser
2. Walks the DOM tree and scores every element by subtree size and
sizing constraints
3. For each high-scoring candidate, applies `contain: layout style` via
JS
4. Measures rendering performance (style recalcs, layouts, task
duration) before and after
5. Takes before/after screenshots to detect visual breakage
6. Outputs a ranked report to console

## Test plan

- [ ] `pnpm typecheck` passes
- [ ] `pnpm typecheck:browser` passes
- [ ] `pnpm lint` passes
- [ ] Existing Playwright tests unaffected (`@audit` excluded from CI
via `grepInvert`)
- [ ] Run `pnpm exec playwright test
browser_tests/tests/containAudit.spec.ts --project=chromium` locally
with dev server

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10026-tool-add-CSS-containment-audit-skill-and-Playwright-diagnostic-3256d73d365081b29470df164f798f7d)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 10:08:41 +09:00
Christian Byrne
8e215b3174 feat: add performance testing infrastructure with CDP metrics (#9170)
## Summary

Add a permanent, non-failing performance regression detection system
using Chrome DevTools Protocol metrics, with automatic PR commenting.

## Changes

- **What**: Performance testing infrastructure — `PerformanceHelper`
fixture class using CDP `Performance.getMetrics` to collect
`RecalcStyleCount`, `LayoutCount`, `LayoutDuration`, `TaskDuration`,
`JSHeapUsedSize`. Adds `@perf` Playwright project (Chromium-only,
single-threaded, 60s timeout), 4 baseline perf tests, CI workflow with
sticky PR comment reporting, and `perf-report.js` script for generating
markdown comparison tables.

## Review Focus

- `PerformanceHelper` uses `page.context().newCDPSession(page)` — CDP is
Chromium-only, so perf metrics are not collected on Firefox. This is
intentional since CDP gives us browser-level style recalc/layout counts
that `performance.mark/measure` cannot capture.
- The CI workflow uses `continue-on-error: true` so perf tests never
block merging.
- Baseline comparison uses `dawidd6/action-download-artifact` to
download metrics from the target branch, following the same pattern as
`pr-size-report.yaml`.

## Stack

This is the foundation PR for the Firefox performance fix stack:
1. **→ This PR: perf testing infrastructure**
2. `perf/fix-cursor-cache` — cursor style caching (depends on this)
3. `perf/fix-subgraph-svg` — SVG pre-rasterization (depends on this)
4. `perf/fix-clippath-raf` — RAF batching for clip-path (depends on
this)

PRs 2-4 are independent of each other.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-9170-feat-add-performance-testing-infrastructure-with-CDP-metrics-3116d73d3650817cb43def6f8e9917f8)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-02-25 20:09:57 -08:00
Alexander Brown
60647fd5b9 devex: Add script to bake in local options for Playwright runs (#6668)
## Summary

Try it out: `pnpm test:browser:local`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6668-devex-Add-script-to-bake-in-local-options-for-Playwright-runs-2aa6d73d36508130b1d6d0b2e79a4641)
by [Unito](https://www.unito.io)
2025-11-12 18:39:41 -08:00
Alexander Brown
2bf92a0e57 devex: Add Suggested local Playwright config modifications from \rowser_tests/README.md\ as a toggle block (#5256) 2025-08-29 13:27:21 -07:00
Alexander Brown
1b83d6b5a6 Migration: pnpm (#5215)
* migration: npm to pnpm
Step 1, package and lockfile

* migration: npm to pnpm
Step 2: docs / LLM instructions

* migration: npm to pnpm
Step 3: More documentation updates

* migration: npm to pnpm
Step 4: Even more documentation

* migration: npm to pnpm
Step 5: GitHub Actions

* migration: npm to pnpm
Step 6: PNPM installation in actions. This merge is going to be painful.

* migration: npm to pnpm
Unignore and add pnpm lockfile.

* migration: npm to pnpm
package-lock.json -> pnpm-lock.yaml

* migration: explicit @primeuix/styled, move glob to prod deps

* migration: more explicit deps required by the importmap plugin and vite

* fix: missed merge artifact

* fix: Make sure pnpm is available to install wrangler

* migration: pnpm for dev-release.yaml

* migration: new setup action version
Won't work until that is updated and a new release is cut.

* migration: Playwright needs uuid

* migration: Add explicit deps for lobehub

* chore(version-bump.yaml): change cache from npm to pnpm to optimize package management and improve build performance

* migration: install pnpm in version-bump action

---------

Co-authored-by: snomiao <snomiao@gmail.com>
2025-08-27 06:10:15 -07:00
snomiao
7d7c6a324a [feat] Enhanced auto-retry configuration for flaky tests (#5089)
- Increased Playwright test retries from 2 to 3 in CI environment
- Added Vitest retry configuration (2 retries) for unit tests in CI
- Addresses issues #4658 and #4416 with consistently flaky tests

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-18 21:41:42 -07:00
Christian Byrne
5cc269eff1 Fix Alt+click create reroute (2/2) (#4863)
Co-authored-by: github-actions <github-actions@github.com>
2025-08-09 11:13:37 -07:00
Chenlei Hu
eff68ae7a8 [Test] Only run chromium-2x test for tagged items (#3050) 2025-03-14 13:57:23 -04:00
filtered
4c19e1ba3a Speed up E2E tests using fully parallel (#1429)
With flaky tests / async bugs all dealt with, fullyParallel can be restored.
2024-11-04 20:33:50 -05:00
filtered
caa3ac2068 Add playwright concurrency - multi-user mode (#1400)
* Add playwright concurrency - multi-user mode

* Add extra server params

* nit

* Update to v2.1 action

---------

Co-authored-by: huchenlei <huchenlei@proton.me>
2024-11-02 09:51:24 -04:00
bymyself
fabcbaec82 Restore backend state when Playwright test finishes (#1168)
* Restore backend state when Playwright test finishes. Resolve #1094

* Add warning when env var not set

* Rename and replace with scaffolding option for models dir

* Rename

* Define another env var [skip ci]

* Fix paths [skip ci]

* Update README.md

---------

Co-authored-by: Chenlei Hu <huchenlei@proton.me>
2024-10-09 15:05:19 -04:00
Chenlei Hu
fdb01c06f2 Split playwright tests to multiple runners by project (#789) 2024-09-12 11:35:36 +09:00
Chenlei Hu
baf0bc8de4 Run all playwright tests under dpr=2 (#779)
* Run all playwright tests under dpr=2

* Update test expectations [skip ci]

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-09-11 11:33:40 +09:00
bymyself
15758101aa Fix searchbox popover on touch devices (#773)
* Add delay on touch pointer event when opening searchbox

* Add playwright mobile test
2024-09-11 08:55:04 +09:00
Chenlei Hu
4b83ee3918 link release pops up searchbox by default (#348) (#350)
* link release pops up searchbox by default (#348)

* link release pops up searchbox by default

* Update browser test

* Fix tests

* Update test expectations [skip ci]

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-08-09 16:33:30 -04:00
Chenlei Hu
e179f75387 Apply new code format standard (#217) 2024-07-25 10:10:18 -04:00
Chenlei Hu
1b7db43f8a Format everything (#211) 2024-07-23 15:40:54 -04:00
Chenlei Hu
a2afdd74b2 Mount vue app after comfy app init (#167)
* Mount vue app after comfy app

* Emit event when vue app loaded

* Dispatch event to window

* Fix test timeout

* Try observe variable

* Revert

* Update test expectations [skip ci]

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-07-19 18:47:53 -04:00
Chenlei Hu
86ee0767c3 Auto link node on creation (#110)
* Auto link node on creation

* Handle corner case

* Add some browser tests

* Add auto link test

* Force enable

* Confirm setting before running test

* Update test expectations [skip ci]

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-07-11 12:25:46 -04:00
Chenlei Hu
51b925f7ef Add playwright browser test (#48)
* Add playwright

* Add test:browser command

* Remove test examples

* Add basic node tests

* Add drag node test

* Merge workflows

* nit

* Localize jest

* Add local config

* Change working dir

* Use consistent fonts

* Fix emoji fonts

* Update github action to save expectation

* update on test failure

* push to head

* Update test expectations [skip ci]

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-06-24 09:41:40 -04:00