## Motivation
Browser tests mock API responses with `route.fulfill()` using untyped
inline JSON. When the OpenAPI spec changes, these mocks silently drift —
mismatches aren't caught at compile time and only surface as test
failures at runtime.
We already have auto-generated types from OpenAPI and manual Zod
schemas. This PR makes those types the source of truth for test mock
data.
From Mar 27 PR review session action item: "instruct agents to use
schemas and types when writing browser tests."
## Type packages and their API coverage
The frontend has two OpenAPI-generated type packages, each targeting a
different backend API with a different code generation tool:
| Package | Target API | Generator | TS types | Zod schemas |
|---------|-----------|-----------|----------|-------------|
| `@comfyorg/registry-types` | Registry API (node packages, releases,
subscriptions, customers) | `openapi-typescript` | Yes | **No** |
| `@comfyorg/ingest-types` | Ingest API (hub workflows, asset uploads,
workspaces) | `@hey-api/openapi-ts` | Yes | Yes |
Additionally, Python backend endpoints (`/api/queue`, `/api/features`,
`/api/settings`, etc.) are typed via manual Zod schemas in
`src/schemas/apiSchema.ts`.
This PR applies **compile-time type checking** using these existing
types. Runtime validation via Zod `.parse()` is not yet possible for all
endpoints because `registry-types` does not generate Zod schemas — this
requires a separate migration of `registry-types` to
`@hey-api/openapi-ts` (#10674).
## Summary
- Add "Typed API Mocks" guideline to `docs/guidance/playwright.md` with
a sources-of-truth table mapping endpoint categories to their type
packages
- Add rule to `AGENTS.md` Playwright section requiring typed mock data
- Refactor `releaseNotifications.spec.ts` to use `ReleaseNote` type
(from `registry-types`) via `createMockRelease()` factory
- Annotate template mock in `templates.spec.ts` with
`WorkflowTemplates[]` type
Refs #10656
## Example workflow: writing a new typed E2E test mock
When adding a new `route.fulfill()` mock, follow these steps:
### 1. Identify the type source
Check which API the endpoint belongs to:
| Endpoint category | Type source | Zod available |
|---|---|---|
| Ingest API (hub, billing, workflows) | `@comfyorg/ingest-types` | Yes
— use `.parse()` |
| Registry API (releases, nodes, publishers) |
`@comfyorg/registry-types` | Not yet (#10674) — TS type only |
| Python backend (queue, history, settings) | `src/schemas/apiSchema.ts`
| Yes — use `z.infer` |
| Templates | `src/platform/workflow/templates/types/template.ts` | No —
TS type only |
### 2. Create a typed factory (with Zod when available)
**Ingest API endpoints** — Zod schemas exist, use `.parse()` for runtime
validation:
```typescript
import { zBillingStatusResponse } from '@comfyorg/ingest-types/zod'
import type { BillingStatusResponse } from '@comfyorg/ingest-types'
function createMockBillingStatus(
overrides?: Partial<BillingStatusResponse>
): BillingStatusResponse {
return zBillingStatusResponse.parse({
plan: 'free',
credits_remaining: 100,
renewal_date: '2026-04-28T00:00:00Z',
...overrides
})
}
```
**Registry API endpoints** — TS type only (Zod not yet generated):
```typescript
import type { ReleaseNote } from '../../src/platform/updates/common/releaseService'
function createMockRelease(
overrides?: Partial<ReleaseNote>
): ReleaseNote {
return {
id: 1,
project: 'comfyui',
version: 'v0.3.44',
attention: 'medium',
content: '## New Features',
published_at: new Date().toISOString(),
...overrides
}
}
```
### 3. Use in test
```typescript
test('should show upgrade banner for free plan', async ({ comfyPage }) => {
await comfyPage.page.route('**/billing/status', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify(createMockBillingStatus({ plan: 'free' }))
})
})
await comfyPage.setup()
await expect(comfyPage.page.getByText('Upgrade')).toBeVisible()
})
```
The factory pattern keeps test bodies focused on **what varies** (the
override) rather than the full response shape.
## Scope decisions
| File | Decision | Reason |
|------|----------|--------|
| `releaseNotifications.spec.ts` | Typed | `ReleaseNote` type available
from `registry-types` |
| `templates.spec.ts` | Typed | `WorkflowTemplates` type available in
`src/platform/workflow/templates/types/` |
| `QueueHelper.ts` | Skipped | Dead code — instantiated but never called
in any test |
| `FeatureFlagHelper.ts` | Skipped | Response type is inherently
`Record<string, unknown>`, no stronger type exists |
| Fixture factories | Deferred | Coordinate with Ben's fixture
restructuring work to avoid duplication |
## Follow-up work
Sub-issues of #10656:
- #10670 — Clean up dead `QueueHelper` or rewrite against `/api/jobs`
endpoint
- #10671 — Expand typed factory pattern to more endpoints
- #10672 — Evaluate OpenAPI generation for excluded Python backend
endpoints
- #10674 — Migrate `registry-types` from `openapi-typescript` to
`@hey-api/openapi-ts` to enable Zod schema generation
## Test plan
- [x] `pnpm typecheck:browser` passes
- [x] `pnpm lint` passes
- [ ] Existing `releaseNotifications` and `templates` tests pass in CI
## Summary
Major refactoring of browser tests to improve reliability,
maintainability, and type safety.
## Changes
### Test Infrastructure Decomposition
- Decomposed `ComfyPage.ts` (~1000 lines) into focused helpers:
- `CanvasHelper`, `DebugHelper`, `SubgraphHelper`,
`NodeOperationsHelper`
- `SettingsHelper`, `WorkflowHelper`, `ClipboardHelper`,
`KeyboardHelper`
- Created `ContextMenu` page object, `BaseDialog` base class, and
`BottomPanel` page object
- Extracted `DefaultGraphPositions` constants
### Locator Stability
- Added `data-testid` attributes to Vue components (sidebar, dialogs,
node library)
- Created centralized `selectors.ts` with test ID constants
- Replaced fragile CSS selectors (`.nth()`, `:nth-child()`) with
`getByTestId`/`getByRole`
### Performance & Reliability
- Removed `setTimeout` anti-patterns (replaced with `waitForFunction`)
- Replaced `waitForTimeout` with retrying assertions
- Replaced hardcoded coordinates with computed `NodeReference` positions
- Enforced LF line endings for all text files
### Type Safety
- Enabled `no-explicit-any` lint rule for browser_tests via oxlint
- Purged `as any` casts from browser_tests
- Added Window type augmentation for standardized window access
- Added proper type annotations throughout
### Bug Fixes
- Restored `ExtensionManager` API contract
- Removed test-only settings from production schema
- Fixed flaky selectors and missing test setup
## Testing
- All browser tests pass
- Typecheck passes
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Overhauled browser E2E test infrastructure with many new
helpers/fixtures, updated test APIs, and CI test container image bumped
for consistency.
* **Chores**
* Standardized line endings and applied stricter lint rules for browser
tests; workspace dependency version updated.
* **Documentation**
* Updated Playwright and TypeScript testing guidance and test-run
commands.
* **UI**
* Added stable data-testids to multiple components to improve
testability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
