Commit Graph

11 Commits

Author SHA1 Message Date
Alexander Brown
8c9328c1b2 feat: add eslint-plugin-playwright via oxlint JS plugins (#11136)
## Summary

Add eslint-plugin-playwright as an oxlint JS plugin scoped to
browser_tests/, enforcing Playwright best practices at lint time.

## Changes

- **What**: Configure eslint-plugin-playwright@2.10.1 via oxlint's alpha
`jsPlugins` field (`.oxlintrc.json` override scoped to
`browser_tests/**/*.ts`). 18 recommended rules +
`prefer-native-locators` + `require-to-pass-timeout` at error severity.
All 173 initial violations resolved (config, auto-fix, manual fixes).
`no-force-option` set to off — 28 violations need triage (canvas overlay
workarounds vs unnecessary force) in a dedicated PR.
- **Dependencies**: `eslint-plugin-playwright@^2.10.1` (devDependency,
required by oxlint jsPlugins at runtime)

## Review Focus

- `.oxlintrc.json` override structure — this is the first use of
oxlint's JS plugins alpha feature in this repo
- Manual fixes in spec files: `waitForSelector` → `locator.waitFor`,
deprecated page methods → locator equivalents, `toPass()` timeout
additions
- Compound CSS selectors replaced with `.and()` (Playwright native
locator composition) to avoid `prefer-native-locators` suppressions
- Lint script changes in `package.json` to include `browser_tests/` in
oxlint targets

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
2026-04-11 01:25:14 +00:00
Alexander Brown
0132c77c7d test: harden 82 Playwright specs for deterministic CI runs (#10967)
## Summary

Harden 98 E2E spec files and 8 fixtures/helpers for deterministic CI
runs by replacing race-prone patterns with retry-safe alternatives.

No source code changes -- only `browser_tests/` is touched.

## Changes

- **E2E spec hardening** (98 spec files, 6 fixtures, 2 helpers):

  | Fix class | Sites | Examples |
  |-----------|-------|---------:|
| `expect(await ...)` -> `expect.poll()` | ~153 | interaction,
defaultKeybindings, workflows, featureFlags |
| `const x = await loc.count(); expect(x)` -> `toHaveCount()` | ~19 |
menu, linkInteraction, assets, bottomPanelShortcuts |
| `nextFrame()` -> `waitForHidden()` after menu clicks | ~22 |
contextMenu, rightClickMenu, subgraphHelper |
| Redundant `nextFrame()` removed | many | defaultKeybindings, minimap,
builderSaveFlow |
| `expect(async () => { ... }).toPass()` retry blocks | 5 | interaction
(graphdialog dismiss guard) |
| `force:true` removed from `BaseDialog.close()` | 1 | BaseDialog
fixture |
| ContextMenu `waitForHidden` simplified (check-then-act race removed) |
1 | ContextMenu fixture |
| Non-deterministic node order -> proximity-based selection | 1 |
interaction (toggle dom widget) |
  | Tight poll timeout (250ms) -> >=2000ms | 2 | templates |

- **Helper improvements**: Exposed locator getters on
`ComfyPage.domWidgets`, `ToastHelper.toastErrors`, and
`WorkflowsSidebarTab.activeWorkflowLabel` so callers can use retrying
assertions (`toHaveCount()`, `toHaveText()`) directly.

- **Flake pattern catalog**: Added section 7 table to
`browser_tests/FLAKE_PREVENTION_RULES.md` documenting 8 pattern classes
for reviewers and future authors.

- **Docs**: Fixed bad examples in `browser_tests/README.md` to use
`expect.poll()`.

- **Breaking**: None
- **Dependencies**: None

## Review Focus

- All fixes follow the rules in
`browser_tests/FLAKE_PREVENTION_RULES.md`
- No behavioral changes to tests -- only timing/retry strategy is
updated
- The `ContextMenu.waitForHidden` simplification removes a
swallowed-error anti-pattern; both locators now use direct `waitFor({
state: 'hidden' })`

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: github-actions <github-actions@github.com>
2026-04-09 20:50:56 -07:00
Alexander Brown
3b78dfbe1c test: migrate browser_tests/ to @e2e/ path alias and add lint rule (#10958)
## Summary

Complete the @e2e/ path alias migration started in #10735 by converting
all 354 remaining relative imports and adding a lint rule to prevent
backsliding.

## Changes

- **What**: Migrate all relative imports in browser_tests/ to use
`@e2e/` (intra-directory) and `@/` (src/ imports) path aliases. Add
`no-restricted-imports` ESLint rule banning `./` and `../` imports in
`browser_tests/**/*.ts`. Suppress pre-existing oxlint `no-eval` and
`no-console` warnings exposed by touching those files.

## Review Focus

- ESLint flat-config merging: the `@playwright/test` ban and
relative-import ban are in two separate blocks to avoid last-match-wins
collision with the `useI18n`/`useVirtualList` blocks higher in the
config.
- The `['./**', '../**']` glob patterns (not `['./*', '../*']`) are
needed to catch multi-level relative paths like `../../../src/foo`.

Follows up on #10735

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10958-test-migrate-browser_tests-to-e2e-path-alias-and-add-lint-rule-33c6d73d365081649d1be771eac986fd)
by [Unito](https://www.unito.io)

Co-authored-by: Amp <amp@ampcode.com>
2026-04-08 11:28:59 -07:00
jaeone94
a321d66583 refactor: remove legacy missing nodes dialog (#10102)
## Summary

- Remove the legacy missing nodes modal dialog and migrate all
functionality to the existing Error Overlay / TabErrors system
- Migrate core node version warning from `MissingCoreNodesMessage.vue`
to `MissingNodeCard` in the errors tab
- Remove `Comfy.Workflow.ShowMissingNodesWarning` setting (errors tab
always surfaces missing nodes)
- Delete 6 legacy files: `useMissingNodesDialog.ts`,
`MissingNodesContent.vue`, `MissingNodesFooter.vue`,
`MissingNodesHeader.vue`, `MissingCoreNodesMessage.vue` and its test
- Rename `showMissingNodesDialog`/`showMissingModelsDialog` params to
`showMissingNodes`/`showMissingModels`
- Add `errorOverlay` and `missingNodeCard` to centralized `TestIds`
- Migrate all E2E tests from legacy dialog selectors to error overlay
testIds
- Add new E2E test: MissingNodeCard visible via "See Errors" button flow
- Add new E2E test: subgraph missing node type verified by expanding
pack row
- Add `surfaceMissingNodes` unit tests to `executionErrorStore`
- Guard `semver.compare` against invalid version strings
- Add `role="alert"`, `aria-hidden` for accessibility
- Use reactive props destructuring in `MissingNodeCard` and
`MissingPackGroupRow`

**Net change: -669 lines** (19 files, +323 / -992)


<img width="733" height="579" alt="image"
src="https://github.com/user-attachments/assets/c497809d-b176-43bf-9872-34bd74c6ea0d"
/>


## Test plan

- [x] Unit tests: MissingNodeCard core node warning (7 tests)
- [x] Unit tests: surfaceMissingNodes (4 tests)
- [x] Unit tests: workflowService showPendingWarnings (updated)
- [x] E2E: Error overlay visible on missing nodes workflow
- [x] E2E: Error overlay visible on subgraph missing nodes
- [x] E2E: MissingNodeCard visible via See Errors button
- [x] E2E: Subgraph node type visible after expanding pack row
- [x] E2E: Error overlay does not resurface on undo/redo
- [x] E2E: Error overlay does not reappear on workflow tab switch
- [x] Typecheck, lint, knip all passing

## Related issues

- Closes #9923 (partially — `errorOverlay` and `missingNodeCard` added
to TestIds)
- References #10027 (mock hoisting inconsistency)
- References #10033 (i18n-based test selectors)
- References #10085 (DDD layer violation + focusedErrorNodeId)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10102-refactor-remove-legacy-missing-nodes-dialog-3256d73d365081c194d2e90bc6401846)
by [Unito](https://www.unito.io)
2026-03-17 11:14:44 +09:00
Alexander Brown
f2d5bfab73 test(browser): refactor browser tests for reliability and maintainability (#8510)
## Summary

Major refactoring of browser tests to improve reliability,
maintainability, and type safety.

## Changes

### Test Infrastructure Decomposition
- Decomposed `ComfyPage.ts` (~1000 lines) into focused helpers:
- `CanvasHelper`, `DebugHelper`, `SubgraphHelper`,
`NodeOperationsHelper`
- `SettingsHelper`, `WorkflowHelper`, `ClipboardHelper`,
`KeyboardHelper`
- Created `ContextMenu` page object, `BaseDialog` base class, and
`BottomPanel` page object
- Extracted `DefaultGraphPositions` constants

### Locator Stability
- Added `data-testid` attributes to Vue components (sidebar, dialogs,
node library)
- Created centralized `selectors.ts` with test ID constants
- Replaced fragile CSS selectors (`.nth()`, `:nth-child()`) with
`getByTestId`/`getByRole`

### Performance & Reliability
- Removed `setTimeout` anti-patterns (replaced with `waitForFunction`)
- Replaced `waitForTimeout` with retrying assertions
- Replaced hardcoded coordinates with computed `NodeReference` positions
- Enforced LF line endings for all text files

### Type Safety
- Enabled `no-explicit-any` lint rule for browser_tests via oxlint
- Purged `as any` casts from browser_tests
- Added Window type augmentation for standardized window access
- Added proper type annotations throughout

### Bug Fixes
- Restored `ExtensionManager` API contract
- Removed test-only settings from production schema
- Fixed flaky selectors and missing test setup

## Testing
- All browser tests pass
- Typecheck passes


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
* Overhauled browser E2E test infrastructure with many new
helpers/fixtures, updated test APIs, and CI test container image bumped
for consistency.

* **Chores**
* Standardized line endings and applied stricter lint rules for browser
tests; workspace dependency version updated.

* **Documentation**
* Updated Playwright and TypeScript testing guidance and test-run
commands.

* **UI**
* Added stable data-testids to multiple components to improve
testability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-03 12:29:40 -08:00
Alexander Brown
82bacb82a7 test: add Playwright test tags for filtering (@smoke, @slow, @screenshot, domains) (#8441)
## Summary

Adds structured test tags to all 54 Playwright test files to enable
flexible test filtering during development and CI.

## Tags Added

| Tag | Count | Purpose |
|-----|-------|---------|
| `@screenshot` | 32 files | Tests with visual assertions
(`toHaveScreenshot`) |
| `@smoke` | 5 files | Quick essential tests for fast validation |
| `@slow` | 5 files | Long-running tests (templates, subgraph,
featureFlags) |
| `@canvas` | 15 files | Canvas/graph rendering tests |
| `@node` | 10 files | Node behavior tests |
| `@ui` | 8 files | UI component tests |
| `@widget` | 5 files | Widget-specific tests |
| `@workflow` | 3 files | Workflow operations |
| `@subgraph` | 1 file | Subgraph functionality |
| `@keyboard` | 2 files | Keyboard shortcuts |
| `@settings` | 2 files | Settings/preferences |

## Usage Examples

```bash
# Quick validation (~16 tests, ~30s)
pnpm test:browser -- --grep @smoke

# Skip slow tests for faster CI feedback
pnpm test:browser -- --grep-invert @slow

# Skip visual tests (useful for local development without snapshots)
pnpm test:browser -- --grep-invert @screenshot

# Run only canvas-related tests
pnpm test:browser -- --grep @canvas

# Combine filters
pnpm test:browser -- --grep @smoke --grep-invert @screenshot
```

## Implementation Details

- Uses Playwright's native tag syntax: `test.describe('Name', { tag:
'@tag' }, ...)`
- Tags inherit from describe blocks to child tests
- Preserves existing project-level tags: `@mobile`, `@2x`, `@0.5x`
- Multiple tags supported: `{ tag: ['@screenshot', '@smoke'] }`

## Test Plan

- [x] All existing tests pass unchanged
- [x] Tag filtering works with `--grep` and `--grep-invert`

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-8441-test-add-Playwright-test-tags-for-filtering-smoke-slow-screenshot-domains-2f76d73d36508184990ec859c8fd7629)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: GitHub Action <action@github.com>
2026-01-29 16:34:56 -08:00
Benjamin Lu
bc4549244e test(e2e): align test default menu to Top; make legacy specs explicit (#5746)
## Summary
UseNewMenu has been defaulted to Top in the app for over a year;
Playwright’s test default lagged behind. This PR aligns the test default
with reality and keeps legacy specs stable.

## Changes
- tests(e2e): default to 'Top' via fixture; specs that previously relied
on the old implicit default now explicitly set 'Comfy.UseNewMenu' to
'Disabled'.
- docs(browser-tests): remove outdated README note suggesting tests set
'Top' manually.

## Review Focus
- Intentional uses of 'Top' and 'Bottom' remain unchanged.
- Confirm ComfyPage default remains 'Top' (see
browser_tests/fixtures/ComfyPage.ts).

## Screenshots (if applicable)
N/A

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-5746-test-e2e-align-test-default-menu-to-Top-make-legacy-specs-explicit-2786d73d365081218d06c1346f3ae18e)
by [Unito](https://www.unito.io)

---------

Co-authored-by: github-actions <github-actions@github.com>
2025-09-24 18:43:22 -07:00
Christian Byrne
efd9b04a6e [refactor] Organize all browser test assets into logical folders (#5058)
* move subgraph test assets into subfolder

* [refactor] Organize browser test assets into logical folders

Reorganized test assets for better maintainability:
- groupnodes/: GroupNode feature tests
- groups/: Visual grouping tests
- missing/: Missing nodes/models tests
- links/: Link-related tests
- inputs/: Input widget tests
- widgets/: Widget-specific tests
- nodes/: Node-related tests
- workflowInMedia/: Workflow media files

Updated all loadWorkflow references to use new folder structure.
Fixed programmatic filename references to prevent test failures.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [fix] Update mobile test to use new asset path

* [fix] Update remaining loadWorkflow calls to use new folder structure

* [fix] Fix remaining programmatic filename references

* [fix] Run prettier formatting

* [fix] Fix setupWorkflowsDirectory references to use correct folder paths

* [refactor] Rename subgraph folder to subgraphs for consistency

* [fix] Fix breadcrumb name in subgraph DOM widget test

* Update test expectations [skip ci]

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions <github-actions@github.com>
2025-08-18 10:39:53 -07:00
Chenlei Hu
05023b7889 Migrate defaultInput widget option (#3364)
Co-authored-by: github-actions <github-actions@github.com>
2025-04-09 13:51:34 -04:00
Chenlei Hu
ac3bd7a848 [Test] Add test on dynamically added inputs (#3344)
Co-authored-by: github-actions <github-actions@github.com>
2025-04-07 22:22:32 -04:00
Chenlei Hu
8923ec51fd [Refactor] Re-organize browser_tests directory (#3105) 2025-03-17 15:32:14 -04:00