docs: add QA report 2026-03-07 (macOS CI)

Automated QA run covering all frontend routes and features.
34 pass, 1 fail (404 route → download), 42 skipped (canvas/node ops require nodes).
Notable: persistent overlay bug after concurrent zoom dropdown + error dialog.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Anka
2026-03-07 04:33:45 +00:00
committed by snomiao
parent 7d73327f0c
commit 7b1eea5f96
19 changed files with 199 additions and 0 deletions

View File

@@ -0,0 +1,199 @@
# QA Report: ComfyUI Frontend
**Date**: 2026-03-07
**Environment**: CI (macOS, Chromium via Playwright MCP)
**Frontend Version**: ComfyUI_frontend v1.41.13 (commit 9cbc0d97b28f4be3d959e7df30bf306a38f76d84)
**Backend Version**: ComfyUI 0.16.3
**Agent**: Claude Sonnet 4.6
**Server URL**: http://127.0.0.1:8188
**Branch**: sno-skills
## Summary
| Category | Pass | Fail | Skip | Total |
| --------------- | ---- | ---- | ---- | ----- |
| Routes & Load | 3 | 1 | 0 | 4 |
| Canvas | 3 | 0 | 11 | 14 |
| Node Operations | 0 | 0 | 10 | 10 |
| Sidebar | 6 | 0 | 1 | 7 |
| Topbar | 6 | 0 | 1 | 7 |
| Settings | 6 | 0 | 1 | 7 |
| Bottom Panel | 3 | 0 | 1 | 4 |
| Execution | 2 | 0 | 3 | 5 |
| File Operations | 0 | 0 | 6 | 6 |
| Advanced | 3 | 0 | 2 | 5 |
| Error Handling | 1 | 0 | 4 | 5 |
| Responsive | 1 | 0 | 2 | 3 |
| **Total** | **34** | **1** | **42** | **77** |
## Results
### Routes & Load
- [x] Root route loads — redirects to `/user-select` (multi-user mode), GraphView renders after login
- [x] User-select route — displays user selection UI with new user / existing user fields
- [x] Default redirect — `/` correctly redirects to `/user-select` in multi-user mode
- [ ] 404 handling — navigating to `/nonexistent` triggered a file **download** instead of a 404 page (backend serves unknown paths as files)
### Canvas & Graph View
- [x] Canvas renders — LiteGraph canvas visible with grid, FPS counter active (8243 FPS observed)
- [x] Fit view — Period (`.`) key and Fit View button both respond; "Empty canvas" alert shown correctly
- [x] Zoom controls — Zoom Controls button opens dropdown with Zoom In/Out/Fit and editable % spinbutton
- [ ] Pan canvas — skip (canvas `<canvas>` element not reachable via accessibility tree in Playwright)
- [ ] Add node via double-click — skip (canvas not reachable via accessibility tree)
- [ ] Add node via search — tested via Node Library search (see Sidebar); in-canvas search not testable
- [ ] Delete node — skip (no nodes present)
- [ ] Connect nodes — skip (no nodes present)
- [ ] Disconnect nodes — skip (no nodes present)
- [ ] Multi-select — skip (no nodes present)
- [ ] Copy/Paste — skip (no nodes present)
- [ ] Undo/Redo — skip (no undoable actions taken)
- [ ] Node context menu — skip (no nodes present)
- [ ] Canvas context menu — skip (canvas not reachable)
### Node Operations
- [ ] All node operations — skip (no nodes present on canvas; requires adding nodes first)
### Sidebar Tabs
- [x] Node Library tab (N) — opens with Bookmarked, Subgraph Blueprints, Partner Nodes, Comfy Nodes, Extensions categories
- [x] Model Library tab (M) — opens with model folders (checkpoints, loras, vae, text_encoders, diffusion_models, etc.)
- [x] Workflows tab (W) — opens with empty state ("No workflows found")
- [x] Assets tab (A) — opens with Generated / Imported tabs; "No generated files found" shown correctly
- [x] Tab toggle — clicking active sidebar button closes the panel
- [x] Search in sidebar — Node Library search for "KSampler" returns KSampler, KSampler (Advanced), and related sampler nodes
- [ ] Drag node from library — skip (drag-and-drop onto canvas not testable via accessibility)
### Topbar & Workflow Tabs
- [x] Workflow tab display — "Unsaved Workflow" shown in tab bar with unsaved indicator (•)
- [x] New workflow — `+` button creates "Unsaved Workflow (2)" tab; tab bar shows both
- [x] Tab context menu — right-click shows: Rename (disabled), Duplicate, Add to Bookmarks (disabled), Save, Save As, Export, Export (API), Clear Workflow, Close Tab, Close Tabs to Left/Right, Close Other Tabs
- [x] Multiple tabs — two tabs open simultaneously; switching between them works correctly
- [x] Queue button — Run button present and triggers execution (error shown for empty workflow as expected)
- [x] Batch count — Increment/Decrement buttons work; value changes from 1 → 2
- [x] Workflow actions menu — Graph dropdown shows: Rename, Duplicate, Add to Bookmarks, Save, Save As, Export, Export (API), Clear Workflow, Delete Workflow
- [ ] Rename workflow — skip (Rename disabled for unsaved workflow)
### Settings Dialog
- [x] Open settings — Ctrl+, opens Settings dialog correctly
- [x] Settings navigation — left panel shows: Account (User), Application Settings (Comfy, Lite Graph, Appearance, Mask Editor, 3D), Special Settings (Keybinding, Extension, About)
- [x] Search settings — searching "locale" filters to show Comfy > Locale > Language setting
- [x] Keybindings tab — table renders with all keybindings (New Blank Workflow, Open Workflow, Save Workflow, etc.) with Edit/Reset/Delete actions
- [x] About tab — shows ComfyUI 0.16.3, ComfyUI_frontend v1.41.13, Templates v0.9.10, system info (OS: darwin, Python 3.10.11, PyTorch 2.10.0, RAM 7 GB, CPU device)
- [x] Close settings — Escape key closes dialog
- [ ] Change a setting — skip (not tested to avoid side effects)
### Bottom Panel
- [x] Toggle panel — "Toggle Bottom Panel" button opens/closes bottom panel correctly
- [x] Logs tab — server log output displayed (including backend tracebacks)
- [x] Shortcuts tab — Keyboard Shortcuts panel with Essential and View Controls tabs, Manage Shortcuts link
- [ ] Ctrl+` shortcut — skip (backtick key behavior not explicitly verified)
### Execution & Queue
- [x] Queue prompt — Run button queues execution; shows "1 ERROR" alert for empty workflow (correct)
- [x] Job history — queue panel opens with "All" and "Completed" tabs, Clear queue, Filter/Sort buttons
- [ ] Queue progress — skip (no GPU, CPU execution not triggered)
- [ ] Interrupt — skip (no running execution)
- [ ] Clear history — skip (no history present)
### Workflow File Operations
- [ ] Save workflow — skip (file dialog not available in CI headless mode)
- [ ] Open workflow — skip (file dialog not available in CI headless mode)
- [ ] Export JSON — skip (file download not verifiable in CI)
- [ ] Import workflow — skip (file dialog not available in CI)
- [ ] Load default — skip (not tested)
- [ ] Clear workflow — skip (not tested)
### Advanced Features
- [x] Minimap — Alt+M / Show Minimap button toggles minimap overlay in bottom-right of canvas
- [x] Focus mode (App mode) — "Enter app mode" button shows simplified app view; "Enter node graph" in hamburger menu returns to graph
- [x] Link visibility — Hide Links / Show Links toggle button works correctly
- [ ] Canvas lock — skip (H/V key behavior not tested)
- [ ] Subgraph — skip (no nodes to create subgraph from)
### Error Handling
- [x] Empty workflow error — running empty workflow shows "The workflow does not contain any output nodes (e.g. Save Image, Preview Image) to produce a result." with Dismiss / See Errors options
- [ ] Missing nodes dialog — skip (no workflow with unknown nodes loaded)
- [ ] Missing models dialog — skip (not triggered)
- [ ] Network error — skip (not tested)
- [ ] Invalid workflow — skip (not tested)
### Responsive & Accessibility
- [x] Properties panel resize — Workflow Overview properties panel opens/closes as a side panel
- [ ] Window resize — skip (not tested)
- [ ] Keyboard navigation — skip (not fully tested)
---
## Issues Found
### Issue 1: Persistent pointer-blocking overlay after concurrent zoom dropdown + error dialog
- **Severity**: Major
- **Steps to reproduce**:
1. Open the zoom controls dropdown (click "Zoom Controls" button)
2. While dropdown is open, the Run button error alert appears simultaneously
3. Press Escape to attempt dismissal
4. A `<div class="fixed inset-0 z-1200">` overlay from the PrimeVue splitter component remains on the page
5. All subsequent pointer interactions on the page fail with "subtree intercepts pointer events"
6. Full page reload required to recover
- **Expected**: Dismissing dialogs/dropdowns should cleanly remove any overlay divs; interaction should be restored immediately
- **Actual**: Overlay persists indefinitely, blocking all clicks until page reload
- **Screenshot**: `docs/qa/screenshots/16-run-error-empty.png` (state before block), page reload required to continue
### Issue 2: Navigating to unknown route triggers file download
- **Severity**: Minor
- **Steps to reproduce**: Navigate to `http://127.0.0.1:8188/nonexistent`
- **Expected**: 404 page or redirect to main application
- **Actual**: Browser triggers a file download — the ComfyUI backend serves unknown paths as file downloads (backend behavior, not a frontend bug)
- **Note**: This is expected ComfyUI backend behavior but may surprise users who mistype the URL
### Issue 3: Backend error in server logs — Unknown user: default
- **Severity**: Minor / Informational
- **Observed in**: Bottom panel > Logs tab
- **Log content**: `KeyError: 'Unknown user: default'` from `app/user_manager.py line 67 in get_request_user_id`
- **Context**: Occurs on page load in multi-user mode when the session tries to access a "default" user that was not created. CI creates a "qa-user" but the system also attempts to look up "default".
### Issue 4: App Mode "Back to workflow" button unreachable
- **Severity**: Minor
- **Steps to reproduce**: Enter App Mode with empty workflow; attempt to click "Back to workflow" button
- **Expected**: Button should be clickable
- **Actual**: Button is outside the viewport and Playwright click times out. Workaround: use hamburger menu > "Enter node graph"
- **Note**: May be a layout issue specific to empty-workflow App Mode at default viewport size (756×441 or similar)
### Issue 5: Console 404 errors on startup
- **Severity**: Cosmetic / Informational
- **URLs failing**:
- `/api/userdata/user.css` — user-specific CSS file (expected when not uploaded)
- `/user.css` — same
- `workflows/.index.json` — workflow index (expected for new user)
- `api/userdata/comfy.templates.json` — user-level templates override (expected)
- `/favicon.ico` — missing favicon
- `subgraphs` endpoint — 404 in CI environment
- **Note**: All are gracefully handled; UI continues to function normally
---
## Notes
- **Environment**: macOS CI runner, CPU-only PyTorch (--cpu flag), 7 GB RAM, Python 3.10.11
- **Multi-user mode**: Server started with `--multi-user` flag; user "qa-user" was created for this test run
- **Canvas interaction**: The LiteGraph canvas is a `<canvas>` element that does not expose an accessibility tree. Node-level operations (add, connect, delete, bypass, mute, etc.) could not be tested via Playwright's accessibility-based selectors. These require coordinate-based interaction or direct JS injection.
- **Performance**: FPS counter ranged from 8 FPS (initial load) to 243 FPS (after clearing overlays), indicating normal canvas performance variation.
- **Templates**: Template gallery loaded successfully with 13 templates across Getting Started and other categories.
- **Workflow persistence**: Page reload correctly restored the last workflow state (Unsaved Workflow 2 was active after reload).
- **Overall**: The application loads correctly, all major UI surfaces are functional, sidebar navigation works, settings are comprehensive, and error handling for common cases (empty workflow run) is clear and informative.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB