Compare commits

..

7 Commits

Author SHA1 Message Date
DrJKL
78cbfd2a7a fix: DO NOT MERGE: Try to fix the types, behavior is not correct 2025-09-30 16:13:07 -07:00
bymyself
9a3545b42b Fix Rectangle/Rect type compatibility and unify type system
This commit addresses PR review comments by fixing the fundamental
type incompatibility between Rectangle (Array<number>) and strict
tuple types (Rect = [x, y, width, height]).

Key changes:
- Updated Rectangle class methods to accept both Rect and Rectangle unions
- Fixed all measure functions (containsRect, overlapBounding, etc.) to accept Rectangle
- Updated boundingRect interfaces to consistently use Rectangle instead of Rect
- Fixed all tests and Vue components to use Rectangle instead of array literals
- Resolved Point type conflicts between litegraph and pathRenderer modules
- Removed ReadOnlyPoint types and unified with Point arrays

The type system is now consistent: boundingRect properties return Rectangle
objects with full functionality, while Rect remains as a simple tuple type
for data interchange.
2025-09-30 13:11:53 -07:00
bymyself
493fe667b7 [refactor] convert Rectangle from Float64Array to Array inheritance - addresses review feedback
Replace Float64Array inheritance with Array<number> to remove legacy typed array usage.
Maintains compatibility with array indexing (rect[0], rect[1], etc.) and property access
(rect.x, rect.y, etc.) while adding .set() and .subarray() methods for compatibility.

Co-authored-by: webfiltered <webfiltered@users.noreply.github.com>
2025-09-30 13:10:39 -07:00
Christian Byrne
7636b9eaa1 Update src/lib/litegraph/src/LGraphCanvas.ts
Co-authored-by: filtered <176114999+webfiltered@users.noreply.github.com>
2025-09-30 13:10:38 -07:00
bymyself
b661412c5b Update test snapshots to reflect Float32Array removal
- Updated snapshots to show regular arrays instead of Float32Array
- Regenerated snapshots for LGraph, ConfigureGraph tests
- All actively running tests now have correct array expectations

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 13:10:36 -07:00
bymyself
ba4c3525f4 [perf] Fix Float32Array test assertions and link adapter
Fix the remaining Float32Array usage that was causing test failures:
- Update test assertions to expect regular arrays instead of Float32Array
- Convert link adapter Float32Array creation to regular arrays

Resolves: AssertionError: expected [ 50, 60 ] to deeply equal Float32Array[ 50, 60 ]
2025-09-30 13:10:35 -07:00
bymyself
58024fb4f4 [perf] Remove legacy Float32Array usage from LiteGraph
Removes Float32Array usage throughout LiteGraph codebase, eliminating
type conversion overhead for Canvas 2D rendering operations.

## Changes Made

### Core Data Structures
- **LGraphNode**: Convert position/size arrays from Float32Array to regular arrays
- **LLink**: Convert coordinate storage from Float32Array to regular arrays
- **Reroute**: Replace malloc pattern with standard properties
- **LGraphGroup**: Convert boundary arrays from Float32Array to regular arrays

### Rendering Optimizations
- **LGraphCanvas**: Convert static rendering buffers to regular arrays
- **Drawing Functions**: Replace .set() method calls with direct array assignment
- **Measurement**: Update bounds calculation to use regular arrays

### Type System Updates
- **Interfaces**: Update Point/Size/Rect types to support both regular arrays and typed arrays
- **API Compatibility**: Maintain all existing property names and method signatures

## Performance Benefits

- Eliminates Float32Array conversion overhead in Canvas 2D operations
- Reduces memory allocation complexity (removed malloc patterns)
- Improves TypeScript integration with native array support
- Maintains full API compatibility with zero breaking changes

## Background

Float32Array usage was originally designed for WebGL integration, but the
current Canvas 2D rendering pipeline doesn't use WebGL, making the typed
arrays an unnecessary performance overhead. Every Canvas 2D operation that
reads coordinates from Float32Array incurs type conversion costs.
2025-09-30 13:10:33 -07:00
3368 changed files with 136397 additions and 583788 deletions

View File

@@ -1,28 +0,0 @@
---
name: accessibility
description: Reviews UI code for WCAG 2.2 AA accessibility violations
severity-default: medium
tools: [Read, Grep]
---
You are an accessibility auditor reviewing a code diff for WCAG 2.2 AA compliance. Focus on UI changes that affect users with disabilities.
Check for:
1. **Missing form labels** - inputs, selects, textareas without associated `<label>` or `aria-label`/`aria-labelledby`
2. **Missing alt text** - images without `alt` attributes, or decorative images missing `alt=""`
3. **Keyboard navigation** - interactive elements not focusable, custom widgets missing keyboard handlers (Enter, Space, Escape, Arrow keys), focus traps without escape
4. **Focus management** - modals/dialogs that don't trap focus, dynamic content that doesn't move focus appropriately, removed elements without focus recovery
5. **ARIA misuse** - invalid `aria-*` attributes, roles without required children/properties, `aria-hidden` on focusable elements
6. **Color as sole indicator** - using color alone to convey meaning (errors, status) without text/icon alternative
7. **Touch targets** - interactive elements smaller than 24x24 CSS pixels (WCAG 2.2 SC 2.5.8)
8. **Screen reader support** - dynamic content changes without `aria-live` announcements, unlabeled icon buttons, links with only "click here"
9. **Heading hierarchy** - skipped heading levels (h1 → h3), missing page landmarks
Rules:
- Focus on NEW or CHANGED UI in the diff — do not audit the entire existing codebase
- Only flag issues in .vue, .tsx, .jsx, .html, or template-containing files
- Skip non-UI files entirely (stores, services, utils)
- Skip canvas-based content: the LiteGraph node editor renders on `<canvas>` elements, not DOM-based UI. WCAG rules don't fully apply to canvas rendering internals — only audit the DOM-based controls around it (toolbars, panels, dialogs)
- "Critical" for completely inaccessible interactive elements, "major" for missing labels/ARIA, "minor" for enhancement opportunities

View File

@@ -1,118 +0,0 @@
---
name: adr-compliance
description: Checks code changes against Architecture Decision Records, with emphasis on ECS (ADR 0008) and command-pattern (ADR 0003) compliance
severity-default: medium
tools: [Read, Grep, glob]
---
Check that code changes are consistent with the project's Architecture Decision Records in `docs/adr/`.
## Priority 1: ECS and Command-Pattern Compliance (ADR 0008 + ADR 0003)
These are the primary architectural guardrails. Every entity/litegraph change must be checked against them.
### Command Pattern (ADR 0003)
All entity state mutations MUST be expressible as **serializable, idempotent, deterministic commands**. This is required for CRDT sync, undo/redo, cross-environment portability, and gateway backends.
Flag:
- **Direct spatial mutation** — `node.pos = ...`, `node.size = ...`, `group.pos = ...` outside of a store or command. All spatial data flows through `layoutStore` commands.
- **Imperative fire-and-forget mutation** — Any new API that mutates entity state as a side effect rather than producing a serializable command object. Systems should produce command batches, not execute mutations directly.
- **Void-returning mutation APIs** — New entity mutation functions that return `void` instead of a result type (`{ status: 'applied' | 'rejected' | 'no-op' }`). Commands need error/rejection semantics.
- **Auto-incrementing IDs in new entity code** — New entity creation using auto-increment counters without acknowledging the CRDT collision problem. Concurrent environments need globally unique, stable identifiers.
### ECS Architecture (ADR 0008)
The graph domain model is migrating to ECS. New code must not make the migration harder.
Flag:
- **God-object growth** — New methods/properties added to `LGraphNode` (~4k lines), `LGraphCanvas` (~9k lines), `LGraph` (~3k lines), or `Subgraph`. Extract to systems, stores, or composables instead.
- **Mixed data and behavior** — New component-like data structures that contain methods or back-references to parent entities. ECS components are plain data objects.
- **New circular entity dependencies** — New circular imports between `LGraph``Subgraph`, `LGraphNode``LGraphCanvas`, or similar entity classes.
- **Direct `graph._version++`** — Mutating the private version counter directly instead of through a public API. Extensions already depend on this side-channel; it must become a proper API.
### Centralized Registries and ECS-Style Access
All entity data access should move toward centralized query patterns, not instance property access.
Flag:
- **New instance method/property patterns** — Adding `node.someProperty` or `node.someMethod()` for data that should be a component in the World, queried via `world.getComponent(entityId, ComponentType)`.
- **OOP inheritance for entity modeling** — Extending entity classes with new subclasses instead of composing behavior through components and systems.
- **Scattered state** — New entity state stored in multiple locations (class properties, stores, local variables) instead of being consolidated in the World or in a single store.
### Extension Ecosystem Impact
Entity API changes affect 40+ custom node repos. Changes to these patterns require an extension migration path.
Flag when changed without migration guidance:
- `onConnectionsChange`, `onRemoved`, `onAdded`, `onConfigure` callbacks
- `onConnectInput` / `onConnectOutput` validation hooks
- `onWidgetChanged` handlers
- `node.widgets.find(w => w.name === ...)` patterns
- `node.serialize` overrides
- `graph._version++` direct mutation
- `getNodeById` usage patterns
## Priority 2: General ADR Compliance
For all other ADRs, iterate through each file in `docs/adr/` and extract the core lesson. Ensure changed code does not contradict accepted ADRs. Flag contradictions with proposed ADRs as directional guidance.
### How to Apply
1. Read `docs/adr/README.md` to get the full ADR index
2. For each ADR, read the Decision and Consequences sections
3. Check the diff against each ADR's constraints
4. Only flag ACTUAL violations in changed code, not pre-existing patterns
### Skip List
These ADRs can be skipped for most reviews (they cover completed or narrow-scope decisions):
- **ADR 0004** (Rejected — Fork PrimeVue) — only relevant if someone proposes forking PrimeVue again
## How to Check
1. Identify changed files in the entity/litegraph layer: `src/lib/litegraph/`, `src/ecs/`, `src/platform/`, entity-related stores
2. For Priority 1 patterns, use targeted searches:
```
# Direct position mutation
Grep: pattern="\.pos\s*=" path="src/lib/litegraph"
Grep: pattern="\.size\s*=" path="src/lib/litegraph"
# God object growth (new methods)
Grep: pattern="(class LGraphNode|class LGraphCanvas|class LGraph\b)" path="src/lib/litegraph"
# Version mutation
Grep: pattern="_version\+\+" path="src/lib/litegraph"
# Extension callback changes
Grep: pattern="on(ConnectionsChange|Removed|Added|Configure|ConnectInput|ConnectOutput|WidgetChanged)" path="src/lib/litegraph"
```
3. For Priority 2, read `docs/adr/` files and check for contradictions
## Severity Guidelines
| Issue | Severity |
| -------------------------------------------------------- | -------- |
| Imperative mutation API without command-pattern wrapper | high |
| New god-object method on LGraphNode/LGraphCanvas/LGraph | high |
| Breaking extension callback without migration path | high |
| New circular entity dependency | high |
| Direct spatial mutation bypassing command pattern | medium |
| Mixed data/behavior in component-like structures | medium |
| New OOP inheritance pattern for entities | medium |
| Contradicts accepted ADR direction | medium |
| Contradicts proposed ADR direction without justification | low |
## Rules
- Only flag ACTUAL violations in changed code, not pre-existing patterns
- If a change explicitly acknowledges an ADR tradeoff in comments or PR description, lower severity
- Proposed ADRs carry less weight than accepted ones — flag as directional guidance
- Reference the specific ADR number in every finding

View File

@@ -1,35 +0,0 @@
---
name: api-contract
description: Catches breaking changes to public interfaces, window-exposed APIs, event contracts, and exported symbols
severity-default: high
tools: [Grep, Read, glob]
---
You are an API contract reviewer. Your job is to catch breaking changes and contract violations in public-facing interfaces.
## What to Check
1. **Breaking changes to globally exposed APIs** — anything on `window` or other global objects that consumers depend on. Renamed properties, removed methods, changed signatures, changed return types.
2. **Event contract changes** — renamed events, changed event payloads, removed events that listeners may depend on.
3. **Changed function signatures** — parameters reordered, required params added, return type changed on exported functions.
4. **Removed or renamed exports** — any `export` that was previously available and is now gone or renamed without a re-export alias.
5. **REST API changes** — changed endpoints, added required fields, removed response fields, changed status codes.
6. **Type contract narrowing** — a function that used to accept `string | number` now only accepts `string`, or a return type that narrows unexpectedly.
7. **Default value changes** — changing defaults on optional parameters that consumers may rely on.
8. **Store/state shape changes** — renamed store properties, changed state structure that computed properties or watchers may depend on.
## How to Identify the Public API
- Check `package.json` for `"exports"` or `"main"` fields.
- **Window globals**: This repo exposes LiteGraph classes on `window` (e.g., `window['LiteGraph']`, `window['LGraphNode']`, `window['LGraphCanvas']`) and `window['__COMFYUI_FRONTEND_VERSION__']`. These are consumed by custom node extensions and must not be renamed or removed.
- **Extension hooks**: The `app` object and its extension registration system (`app.registerExtension`) is a public contract for third-party custom nodes. Changes to `ComfyApp`, `ComfyApi`, or the extension lifecycle are breaking changes.
- Check AGENTS.md for project-specific API surface definitions.
- Any exported symbol from common entry points (e.g., `src/types/index.ts`) should be treated as potentially public.
## Rules
- ONLY flag changes that break existing consumers.
- Do NOT flag additions (new methods, new exports, new endpoints).
- Do NOT flag internal/private API changes.
- Always check if a re-export or compatibility shim was added before flagging.
- Critical for removed/renamed globals, high for changed export signatures, medium for changed defaults.

View File

@@ -1,27 +0,0 @@
---
name: architecture-reviewer
description: Reviews code for architectural issues like over-engineering, SOLID violations, coupling, and API design
severity-default: medium
tools: [Grep, Read, glob]
---
You are a software architect reviewing a code diff. Focus on structural and design issues.
## What to Check
1. **Over-engineering** — abstractions for single-use cases, premature generalization, unnecessary indirection layers
2. **SOLID violations** — god classes, mixed responsibilities, rigid coupling, interface segregation issues
3. **Separation of concerns** — business logic in UI components, data access in controllers, mixed layers
4. **API design** — inconsistent interfaces, leaky abstractions, unclear contracts
5. **Coupling** — tight coupling between modules, circular dependencies, feature envy
6. **Consistency** — breaking established patterns without justification, inconsistent approaches to similar problems
7. **Dependency direction** — imports going the wrong way in the architecture, lower layers depending on higher
8. **Change amplification** — designs requiring changes in many places for simple feature additions
## Rules
- Focus on structural issues that affect maintainability and evolution.
- Do NOT report bugs, security, or performance issues (other checks handle those).
- Consider whether the code is proportional to the problem it solves.
- "Under-engineering" (missing useful abstractions) is as valid as over-engineering.
- Rate severity by impact on future maintainability.

View File

@@ -1,34 +0,0 @@
---
name: bug-hunter
description: Finds logic errors, off-by-ones, null safety issues, race conditions, and edge cases
severity-default: high
tools: [Read, Grep]
---
You are a bug hunter reviewing a code diff. Your ONLY job is to find bugs - logic errors that will cause incorrect behavior at runtime.
Focus areas:
1. **Off-by-one errors** in loops, slices, and indices
2. **Null/undefined dereferences** - any path where a value could be null but isn't checked
3. **Race conditions** - shared mutable state, async ordering assumptions
4. **Edge cases** - empty arrays, zero values, empty strings, boundary conditions
5. **Type coercion bugs** - loose equality, implicit conversions
6. **Error handling gaps** - unhandled promise rejections, swallowed errors
7. **State mutation bugs** - mutating props, shared references, stale closures
8. **Incorrect boolean logic** - flipped conditions, missing negation, wrong operator precedence
Rules:
- ONLY report actual bugs that will cause wrong behavior
- Do NOT report style issues, naming, or performance
- Do NOT report hypothetical bugs that require implausible inputs
- Each finding must explain the specific runtime failure scenario
## Repo-Specific Bug Patterns
- `z.any()` in Zod schemas disables validation and propagates `any` into TypeScript types — always flag
- Destructuring reactive objects (props, reactive()) without `toRefs()` loses reactivity — flag outside of `defineProps` destructuring
- `ComputedRef<T>` exposed via `defineExpose` or public API should be unwrapped first
- LiteGraph node operations: check for missing null guards on `node.graph` (can be null when node is removed)
- Watch/watchEffect without cleanup for side effects (timers, listeners) — leak on component unmount

View File

@@ -1,50 +0,0 @@
---
name: coderabbit
description: Runs CodeRabbit CLI for AST-aware code quality review
severity-default: medium
tools: [Bash, Read]
---
Run CodeRabbit CLI review on the current changes.
## Steps
1. Check if CodeRabbit CLI is installed:
```bash
which coderabbit
```
If not installed, skip this check and report:
"Skipped: CodeRabbit CLI not installed. Install and authenticate:
```
npm install -g coderabbit
coderabbit auth login
```
See https://docs.coderabbit.ai/guides/cli for setup."
2. Run review:
```bash
coderabbit --prompt-only --type uncommitted
```
If there are committed but unpushed changes, use `--type committed` instead.
3. Parse CodeRabbit's output. Each finding should include:
- File path and line number
- Severity mapped from CodeRabbit's own levels
- Category (logic, security, performance, style, test, architecture, dx)
- Description and suggested fix
## Rate Limiting
If a rate limit is hit, skip and note it. Prefer reading the current quota from CLI/API output rather than assuming a fixed reviews/hour limit.
## Error Handling
- Auth expired: skip and report "CodeRabbit auth expired, run: coderabbit auth login"
- CLI timeout (>120s): skip and note
- Parse error: return raw output with a warning

View File

@@ -1,28 +0,0 @@
---
name: complexity
description: Reviews code for excessive complexity and suggests refactoring opportunities
severity-default: medium
tools: [Grep, Read, glob]
---
You are a complexity and refactoring advisor reviewing a code diff. Focus on code that is unnecessarily complex and will be hard to maintain.
## What to Check
1. **High cyclomatic complexity** — functions with many branching paths (if/else chains, switch statements with >7 cases, nested ternaries). Threshold: complexity >10 is high severity, >15 is critical.
2. **Deep nesting** — code nested >4 levels deep (nested if/for/try blocks). Suggest guard clauses, early returns, or extraction.
3. **Oversized functions** — functions >50 lines that do multiple things. Suggest extraction of cohesive sub-functions.
4. **God classes/modules** — files >500 lines mixing multiple responsibilities. Suggest splitting by concern.
5. **Long parameter lists** — functions with >5 parameters. Suggest parameter objects or builder patterns.
6. **Complex boolean expressions** — conditions with >3 clauses that are hard to parse. Suggest extracting to named boolean variables.
7. **Feature envy** — methods that use data from another class more than their own, suggesting the method belongs elsewhere.
8. **Duplicate logic** — two or more code blocks in the diff doing essentially the same thing with minor variations.
9. **Unnecessary indirection** — wrapper functions that add no value, abstractions for single-use cases, premature generalization.
## Rules
- Only flag complexity in NEW or SIGNIFICANTLY CHANGED code.
- Do NOT suggest refactoring stable, well-tested code that happens to be complex.
- Do NOT flag complexity that is inherent to the problem domain (e.g., state machines, protocol handlers).
- Provide a concrete refactoring approach, not just "this is too complex".
- High severity for code that will likely cause bugs during future modifications, medium for readability improvements, low for optional simplifications.

View File

@@ -1,86 +0,0 @@
---
name: ddd-structure
description: Reviews whether new code is placed in the right domain/layer and follows domain-driven structure principles
severity-default: medium
tools: [Grep, Read, glob]
---
You are a domain-driven design reviewer. Your job is to check whether new or moved code is placed in the correct architectural layer and domain folder.
## Principles
1. **Domain over Technical Layer** — code should be organized by what it does (domain/feature), not by what it is (component/service/store). New files in flat technical folders like `src/components/`, `src/services/`, `src/stores/`, `src/utils/` are a smell if the repo already has domain folders.
2. **Cohesion** — files that change together should live together. A component, its store, its service, and its types for a single feature should be co-located.
3. **Import Direction** — lower layers must not import from higher layers. Check that imports flow in the allowed direction (see Layer Architecture below).
4. **Bounded Contexts** — each domain/feature should have clear boundaries. Cross-domain imports should go through public interfaces, not reach into internal files.
5. **Naming** — folders and files should reflect domain concepts, not technical roles. `workflowExecution.ts` > `service.ts`.
## Layer Architecture
This repo uses a VSCode-style layered architecture with strict unidirectional imports:
```
base → platform → workbench → renderer
```
| Layer | Purpose | Can Import From |
| ------------ | -------------------------------------- | ---------------------------------- |
| `base/` | Pure utilities, no framework deps | Nothing |
| `platform/` | Core domain services, business logic | `base/` |
| `workbench/` | Features, workspace orchestration | `base/`, `platform/` |
| `renderer/` | UI layer (Vue components, composables) | `base/`, `platform/`, `workbench/` |
### Import Direction Violations to Check
```bash
# platform must NOT import from workbench or renderer
grep -r "from '@/renderer'" src/platform/ --include="*.ts" --include="*.vue"
grep -r "from '@/workbench'" src/platform/ --include="*.ts" --include="*.vue"
# base must NOT import from platform, workbench, or renderer
grep -r "from '@/platform'" src/base/ --include="*.ts" --include="*.vue"
grep -r "from '@/workbench'" src/base/ --include="*.ts" --include="*.vue"
grep -r "from '@/renderer'" src/base/ --include="*.ts" --include="*.vue"
# workbench must NOT import from renderer
grep -r "from '@/renderer'" src/workbench/ --include="*.ts" --include="*.vue"
```
### Legacy Flat Folders
Flag NEW files added to these legacy flat folders (they should go in a domain folder under the appropriate layer instead):
- `src/components/` → should be in `src/renderer/` or `src/workbench/extensions/{feature}/components/`
- `src/stores/` → should be in `src/platform/{domain}/` or `src/workbench/extensions/{feature}/stores/`
- `src/services/` → should be in `src/platform/{domain}/`
- `src/composables/` → should be in `src/renderer/` or `src/platform/{domain}/ui/`
Do NOT flag modifications to existing files in legacy folders — only flag NEW files.
## How to Review
1. Look at the diff to see where new files are created or where code is added.
2. Check if the repo has an established domain folder structure (look for domain-organized directories like `src/platform/`, `src/workbench/`, `src/renderer/`, `src/base/`, or similar).
3. If domain folders exist but new code was placed in a flat technical folder, flag it.
4. Run import direction checks:
- Use `Grep` or `Read` to check if new imports violate layer boundaries.
- Flag any imports from a higher layer to a lower one using the rules above.
5. Check for new files in legacy flat folders and flag them per the Legacy Flat Folders section.
## Generic Checks (when no domain structure is detected)
- God files (>500 lines mixing concerns)
- Circular imports between modules
- Business logic in UI components
## Severity Guidelines
| Issue | Severity |
| ------------------------------------------------------------- | -------- |
| Import direction violation (lower layer imports higher layer) | high |
| New file in legacy flat folder when domain folders exist | medium |
| Business logic in UI component | medium |
| Missing domain boundary (cross-cutting import into internals) | low |
| Naming uses technical role instead of domain concept | low |

View File

@@ -1,66 +0,0 @@
---
name: dep-secrets-scan
description: Runs dependency vulnerability audit and secrets detection
severity-default: critical
tools: [Bash, Read]
---
Run dependency audit and secrets scan to detect known CVEs in dependencies and leaked secrets in code.
## Steps
1. Check which tools are available:
```bash
pnpm --version
gitleaks version
```
- If **neither** is installed, skip this check and report: "Skipped: neither pnpm nor gitleaks installed. Install pnpm: `npm i -g pnpm`. Install gitleaks: `brew install gitleaks` or see https://github.com/gitleaks/gitleaks#installing"
- If only one is available, run that one and note the other was skipped.
2. **Dependency audit** (if pnpm is available):
```bash
pnpm audit --json 2>/dev/null || true
```
Parse the JSON output. Map advisory severity:
- `critical` advisory → `critical`
- `high` advisory → `major`
- `moderate` advisory → `minor`
- `low` advisory → `nitpick`
Report each finding with: package name, version, advisory title, CVE, and suggested patched version.
3. **Secrets detection** (if gitleaks is available):
```bash
gitleaks detect --no-banner --report-format json --source . 2>/dev/null || true
```
Parse the JSON output. All secret findings are `critical` severity.
Report each finding with: file and line, rule description, and a redacted match. Always suggest removing the secret and rotating credentials.
## What This Catches
### Dependency Audit
- Known CVEs in direct and transitive dependencies
- Vulnerable packages from the npm advisory database
### Secrets Detection
- API keys and tokens in code
- AWS credentials, GCP service account keys
- Database connection strings with passwords
- Private keys and certificates
- Generic high-entropy secrets
## Error Handling
- If pnpm audit fails, log the error and continue with gitleaks.
- If gitleaks fails, log the error and continue with audit results.
- If JSON parsing fails for either tool, include raw output with a warning.
- If both tools produce no findings, report "No issues found."

View File

@@ -1,40 +0,0 @@
---
name: doc-freshness
description: Reviews whether code changes are reflected in documentation
severity-default: medium
tools: [Read, Grep]
---
You are a documentation freshness reviewer. Your job is to check whether code changes are properly reflected in documentation, and whether new features need documentation.
Check for:
1. **Stale README sections** - code changes that invalidate setup instructions, API examples, or architecture descriptions in README.md
2. **Outdated code comments** - comments referencing removed functions, old parameter names, previous behavior, or TODO items that are now done
3. **Missing JSDoc on public APIs** - exported functions, classes, or interfaces without JSDoc descriptions, especially those used by consumers of the library
4. **Changed behavior without changelog** - user-facing behavior changes that should be noted in a changelog or release notes
5. **Dead documentation links** - links in markdown files pointing to moved or deleted files
6. **Missing migration guidance** - breaking changes without upgrade instructions
Rules:
- Focus on documentation that needs to CHANGE due to the diff — don't audit all existing docs
- Do NOT flag missing comments on internal/private functions
- Do NOT flag missing changelog entries for purely internal refactors
- "Major" for stale docs that will mislead users, "minor" for missing JSDoc on public APIs, "nitpick" for minor doc improvements
## ComfyUI_frontend Documentation
This repository's public APIs are used by custom node and extension authors. Documentation lives at [docs.comfy.org](https://docs.comfy.org) (repo: Comfy-Org/docs).
For any NEW API, event, hook, or configuration that extensions or custom nodes can use:
- Flag with a suggestion to open a PR to Comfy-Org/docs to document the new API
- Example: "This new extension API should be documented at docs.comfy.org — consider opening a PR to Comfy-Org/docs"
For changes to existing extension-facing APIs:
- Check if the existing docs at docs.comfy.org may need updating
- Flag stale references in CONTRIBUTING.md or developer guides
Anything relevant to custom extension authors should trigger a documentation suggestion.

View File

@@ -1,25 +0,0 @@
---
name: dx-readability
description: Reviews code for developer experience issues including naming clarity, cognitive complexity, dead code, and confusing patterns
severity-default: low
tools: [Read, Grep]
---
You are a developer experience reviewer. Focus on code that will confuse the next developer who reads it.
Check for:
1. **Unclear naming** - variables/functions that don't communicate intent, abbreviations, misleading names
2. **Cognitive complexity** - deeply nested conditions, long functions doing multiple things, complex boolean expressions
3. **Dead code** - unreachable branches, unused variables, commented-out code, vestigial parameters
4. **Confusing patterns** - clever tricks over simple code, implicit behavior, action-at-a-distance, surprising side effects
5. **Missing context** - complex business logic without explaining why, non-obvious algorithms without comments
6. **Inconsistent abstractions** - mixing raw and wrapped APIs, different error handling styles in same module
7. **Implicit knowledge** - code that only works because of undocumented assumptions or conventions
Rules:
- Only flag things that would genuinely confuse a competent developer
- Do NOT flag established project conventions even if you'd prefer different ones
- "Minor" for things that slow comprehension, "nitpick" for pure style preferences
- Major is reserved for genuinely misleading code (names that lie, silent behavior changes)

View File

@@ -1,58 +0,0 @@
---
name: ecosystem-compat
description: Checks whether changes break exported symbols that downstream consumers may depend on
severity-default: high
tools: [Grep, Read, glob, mcp__comfy_codesearch__search_code]
---
Check whether this PR introduces breaking changes to exported symbols that downstream consumers may depend on.
## What to Check
- Renamed or removed exported functions/classes/types
- Changed function signatures (parameters added/removed/reordered)
- Changed return types
- Removed or renamed CSS classes used for selectors
- Changed event names or event payload shapes
- Changed global registrations or extension hooks
- Modified integration points with external systems
## Method
1. Read the diff and identify any changes to exported symbols.
2. For each potentially breaking change, try to determine if downstream consumers exist:
- If `mcp__comfy_codesearch__search_code` is available, search for usages of the changed symbol across downstream repositories.
- Otherwise, use `Grep` to search for usages within the current repository and note that external usage could not be verified.
3. If consumers are found using the changed API, report it as a finding.
## Severity Guidelines
| Ecosystem Usage | Severity | Guidance |
| --------------- | -------- | ------------------------------------------------------------ |
| 5+ consumers | critical | Must address before merge |
| 2-4 consumers | high | Should address or document |
| 1 consumer | medium | Note in PR, author decides |
| 0 consumers | low | Note potential risk only |
| Unknown usage | medium | Require explicit note that external usage was not verifiable |
## Suggestion Template
When a breaking change is found, suggest:
- Keeping the old export alongside the new one
- Adding a deprecation wrapper
- Explicitly noting this as a breaking change in the PR description so consumers can update
## ComfyUI Code Search MCP
This check works best with the ComfyUI code search MCP tool, which searches across all custom node repositories for usage of changed symbols.
If the `mcp__comfy_codesearch__search_code` tool is not available, install it:
```
amp mcp add comfy-codesearch https://comfy-codesearch.vercel.app/api/mcp
# OR for Claude Code:
claude mcp add -t http comfy-codesearch https://comfy-codesearch.vercel.app/api/mcp
```
Without this MCP, the check will fall back to searching within the current repository only, and cannot verify external ecosystem usage.

View File

@@ -1,38 +0,0 @@
---
name: error-handling
description: Reviews error handling patterns for empty catches, swallowed errors, missing async error handling, and generic error UX
severity-default: high
tools: [Read, Grep]
---
You are an error handling auditor reviewing a code diff. Focus exclusively on how errors are handled, propagated, and surfaced.
Check for:
1. **Empty catch blocks** - errors caught and silently swallowed with no logging or re-throw
2. **Generic catches** - catching all errors without distinguishing types, losing context
3. **Missing async error handling** - unhandled promise rejections, async functions without try/catch or .catch()
4. **Swallowed errors** - errors caught and replaced with a return value that hides the failure
5. **Missing error boundaries** - Vue/React component trees without error boundaries around risky subtrees
6. **No retry or fallback** - network calls, file I/O, or external service calls with no retry logic or graceful degradation
7. **Generic error UX** - user-facing code showing "Something went wrong" without actionable guidance or error codes
8. **Missing cleanup on error** - resources (connections, file handles, timers) not cleaned up in error paths
9. **Error propagation breaks** - catching errors mid-chain and not re-throwing, breaking caller's ability to handle
Rules:
- Focus on NEW or CHANGED error handling in the diff
- Do NOT flag existing error handling patterns in untouched code
- Do NOT suggest adding error handling to code that legitimately cannot fail (pure functions, type-safe internal calls)
- "Critical" for swallowed errors in data-mutation paths, "major" for missing error handling on external calls, "minor" for missing logging
## Repo-Specific Error Handling
- User-facing error messages must be actionable and friendly (per AGENTS.md)
- Use the shared `useErrorHandling` composable (`src/composables/useErrorHandling.ts`) for centralized error handling:
- `wrapWithErrorHandling` / `wrapWithErrorHandlingAsync` automatically catch errors and surface them as toast notifications via `useToastStore`
- `toastErrorHandler` can be used directly for custom error handling flows
- Supports `ErrorRecoveryStrategy` for retry/fallback patterns (e.g., reauthentication, network reconnect)
- API errors from `api.get()`/`api.post()` should be caught and surfaced to the user via `useToastStore` (`src/platform/updates/common/toastStore.ts`)
- Electron/desktop code paths: IPC errors should be caught and not crash the renderer process
- Workflow execution errors should be displayed in the UI status bar, not silently swallowed

View File

@@ -1,60 +0,0 @@
/**
* Strict ESLint config for the sonarjs-lint review check.
*
* Uses eslint-plugin-sonarjs to get SonarQube-grade analysis without a server.
* This config is NOT used for regular development linting — only for the
* code review checks' static analysis pass.
*
* Install: pnpm add -D eslint eslint-plugin-sonarjs
* Run: pnpm dlx eslint --no-config-lookup --config .agents/checks/eslint.strict.config.js --ext .ts,.js,.vue {files}
*/
import sonarjs from 'eslint-plugin-sonarjs'
export default [
sonarjs.configs.recommended,
{
plugins: {
sonarjs
},
rules: {
// Bug detection
'sonarjs/no-all-duplicated-branches': 'error',
'sonarjs/no-element-overwrite': 'error',
'sonarjs/no-identical-conditions': 'error',
'sonarjs/no-identical-expressions': 'error',
'sonarjs/no-one-iteration-loop': 'error',
'sonarjs/no-use-of-empty-return-value': 'error',
'sonarjs/no-collection-size-mischeck': 'error',
'sonarjs/no-duplicated-branches': 'error',
'sonarjs/no-identical-functions': 'error',
'sonarjs/no-redundant-jump': 'error',
'sonarjs/no-unused-collection': 'error',
'sonarjs/no-gratuitous-expressions': 'error',
// Code smell detection
'sonarjs/cognitive-complexity': ['error', 15],
'sonarjs/no-duplicate-string': ['error', { threshold: 3 }],
'sonarjs/no-redundant-boolean': 'error',
'sonarjs/no-small-switch': 'error',
'sonarjs/prefer-immediate-return': 'error',
'sonarjs/prefer-single-boolean-return': 'error',
'sonarjs/no-inverted-boolean-check': 'error',
'sonarjs/no-nested-template-literals': 'error'
},
languageOptions: {
ecmaVersion: 2024,
sourceType: 'module'
}
},
{
ignores: [
'**/node_modules/**',
'**/dist/**',
'**/build/**',
'**/*.config.*',
'**/*.test.*',
'**/*.spec.*'
]
}
]

View File

@@ -1,72 +0,0 @@
---
name: import-graph
description: Validates import rules, detects circular dependencies, and enforces layer boundaries using dependency-cruiser
severity-default: high
tools: [Bash, Read]
---
Run dependency-cruiser import graph analysis on changed files to detect circular dependencies, orphan modules, and import rule violations.
> **Note:** The circular dependency scan in step 4 targets `src/` specifically, since this is a frontend app with source code under `src/`.
## Steps
1. Check if dependency-cruiser is available:
```bash
pnpm dlx dependency-cruiser --version
```
If not available, skip this check and report: "Skipped: dependency-cruiser not available. Install with: `pnpm add -D dependency-cruiser`"
> **Install:** `pnpm add -D dependency-cruiser`
2. Identify changed directories from the diff.
3. Determine config to use:
- If `.dependency-cruiser.js` or `.dependency-cruiser.cjs` exists in the repo root, use it (dependency-cruiser auto-detects it). This config may enforce layer architecture rules (e.g., base → platform → workbench → renderer import direction):
```bash
pnpm dlx dependency-cruiser --output-type json <changed_directories> 2>/dev/null
```
- If no config exists, run with built-in defaults:
```bash
pnpm dlx dependency-cruiser --no-config --output-type json <changed_directories> 2>/dev/null
```
4. Also check for circular dependencies specifically across `src/`:
```bash
pnpm dlx dependency-cruiser --no-config --output-type json --do-not-follow "node_modules" --include-only "^src" src 2>/dev/null
```
Look for modules where `.circular == true` in the output.
5. Parse the JSON output. Each violation has:
- `rule.name`: the violated rule
- `rule.severity`: error, warn, info
- `from`: importing module
- `to`: imported module
6. Map violation severity:
- `error` → `major`
- `warn` → `minor`
- `info` → `nitpick`
- Circular dependencies → `major` (category: architecture)
- Orphan modules → `nitpick` (category: dx)
7. Report each violation with: the rule name, source and target modules, file path, and a suggestion (usually move the import or extract an interface).
## What It Catches
| Rule | What It Detects |
| ------------------------ | ---------------------------------------------------- |
| `no-circular` | Circular dependency chains (A → B → C → A) |
| `no-orphans` | Modules with no incoming or outgoing dependencies |
| `not-to-dev-dep` | Production code importing devDependencies |
| `no-duplicate-dep-types` | Same dependency in multiple sections of package.json |
| Custom layer rules | Import direction violations (e.g., base → platform) |
## Error Handling
- If pnpm dlx is not available, skip and report the error.
- If the config file fails to parse, fall back to `--no-config`.
- If there are more than 50 violations, report the first 20 and note the total count.
- If no violations are found, report "No issues found."

View File

@@ -1,27 +0,0 @@
---
name: memory-leak
description: Scans for memory leak patterns including event listeners without cleanup, timers not cleared, and unbounded caches
severity-default: high
tools: [Read, Grep]
---
You are a memory leak specialist reviewing a code diff. Focus exclusively on patterns that cause memory to grow unboundedly over time.
Check for:
1. **Event listeners without cleanup** - addEventListener without corresponding removeEventListener, especially in Vue onMounted without onBeforeUnmount cleanup
2. **Timers not cleared** - setInterval/setTimeout started in component lifecycle without clearInterval/clearTimeout on unmount
3. **Observer patterns without disconnect** - MutationObserver, IntersectionObserver, ResizeObserver created without .disconnect() on cleanup
4. **WebSocket/Worker connections** - opened connections never closed on component unmount or route change
5. **Unbounded caches** - Maps, Sets, or arrays that grow with usage but never evict entries, especially keyed by user input or dynamic IDs
6. **Stale closures holding references** - closures in event handlers or callbacks that capture large objects or DOM nodes and prevent garbage collection
7. **RequestAnimationFrame without cancel** - rAF loops started without cancelAnimationFrame on cleanup
8. **Vue-specific leaks** - watch/watchEffect without stop(), computed that captures reactive dependencies it shouldn't, provide/inject holding stale references
9. **Global state accumulation** - pushing to global arrays/maps without ever removing entries, console.log keeping object references in dev
Rules:
- Focus on NEW leak patterns introduced in the diff
- Do NOT flag existing cleanup patterns that are correct
- Every finding must explain the specific lifecycle scenario where the leak occurs (e.g., "when user navigates away from this view, the interval keeps running")
- "Critical" for leaks in hot paths or long-lived pages, "major" for component-level leaks, "minor" for dev-only or cold-path leaks

View File

@@ -1,60 +0,0 @@
---
name: pattern-compliance
description: Checks code against repository conventions from AGENTS.md and established patterns
severity-default: medium
tools: [Read, Grep]
---
Check code against repository conventions and framework patterns.
Steps:
1. Read AGENTS.md (and any directory-specific guidance files) for project-specific conventions
2. Read each changed file
3. Check against the conventions found in AGENTS.md and these standard patterns:
### TypeScript
- No `any` types or `as any` assertions
- No `@ts-ignore` without explanatory comment
- Separate type imports (`import type { ... }`)
- Use `import type { ... }` for type-only imports
- Explicit return types on exported functions
- Use `es-toolkit` for utility functions, NOT lodash. Flag any new `import ... from 'lodash'` or `import ... from 'lodash/*'`
- Never use `z.any()` in Zod schemas — use `z.unknown()` and narrow
### Vue (if applicable)
- Composition API with `<script setup lang="ts">`
- Reactive props destructuring (not `withDefaults` pattern)
- New components must use `<script setup lang="ts">` with reactive props destructuring (Vue 3.5 style): `const { color = 'blue' } = defineProps<Props>()`
- Separate type imports from value imports
- All user-facing strings must use `vue-i18n` (`$t()` in templates, `t()` in script). Flag hardcoded English strings in templates that should be translated. The locale file is `src/locales/en/main.json`
### Tailwind (if applicable)
- No `dark:` variants (use semantic theme tokens)
- Use `cn()` utility for conditional classes
- No `!important` in utility classes
- Tailwind 4: CSS variable references use parentheses syntax: `h-(--my-var)` NOT `h-[--my-var]`
- Use design tokens: `bg-secondary-background`, `text-muted-foreground`, `border-border-default`
- No `<style>` blocks in Vue SFCs — use inline Tailwind only
### Testing
- Behavioral tests, not change detectors
- No mock-heavy tests that don't test real behavior
- Test names describe behavior, not implementation
### General
- No commented-out code
- No `console.log` in production code (unless intentional logging)
- No hardcoded URLs, credentials, or environment-specific values
- Package manager is `pnpm`. Never use `npm`, `npx`, or `yarn`. Use `pnpm dlx` for one-off package execution
- Sanitize HTML with `DOMPurify.sanitize()`, never raw `innerHTML` or `v-html` without it
Rules:
- Only flag ACTUAL violations, not hypothetical ones
- AGENTS.md conventions take priority over default patterns if they conflict

View File

@@ -1,35 +0,0 @@
---
name: performance-profiler
description: Reviews code for performance issues including algorithmic complexity, unnecessary work, and bundle size impact
severity-default: medium
tools: [Read, Grep]
---
You are a performance engineer reviewing a code diff. Focus exclusively on performance issues.
Check for:
1. **Algorithmic complexity** - O(n²) or worse in loops, nested iterations over large collections
2. **Unnecessary re-computation** - repeated work in render cycles, missing memoization for expensive ops
3. **Memory leaks** - event listeners not cleaned up, growing caches without eviction, closures holding references
4. **N+1 queries** - database/API calls inside loops
5. **Bundle size** - large imports that could be tree-shaken, dynamic imports for heavy modules
6. **Rendering performance** - unnecessary re-renders, layout thrashing, expensive computed properties
7. **Data structures** - using arrays for lookups instead of maps/sets, unnecessary copying of large objects
8. **Async patterns** - sequential awaits that could be parallel, missing abort controllers
Rules:
- ONLY report actual performance issues, not premature optimization suggestions
- Distinguish between hot paths (major) and cold paths (minor)
- Include Big-O analysis when relevant
- Do NOT suggest micro-optimizations that a JIT compiler handles
- Quantify the impact when possible: "This is O(n²) where n = number of users"
## Repo-Specific Performance Concerns
- **LiteGraph canvas rendering** is the primary hot path. Operations inside `LGraphNode.onDrawForeground`, `onDrawBackground`, `processMouseMove` run every frame at 60fps. Any O(n) or worse operation here on the node/link collections is critical.
- **Node definition lookups** happen frequently — these should use Maps, not array.find()
- **Workflow serialization/deserialization** can involve large JSON objects (1000+ nodes). Watch for deep copies or unnecessary re-parsing.
- **Vue reactivity in canvas code** — reactive getters triggered during canvas render cause performance issues. Canvas-facing code should read raw values, not reactive refs.
- **Bundle size** — check for large imports that could be dynamically imported. The build uses Vite with `build:analyze` for bundle visualization.

View File

@@ -1,74 +0,0 @@
---
name: playwright-e2e
description: Reviews Playwright E2E test code for ComfyUI-specific patterns, flakiness risks, and fixture misuse
severity-default: medium
tools: [Read, Grep]
---
You are reviewing Playwright E2E test code in `browser_tests/`. Focus on issues a **reviewer** would catch that an author might miss — flakiness risks, fixture misuse, test isolation problems, and convention violations.
Reference docs (read if you need full context):
- `browser_tests/README.md` — setup, patterns, screenshot workflow
- `browser_tests/AGENTS.md` — directory structure, fixture overview
- `docs/guidance/playwright.md` — type assertion rules, test tags, forbidden patterns
- `.claude/skills/writing-playwright-tests/SKILL.md` — anti-patterns, retry patterns, Vue Nodes vs LiteGraph decision guide
## Checks
### Flakiness Risks (Major)
1. **`waitForTimeout` usage** — Always wrong. Must use retrying assertions (`toBeVisible`, `toHaveText`), `expect.poll()`, or `expect().toPass()`. See retry patterns in `.claude/skills/writing-playwright-tests/SKILL.md`.
2. **Missing `nextFrame()` after canvas ops** — Any `drag`, `click` on canvas, `resizeNode`, `pan`, `zoom`, or programmatic graph mutation via `page.evaluate` that changes visual state needs `await comfyPage.nextFrame()` before assertions. `loadWorkflow()` does NOT need it. Prefer encapsulating `nextFrame()` calls inside Page Object methods so tests don't manage frame timing directly.
3. **Keyboard actions without prior focus**`page.keyboard.press()` without a preceding `comfyPage.canvas.click()` or element `.focus()` will silently send keys to nothing.
4. **Coordinate-based interactions where node refs exist** — Raw `{ x, y }` clicks on canvas are fragile. If the test targets a node, use `comfyPage.nodeOps.getNodeRefById()` / `getNodeRefsByTitle()` / `getNodeRefsByType()` instead.
5. **Shared mutable state between tests** — Variables declared outside `test()` blocks, `let` state mutated across tests, or tests depending on execution order. Each test must be independently runnable.
6. **Missing cleanup of server-persisted state** — Settings changed via `comfyPage.settings.setSetting()` persist across tests. Must be reset in `afterEach` or at test start. Same for uploaded files or saved workflows. Prefer moving cleanup into [fixture options](https://playwright.dev/docs/test-fixtures#fixtures-options) so individual tests don't manage reset logic.
7. **Double-click without `{ delay }` option**`dblclick()` without `{ delay: 5 }` or similar can be too fast for the canvas event handler.
### Fixture & API Misuse (Medium)
8. **Reimplementing existing fixture helpers** — Before flagging, grep `browser_tests/fixtures/` for the functionality. Common missed helpers:
- `comfyPage.command.executeCommand()` for menu/command actions
- `comfyPage.workflow.loadWorkflow()` for loading test workflows
- `comfyPage.canvasOps.resetView()` for view reset
- `comfyPage.settings.setSetting()` for settings
- Component page objects in `browser_tests/fixtures/components/`
9. **Building workflows programmatically when a JSON asset would work** — Complex `page.evaluate` chains to construct a graph should use a premade JSON workflow in `browser_tests/assets/` loaded via `comfyPage.workflow.loadWorkflow()`.
10. **Selectors not using `TestIds`** — Hard-coded `data-testid` strings should reference `browser_tests/fixtures/selectors.ts` when a matching entry exists. Check `selectors.ts` before flagging.
### Convention Violations (Minor)
11. **Missing test tags** — Every `test.describe` should have `tag` with at least one of: `@smoke`, `@slow`, `@screenshot`, `@canvas`, `@node`, `@widget`, `@mobile`, `@2x`. See `.claude/skills/writing-playwright-tests/SKILL.md` for when to use each.
12. **`as any` type assertions** — Forbidden in E2E tests. Use specific type assertions or test-local type helpers. See `docs/guidance/playwright.md` for acceptable patterns.
13. **Screenshot tests without masking dynamic content** — Timestamps, version numbers, or other non-deterministic content in screenshots will cause flakes. Use `mask` option.
14. **`test.describe` without `afterEach` cleanup when canvas state changes** — Tests that manipulate canvas view (drag, zoom, pan) should include `afterEach` with `comfyPage.canvasOps.resetView()`. Prefer moving canvas reset into the fixture so individual tests don't manage cleanup.
15. **Debug helpers left in committed code**`debugAddMarker`, `debugAttachScreenshot`, `debugShowCanvasOverlay`, `debugGetCanvasDataURL` are for local debugging only.
### Test Design (Nitpick)
16. **Screenshot-only assertions where functional assertions are possible** — Prefer `expect(await node.isPinned()).toBe(true)` over screenshot comparison when testing non-visual behavior.
17. **Overly large test workflows** — Test should load the minimal workflow needed. If a test only needs one node, don't load the full default graph.
18. **Vue Nodes / LiteGraph mismatch** — If testing Vue-rendered node UI (DOM widgets, CSS states), should use `comfyPage.vueNodes.*`. If testing canvas interactions/connections, should use `comfyPage.nodeOps.*`. Mixing both in one test is a smell.
## Rules
- Only review `.spec.ts` files and supporting code in `browser_tests/`
- Do NOT flag patterns in fixture/helper code (`browser_tests/fixtures/`) — those are shared infrastructure with different rules
- "Major" for flakiness risks (items 1-7), "medium" for fixture misuse (8-10), "minor" for convention violations (11-15), "nitpick" for test design (16-18)
- When flagging missing fixture usage (item 8), confirm the helper exists by checking the fixture code — don't assume
- Existing tests that predate conventions are acceptable to modify but not required to fix

View File

@@ -1,44 +0,0 @@
---
name: regression-risk
description: Detects potential regressions by analyzing git blame history of modified lines
severity-default: high
tools: [Bash, Read, Grep]
---
Perform regression risk analysis on the current changes using git blame.
## Method
1. Determine the base branch by examining git context (e.g., `git merge-base origin/main HEAD`, or check the PR's target branch). Never use `HEAD~1` as the base — it compares against the PR's own prior commit and causes false positives.
2. Get the PR's own commits: `git log --format=%H <base>..HEAD`
3. For each changed file, run: `git diff <base>...HEAD -- <file>`
4. Extract the modified line ranges from the diff (lines removed or changed in the base version).
5. For each modified line range, check git blame in the base version:
`git blame <base> -L <start>,<end> -- <file>`
6. Look for blame commits whose messages match bugfix patterns:
- Contains: fix, bug, patch, hotfix, revert, regression, CVE
- Ignore: "fix lint", "fix typo", "fix format", "fix style"
7. **Filter out false positives.** If the blamed commit SHA is in the PR's own commits, skip it.
8. For each verified bugfix line being modified, report as a finding.
## What to Report
For each finding, include:
- The file and line number
- The original bugfix commit (short SHA and subject)
- The date of the original fix
- A suggestion to verify the original bug scenario still works and to add a regression test if one doesn't exist
## Shallow Clone Limitations
When working with shallow clones, `git blame` may not have full history. If blame fails with "no such path in revision" or shows truncated history, report only findings where blame succeeds and note the limitation.
## Edge Cases
| Situation | Action |
| ------------------------ | -------------------------------- |
| Shallow clone (no blame) | Report what succeeds, note limit |
| Blame shows PR's own SHA | Skip finding (false positive) |
| File renamed | Try blame with `--follow` |
| Binary file | Skip file |

View File

@@ -1,34 +0,0 @@
---
name: security-auditor
description: Reviews code for security vulnerabilities aligned with OWASP Top 10
severity-default: critical
tools: [Read, Grep]
---
You are a security auditor reviewing a code diff. Focus exclusively on security vulnerabilities.
Check for:
1. **Injection** - SQL injection, command injection, template injection, XSS (stored/reflected/DOM)
2. **Authentication/Authorization** - auth bypass, privilege escalation, missing access checks
3. **Data exposure** - secrets in code, PII in logs, sensitive data in error messages, overly broad API responses
4. **Cryptography** - weak algorithms, hardcoded keys, predictable tokens, missing encryption
5. **Input validation** - missing sanitization, path traversal, SSRF, open redirects
6. **Dependency risks** - known vulnerable patterns, unsafe deserialization
7. **Configuration** - CORS misconfiguration, missing security headers, debug mode in production
8. **Race conditions with security impact** - TOCTOU, double-spend, auth state races
Rules:
- ONLY report security issues, not general bugs or style
- All findings must be severity "critical" or "major"
- Explain the attack vector: who can exploit this and how
- Do NOT report theoretical issues without a plausible attack scenario
- Reference OWASP category when applicable
## Repo-Specific Patterns
- HTML sanitization must use `DOMPurify.sanitize()` — flag any `v-html` or `innerHTML` without DOMPurify
- API calls should use `api.get(api.apiURL(...))` helpers, not raw `fetch('/api/...')` — direct URL construction can bypass auth
- Firebase/Sentry credentials are configured via environment — flag any hardcoded Firebase config objects
- Electron IPC: check for unsafe `ipcRenderer.send` patterns in desktop code paths

View File

@@ -1,54 +0,0 @@
---
name: semgrep-sast
description: Runs Semgrep SAST with auto-configured rules for JS/TS/Vue
severity-default: high
tools: [Bash, Read]
---
Run Semgrep static analysis on changed files to detect security vulnerabilities, dangerous patterns, and framework-specific issues.
## Steps
1. Check if semgrep is installed:
```bash
semgrep --version
```
If not installed, skip this check and report: "Skipped: semgrep not installed. Install with: `pip3 install semgrep`"
2. Identify changed files (`.ts`, `.js`, `.vue`) from the diff.
If none are found, skip and report: "Skipped: no changed JS/TS/Vue files."
3. Run semgrep against changed files:
```bash
semgrep --config=auto --json --quiet <changed_files>
```
4. Parse the JSON output (`.results[]` array). For each finding, map severity:
- Semgrep `ERROR` → `critical`
- Semgrep `WARNING` → `major`
- Semgrep `INFO` → `minor`
5. Report each finding with:
- The semgrep rule ID (`check_id`)
- File path and line number (`path`, `start.line`)
- The message from `extra.message`
- A fix suggestion from `extra.fix` if available, otherwise general remediation advice
## What Semgrep Catches
With `--config=auto`, Semgrep loads community-maintained rules for:
- **Security vulnerabilities:** injection, XSS, SSRF, path traversal, open redirect
- **Dangerous patterns:** eval(), innerHTML, dangerouslySetInnerHTML, exec()
- **Crypto issues:** weak hashing, hardcoded secrets, insecure random
- **Best practices:** missing security headers, unsafe deserialization
- **Framework-specific:** Express, React, Vue security patterns
## Error Handling
- If semgrep config download fails, skip and report the error.
- If semgrep fails to parse a specific file, skip that file and continue with others.
- If semgrep produces no findings, report "No issues found."

View File

@@ -1,59 +0,0 @@
---
name: sonarjs-lint
description: Runs SonarQube-grade static analysis using eslint-plugin-sonarjs
severity-default: high
tools: [Bash, Read]
---
Run eslint-plugin-sonarjs analysis on changed files to detect bugs, code smells, and security patterns without needing a SonarQube server.
## Steps
1. Check if eslint is available:
```bash
pnpm dlx eslint --version
```
If pnpm dlx or eslint is unavailable, skip this check and report: "Skipped: eslint not available. Ensure Node.js and pnpm dlx are installed."
2. Identify changed files (`.ts`, `.js`, `.vue`) from the diff.
3. Determine eslint config to use. This check uses a **strict sonarjs-specific config** (not the project's own eslint config, which is less strict):
- Look for the colocated strict config at `.agents/checks/eslint.strict.config.js`
- If found, run with `--config .agents/checks/eslint.strict.config.js`
- **Fallback:** if the strict config cannot be found or fails to load, skip this check and report: "Skipped: .agents/checks/eslint.strict.config.js missing; SonarJS rules require explicit config."
4. Run eslint against changed files:
```bash
# Use the strict config
pnpm dlx --yes --package eslint-plugin-sonarjs eslint --no-config-lookup --config .agents/checks/eslint.strict.config.js --format json <changed_files> 2>/dev/null || true
```
5. Parse the JSON array of file results. For each eslint message, map severity:
- `severity 2` (error) → `major`
- `severity 1` (warning) → `minor`
6. Categorize findings by rule ID:
- Rule IDs starting with `sonarjs/no-` → category: `logic`
- Rule IDs containing `cognitive-complexity` → category: `dx`
- Other sonarjs rules → category: `style`
7. Report each finding with:
- The rule ID
- File path and line number
- The message from eslint
- A fix suggestion based on the rule
## What This Catches
- **Bug detection:** duplicated branches, element overwrite, identical conditions/expressions, one-iteration loops, empty return values
- **Code smells:** cognitive complexity (threshold: 15), duplicate strings, redundant booleans, small switches
- **Security patterns:** via sonarjs recommended ruleset
## Error Handling
- If eslint fails to parse a Vue file, skip that file and continue with others.
- If the plugin fails to install, skip and report the error.
- If eslint produces no output or errors, report "No issues found."

View File

@@ -1,37 +0,0 @@
---
name: test-quality
description: Reviews test code for quality issues and coverage gaps
severity-default: medium
tools: [Read, Grep]
---
You are a test quality reviewer. Evaluate the tests included with (or missing from) this code change.
Check for:
1. **Missing tests** - new behavior without test coverage, modified logic without updated tests
2. **Change-detector tests** - tests that assert implementation details instead of behavior (testing that a function was called, not what it produces)
3. **Mock-heavy tests** - tests with so many mocks they don't test real behavior
4. **Snapshot abuse** - large snapshots that no one reviews, snapshots of implementation details
5. **Fragile assertions** - tests that break on unrelated changes, order-dependent tests
6. **Missing edge cases** - happy path only, no empty/null/error scenarios tested
7. **Test readability** - unclear test names, complex setup that obscures intent, shared mutable state between tests
8. **Test isolation** - tests depending on execution order, shared state, external services without mocking
Rules:
- Focus on test quality and coverage gaps, not production code bugs
- "Major" for missing tests on critical logic, "minor" for missing edge case tests
- A change that adds no tests is only an issue if the change adds behavior
- Refactors without behavior changes don't need new tests
- Prefer behavioral tests: test inputs and outputs, not internal implementation
- This repo uses **colocated tests**: `.test.ts` files live next to their source files (e.g., `MyComponent.test.ts` beside `MyComponent.vue`). When checking for missing tests, look for a colocated `.test.ts` file, not a separate `tests/` directory
## Repo-Specific Testing Conventions
- Tests use **Vitest** (not Jest) — run with `pnpm test:unit`
- Test files are **colocated**: `MyComponent.test.ts` next to `MyComponent.vue`
- Use `@vue/test-utils` for component testing, `@pinia/testing` (`createTestingPinia`) for store tests
- Browser/E2E tests use **Playwright** in `browser_tests/` — run with `pnpm test:browser:local`
- Mock composables using the singleton factory pattern inside `vi.mock()` — see `docs/testing/unit-testing.md` for the pattern
- Never use `any` in test code either — proper typing applies to tests too

View File

@@ -1,47 +0,0 @@
---
name: vue-patterns
description: Reviews Vue 3.5+ code for framework-specific anti-patterns
severity-default: medium
tools: [Read, Grep]
---
You are a Vue 3.5 framework specialist reviewing a code diff. Focus on Vue-specific patterns, anti-patterns, and missed framework features.
Check for:
1. **Options API in new files** - new .vue files using Options API instead of Composition API with `<script setup>`. Modifications to existing Options API files are fine.
2. **Reactivity anti-patterns** - destructuring reactive objects losing reactivity, using `ref()` for objects that should be `reactive()`, accessing `.value` inside templates, incorrectly using `toRefs`/`toRef`
3. **Watch/watchEffect cleanup** - watchers without cleanup functions when they set up side effects (timers, listeners, subscriptions)
4. **Flush timing issues** - DOM access in watch callbacks without `{ flush: 'post' }`, `nextTick` misuse, accessing template refs before mount
5. **defineEmits typing** - using array syntax `defineEmits(['event'])` instead of TypeScript syntax `defineEmits<{...}>()`
6. **defineExpose misuse** - exposing internal state via `defineExpose` when events would be more appropriate (expose is for imperative methods: validate, focus, open)
7. **Prop drilling** - passing props through 3+ component levels where provide/inject would be cleaner
8. **VueUse opportunities** - manual implementations of common composables that VueUse already provides (useLocalStorage, useEventListener, useDebounceFn, useIntersectionObserver, etc.)
9. **Computed vs method** - methods used in templates for derived state that should be computed properties, or computed properties that have side effects
10. **PrimeVue usage in new code** - New components must NOT use PrimeVue. This project is migrating to shadcn-vue (Reka UI primitives). If new code imports from `primevue/*`, flag it and suggest the shadcn-vue equivalent.
Available shadcn-vue replacements in `src/components/ui/`:
- `button/` — Button, variants
- `select/` — Select, SelectTrigger, SelectContent, SelectItem
- `textarea/` — Textarea
- `toggle-group/` — ToggleGroup, ToggleGroupItem
- `slider/` — Slider
- `skeleton/` — Skeleton
- `stepper/` — Stepper
- `tags-input/` — TagsInput
- `search-input/` — SearchInput
- `Popover.vue` — Popover
For Reka UI primitives not yet wrapped, create a new component in `src/components/ui/` following the pattern in existing components (see `src/components/ui/AGENTS.md`): use `useForwardProps`, `cn()`, design tokens.
Modifications to existing PrimeVue-based components are acceptable but should note the migration opportunity.
Rules:
- Only review .vue and composable .ts files — skip stores, services, utils
- Do NOT flag existing Options API files being modified (only flag NEW files)
- Flag new PrimeVue imports — the project is migrating to shadcn-vue/Reka UI
- When suggesting shadcn-vue alternatives, reference `src/components/ui/AGENTS.md` for the component creation pattern
- Use Iconify icons (`<i class="icon-[lucide--check]" />`) not PrimeIcons
- "Major" for reactivity bugs and flush timing, "minor" for API style and VueUse opportunities, "nitpick" for preference-level patterns

View File

@@ -3,15 +3,12 @@
## Task: Add English translations for all new localized strings
### Step 1: Identify new translation keys
Find all translation keys that were added in the current branch's changes. These keys appear as arguments to translation functions: `t()`, `st()`, `$t()`, or similar i18n functions.
### Step 2: Add translations to English locale file
For each new translation key found, add the corresponding English text to the file `src/locales/en/main.json`.
### Key-to-JSON mapping rules:
- Translation keys use dot notation to represent nested JSON structure
- Convert dot notation to nested JSON objects when adding to the locale file
- Example: The key `g.user.name` maps to:
@@ -26,7 +23,6 @@ For each new translation key found, add the corresponding English text to the fi
```
### Important notes:
1. **Only modify the English locale file** (`src/locales/en/main.json`)
2. **Do not modify other locale files** - translations for other languages are automatically generated by the `i18n.yaml` workflow
3. **Exception for manual translations**: Only add translations to non-English locale files if:
@@ -34,7 +30,6 @@ For each new translation key found, add the corresponding English text to the fi
- The automated translation would likely be incorrect due to technical terminology or context-specific meaning
### Example workflow:
1. If you added `t('settings.advanced.enable')` in a Vue component
2. Add to `src/locales/en/main.json`:
```json

View File

@@ -1,94 +0,0 @@
# ADR Compliance Audit
Audit the current changes (or a specified PR) for compliance with Architecture Decision Records.
## Step 1: Gather the Diff
- If a PR number is provided, run: `gh pr diff $PR_NUMBER`
- Otherwise, run: `git diff origin/main...HEAD` (or `git diff --cached` for staged changes)
## Step 2: Priority 1 — ECS and Command-Pattern Compliance
Read these documents for context:
```
docs/adr/0003-crdt-based-layout-system.md
docs/adr/0008-entity-component-system.md
docs/architecture/ecs-target-architecture.md
docs/architecture/ecs-migration-plan.md
docs/architecture/appendix-critical-analysis.md
```
### Check A: Command Pattern (ADR 0003)
Every entity state mutation must be a **serializable, idempotent, deterministic command** — replayable, undoable, transmittable over CRDT.
Flag:
1. **Direct spatial mutation**`node.pos = ...`, `node.size = ...`, `group.pos = ...` outside a store/command
2. **Imperative fire-and-forget APIs** — Functions that mutate entity state as side effects rather than producing serializable command objects. Systems should produce command batches, not execute mutations directly.
3. **Void-returning mutation APIs** — Entity mutations returning `void` instead of `{ status: 'applied' | 'rejected' | 'no-op' }`
4. **Auto-increment IDs** — New entity creation via counters without addressing CRDT collision. Concurrent environments need globally unique identifiers.
5. **Missing transaction semantics** — Multi-entity operations without atomic grouping (e.g., node removal = 10+ deletes with no rollback on failure)
### Check B: ECS Architecture (ADR 0008)
Flag:
1. **God-object growth** — New methods/properties on `LGraphNode`, `LGraphCanvas`, `LGraph`, `Subgraph`
2. **Mixed data/behavior** — Component-like structures with methods or back-references
3. **OOP instance patterns** — New `node.someProperty` or `node.someMethod()` for data that should be a World component
4. **OOP inheritance** — New entity subclasses instead of component composition
5. **Circular entity deps** — New `LGraph``Subgraph`, `LGraphNode``LGraphCanvas` circular imports
6. **Direct `_version++`** — Mutating private version counter instead of through public API
### Check C: Extension Ecosystem Impact
If any of these patterns are changed, flag and require migration guidance:
- `onConnectionsChange`, `onRemoved`, `onAdded`, `onConfigure` callbacks
- `onConnectInput` / `onConnectOutput` validation hooks
- `onWidgetChanged` handlers
- `node.widgets.find(w => w.name === ...)` access patterns
- `node.serialize` overrides
- `graph._version++` direct mutation
Reference: 40+ custom node repos depend on these (rgthree-comfy, ComfyUI-Impact-Pack, cg-use-everywhere, etc.)
## Step 3: Priority 2 — General ADR Compliance
1. Read `docs/adr/README.md` for the full ADR index
2. For each ADR (except skip list), read the Decision section
3. Check the diff for contradictions
4. Only flag ACTUAL violations in changed code
**Skip list**: ADR 0004 (Rejected — Fork PrimeVue)
## Step 4: Generate Report
```
## ADR Compliance Audit Report
### Summary
- Files audited: N
- Priority 1 findings: N (command-pattern: N, ECS: N, ecosystem: N)
- Priority 2 findings: N
### Priority 1: Command Pattern & ECS
(List each with ADR reference, file, line, description)
### Priority 1: Extension Ecosystem Impact
(List each changed callback/API with affected custom node repos)
### Priority 2: General ADR Compliance
(List each with ADR reference, file, line, description)
### Compliant Patterns
(Note changes that positively align with ADR direction)
```
## Severity
- **Must fix**: Contradicts accepted ADR, or introduces imperative mutation API without command-pattern wrapper, or breaks extension callback without migration path
- **Should discuss**: Contradicts proposed ADR direction — either align or propose ADR amendment
- **Note**: Surfaces open architectural question not yet addressed by ADRs

View File

@@ -15,7 +15,6 @@ To post inline comments, you will use the GitHub API via the `gh` command. Here'
- Run: `gh pr view $PR_NUMBER --json commits --jq '.commits[-1].oid'` to get the latest commit SHA
2. For each issue you find, post an inline comment using this exact command structure (as a single line):
```
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/OWNER/REPO/pulls/$PR_NUMBER/comments -f body="YOUR_COMMENT_BODY" -f commit_id="COMMIT_SHA" -f path="FILE_PATH" -F line=LINE_NUMBER -f side="RIGHT"
```
@@ -23,15 +22,13 @@ To post inline comments, you will use the GitHub API via the `gh` command. Here'
3. Format your comment body using actual newlines in the command. Use a heredoc or construct the body with proper line breaks:
```
COMMENT_BODY="**[category] severity Priority**
```
**Issue**: Brief description of the problem
**Context**: Why this matters
**Suggestion**: How to fix it"
```
Then use: `-f body="$COMMENT_BODY"`
```
Then use: `-f body="$COMMENT_BODY"`
## Phase 1: Environment Setup and PR Context
@@ -61,12 +58,10 @@ This is critical for better file inspection:
1. Get PR metadata: `gh pr view $PR_NUMBER --json files,title,body,additions,deletions,baseRefName,headRefName > pr_info.json`
2. Extract branch names from pr_info.json using jq
3. Fetch and checkout the PR branch:
```
git fetch origin "pull/$PR_NUMBER/head:pr-$PR_NUMBER"
git checkout "pr-$PR_NUMBER"
```
```
git fetch origin "pull/$PR_NUMBER/head:pr-$PR_NUMBER"
git checkout "pr-$PR_NUMBER"
```
### Step 1.4: Get Changed Files and Diffs
@@ -105,9 +100,9 @@ Intelligently load only relevant knowledge:
1. Use GitHub API to discover available knowledge folders: `https://api.github.com/repos/Comfy-Org/comfy-claude-prompt-library/contents/.claude/knowledge`
2. For each knowledge folder, check if it's relevant by searching for the folder name in:
- Changed file paths
- PR title
- PR body
- Changed file paths
- PR title
- PR body
3. If relevant, download all files from that knowledge folder
### Step 2.4: Load Validation Rules
@@ -198,14 +193,12 @@ Consider:
For each issue found, create a concise inline comment with this structure:
```
**[category] severity Priority**
**Issue**: Brief description of the problem
**Context**: Why this matters
**Suggestion**: How to fix it
````
```
Categories: architecture/security/performance/quality
Severities: critical/high/medium/low
@@ -221,7 +214,7 @@ For EACH issue:
```bash
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/OWNER/REPO/pulls/$PR_NUMBER/comments -f body="$COMMENT_BODY" -f commit_id="COMMIT_SHA" -f path="FILE_PATH" -F line=LINE_NUMBER -f side="RIGHT"
````
```
CRITICAL: The entire command must be on one line. Use actual values, not placeholders.
@@ -230,14 +223,12 @@ CRITICAL: The entire command must be on one line. Use actual values, not placeho
Here's an example of how to review a file with a security issue:
1. First, get the repository info:
```bash
gh repo view --json owner,name
# Output: {"owner":{"login":"Comfy-Org"},"name":"ComfyUI_frontend"}
```
2. Get the commit SHA:
```bash
gh pr view $PR_NUMBER --json commits --jq '.commits[-1].oid'
# Output: abc123def456
@@ -249,17 +240,14 @@ Here's an example of how to review a file with a security issue:
```bash
# First, create the comment body with proper newlines
COMMENT_BODY="**[security] critical Priority**
```
**Issue**: SQL injection vulnerability - user input directly concatenated into query
**Context**: Allows attackers to execute arbitrary SQL commands
**Suggestion**: Use parameterized queries or prepared statements"
# Then post the comment (as a single line)
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/Comfy-Org/ComfyUI_frontend/pulls/$PR_NUMBER/comments -f body="$COMMENT_BODY" -f commit_id="abc123def456" -f path="src/db/queries.js" -F line=42 -f side="RIGHT"
```
# Then post the comment (as a single line)
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/Comfy-Org/ComfyUI_frontend/pulls/$PR_NUMBER/comments -f body="$COMMENT_BODY" -f commit_id="abc123def456" -f path="src/db/queries.js" -F line=42 -f side="RIGHT"
```
Repeat this process for every issue you find in the PR.
@@ -294,9 +282,9 @@ Analyze the PR to determine its type:
1. Extract PR title and body from pr_info.json
2. Count files, additions, and deletions
3. Determine PR type:
- Feature: Check for tests, documentation, backward compatibility
- Bug fix: Verify root cause addressed, includes regression tests
- Refactor: Ensure behavior preservation, tests still pass
- Feature: Check for tests, documentation, backward compatibility
- Bug fix: Verify root cause addressed, includes regression tests
- Refactor: Ensure behavior preservation, tests still pass
## Phase 7: Generate Comprehensive Summary
@@ -304,17 +292,16 @@ After ALL inline comments are posted, create a summary:
1. Calculate total issues by category and severity
2. Use `gh pr review $PR_NUMBER --comment` to post a summary with:
- Review disclaimer
- Issue distribution (counts by severity)
- Category breakdown
- Key findings for each category
- Positive observations
- References to guidelines
- Next steps
- Review disclaimer
- Issue distribution (counts by severity)
- Category breakdown
- Key findings for each category
- Positive observations
- References to guidelines
- Next steps
Include in the summary:
```
# Comprehensive PR Review
This review is generated by Claude. It may not always be accurate, as with human reviewers. If you believe that any of the comments are invalid or incorrect, please state why for each. For others, please implement the changes in one way or another.
@@ -325,14 +312,12 @@ This review is generated by Claude. It may not always be accurate, as with human
**Impact**: [X] additions, [Y] deletions across [Z] files
### Issue Distribution
- Critical: [CRITICAL_COUNT]
- High: [HIGH_COUNT]
- Medium: [MEDIUM_COUNT]
- Low: [LOW_COUNT]
### Category Breakdown
- Architecture: [ARCHITECTURE_ISSUES] issues
- Security: [SECURITY_ISSUES] issues
- Performance: [PERFORMANCE_ISSUES] issues
@@ -341,42 +326,33 @@ This review is generated by Claude. It may not always be accurate, as with human
## Key Findings
### Architecture & Design
[Detailed architectural analysis based on repository patterns]
### Security Considerations
[Security implications beyond basic vulnerabilities]
### Performance Impact
[Performance analysis including bundle size, render impact]
### Integration Points
[How this affects other systems, extensions, etc.]
## Positive Observations
[What was done well, good patterns followed]
## References
- [Repository Architecture Guide](https://github.com/Comfy-Org/comfy-claude-prompt-library/blob/master/project-summaries-for-agents/ComfyUI_frontend/REPOSITORY_GUIDE.md)
- [Frontend Standards](https://github.com/Comfy-Org/comfy-claude-prompt-library/blob/master/.claude/commands/validation/frontend-code-standards.md)
- [Security Guidelines](https://github.com/Comfy-Org/comfy-claude-prompt-library/blob/master/.claude/commands/validation/security-audit.md)
## Next Steps
1. Address critical issues before merge
2. Consider architectural feedback for long-term maintainability
3. Add tests for uncovered scenarios
4. Update documentation if needed
---
_This is a comprehensive automated review. For architectural decisions requiring human judgment, please request additional manual review._
*This is a comprehensive automated review. For architectural decisions requiring human judgment, please request additional manual review.*
```
## Important Guidelines
@@ -399,5 +375,4 @@ This is a COMPREHENSIVE review, not a linting pass. Provide the same quality fee
5. Phase 6: Consider PR type for additional checks
6. Phase 7: Post comprehensive summary ONLY after all inline comments
Remember: Individual inline comments for each issue, then one final summary. Never batch issues into a single comment.
```
Remember: Individual inline comments for each issue, then one final summary. Never batch issues into a single comment.

View File

@@ -7,9 +7,8 @@ Create a frontend release with version type: $ARGUMENTS
Expected format: Version increment type and optional description
Examples:
- `patch` - Bug fixes only
- `minor` - New features, backward compatible
- `minor` - New features, backward compatible
- `major` - Breaking changes
- `prerelease` - Alpha/beta/rc releases
- `patch "Critical security fixes"` - With custom description
@@ -22,9 +21,8 @@ If no arguments provided, the command will always perform prerelease if the curr
## Prerequisites
Before starting, ensure:
- You have push access to the repository
- GitHub CLI (`gh`) is authenticated
- GitHub CLI (`gh`) is authenticated
- You're on a clean main branch working tree
- All intended changes are merged to main
- You understand the scope of changes being released
@@ -32,7 +30,6 @@ Before starting, ensure:
## Critical Checks Before Starting
### 1. Check Current Version Status
```bash
# Get current version and check if it's a pre-release
CURRENT_VERSION=$(node -p "require('./package.json').version")
@@ -43,7 +40,6 @@ fi
```
### 2. Find Last Stable Release
```bash
# Get last stable release tag (no pre-release suffix)
LAST_STABLE=$(git tag -l "v*" | grep -v "\-" | sort -V | tail -1)
@@ -53,7 +49,6 @@ echo "Last stable release: $LAST_STABLE"
## Configuration Options
**Environment Variables:**
- `RELEASE_SKIP_SECURITY_SCAN=true` - Skip security audit
- `RELEASE_AUTO_APPROVE=true` - Skip some confirmation prompts
- `RELEASE_DRY_RUN=true` - Simulate release without executing
@@ -134,14 +129,13 @@ echo "Last stable release: $LAST_STABLE"
### Step 4: Analyze Dependency Updates
1. **Use pnpm's built-in dependency analysis:**
```bash
# Get outdated dependencies with pnpm
pnpm outdated --format table > outdated-deps-${NEW_VERSION}.txt
# Check for license compliance
pnpm licenses ls --json > licenses-${NEW_VERSION}.json
# Analyze why specific dependencies exist
echo "Dependency analysis:" > dep-analysis-${NEW_VERSION}.md
MAJOR_DEPS=("vue" "vite" "@vitejs/plugin-vue" "typescript" "pinia")
@@ -153,23 +147,22 @@ echo "Last stable release: $LAST_STABLE"
```
2. **Check for significant dependency updates:**
```bash
# Extract all dependency changes for major version bumps
OTHER_DEP_CHANGES=""
# Compare major dependency versions (you can extend this list)
MAJOR_DEPS=("vue" "vite" "@vitejs/plugin-vue" "typescript" "pinia")
for dep in "${MAJOR_DEPS[@]}"; do
PREV_VER=$(echo "$PREV_PACKAGE_JSON" | grep -o "\"$dep\": \"[^\"]*\"" | grep -o '[0-9][^"]*' | head -1 || echo "")
CURR_VER=$(echo "$CURRENT_PACKAGE_JSON" | grep -o "\"$dep\": \"[^\"]*\"" | grep -o '[0-9][^"]*' | head -1 || echo "")
if [ "$PREV_VER" != "$CURR_VER" ] && [ -n "$PREV_VER" ] && [ -n "$CURR_VER" ]; then
# Check if it's a major version change
PREV_MAJOR=$(echo "$PREV_VER" | cut -d. -f1 | sed 's/[^0-9]//g')
CURR_MAJOR=$(echo "$CURR_VER" | cut -d. -f1 | sed 's/[^0-9]//g')
if [ "$PREV_MAJOR" != "$CURR_MAJOR" ]; then
OTHER_DEP_CHANGES="${OTHER_DEP_CHANGES}\n- **${dep}**: ${PREV_VER} → ${CURR_VER} (Major version change)"
fi
@@ -180,12 +173,11 @@ echo "Last stable release: $LAST_STABLE"
### Step 5: Generate GTM Feature Summary
1. **Collect PR data for analysis:**
```bash
# Get list of PR numbers from commits
PR_NUMBERS=$(git log ${BASE_TAG}..HEAD --oneline --no-merges --first-parent | \
grep -oE "#[0-9]+" | tr -d '#' | sort -u)
# Save PR data for each PR
echo "[" > prs-${NEW_VERSION}.json
first=true
@@ -197,17 +189,16 @@ echo "Last stable release: $LAST_STABLE"
```
2. **Analyze for GTM-worthy features:**
```
<task>
Review these PRs to identify features worthy of marketing attention.
A feature is GTM-worthy if it meets ALL of these criteria:
- Introduces a NEW capability users didn't have before (not just improvements)
- Would be a compelling reason for users to upgrade to this version
- Can be demonstrated visually or has clear before/after comparison
- Affects a significant portion of the user base
NOT GTM-worthy:
- Bug fixes (even important ones)
- Minor UI tweaks or color changes
@@ -215,20 +206,19 @@ echo "Last stable release: $LAST_STABLE"
- Internal refactoring
- Small convenience features
- Features that only improve existing functionality marginally
For each GTM-worthy feature, note:
- PR number, title, and author
- Media links from the PR description
- One compelling sentence on why users should care
If there are no GTM-worthy features, just say "No marketing-worthy features in this release."
</task>
PR data: [contents of prs-${NEW_VERSION}.json]
```
3. **Generate GTM notification using this EXACT Slack-compatible format:**
```bash
# Only create file if GTM-worthy features exist:
if [ "$GTM_FEATURES_FOUND" = "true" ]; then
@@ -262,8 +252,8 @@ echo "Last stable release: $LAST_STABLE"
```
**CRITICAL Formatting Requirements:**
- Use single asterisk (\*) for emphasis, NOT double (\*\*)
- Use underscore (\_) for italics
- Use single asterisk (*) for emphasis, NOT double (**)
- Use underscore (_) for italics
- Use 4 spaces for indentation (not tabs)
- Convert author names to @username format (e.g., "John Smith" → "@john")
- No section headers (#), no code language specifications
@@ -273,7 +263,6 @@ echo "Last stable release: $LAST_STABLE"
### Step 6: Version Preview
**Version Preview:**
- Current: `${CURRENT_VERSION}`
- Proposed: Show exact version number based on analysis:
- Major version if breaking changes detected
@@ -305,6 +294,7 @@ echo "Last stable release: $LAST_STABLE"
1. Run complete test suite:
```bash
pnpm test:unit
pnpm test:component
```
2. Run type checking:
```bash
@@ -337,7 +327,6 @@ echo "Last stable release: $LAST_STABLE"
done
```
3. Create standardized release notes using this exact template:
```bash
cat > release-notes-${NEW_VERSION}.md << 'EOF'
## ⚠️ Breaking Changes
@@ -371,7 +360,6 @@ echo "Last stable release: $LAST_STABLE"
**Full Changelog**: https://github.com/Comfy-Org/ComfyUI_frontend/compare/${BASE_TAG}...v${NEW_VERSION}
EOF
```
4. **Parse commits and populate template:**
- Group commits by conventional commit type (feat:, fix:, chore:, etc.)
- Extract PR numbers from commit messages
@@ -388,19 +376,16 @@ echo "Last stable release: $LAST_STABLE"
### Step 10: Create Version Bump PR
**For standard version bumps (patch/minor/major):**
```bash
# Trigger the workflow
gh workflow run release-version-bump.yaml -f version_type=${VERSION_TYPE}
gh workflow run version-bump.yaml -f version_type=${VERSION_TYPE}
# Workflow runs quickly - usually creates PR within 30 seconds
echo "Workflow triggered. Waiting for PR creation..."
```
**For releasing a stable version:**
1. Must manually create branch and update version:
```bash
git checkout -b version-bump-${NEW_VERSION}
# Edit package.json to remove pre-release suffix
@@ -410,25 +395,23 @@ echo "Workflow triggered. Waiting for PR creation..."
```
2. Wait for PR creation (if using workflow) or create manually:
```bash
# For workflow-created PRs - wait and find it
sleep 30
# Look for PR from comfy-pr-bot (not github-actions)
PR_NUMBER=$(gh pr list --author comfy-pr-bot --limit 1 --json number --jq '.[0].number')
# Verify we got the PR
if [ -z "$PR_NUMBER" ]; then
echo "PR not found yet. Checking recent PRs..."
gh pr list --limit 5 --json number,title,author
fi
# For manual PRs
gh pr create --title "${NEW_VERSION}" \
--body-file release-notes-${NEW_VERSION}.md \
--label "Release"
```
3. **Update PR with release notes:**
```bash
# For workflow-created PRs, update the body with our release notes
@@ -443,21 +426,28 @@ echo "Workflow triggered. Waiting for PR creation..."
gh pr view ${PR_NUMBER} --json labels | jq -r '.labels[].name' | grep -q "Release" || \
echo "ERROR: Release label missing! Add it immediately!"
```
2. Verify version number in package.json
3. Review all changed files
4. Ensure no unintended changes included
5. Wait for required PR checks:
2. Check for update-locales commits:
```bash
# WARNING: update-locales may add [skip ci] which blocks release workflow!
gh pr view ${PR_NUMBER} --json commits | grep -q "skip ci" && \
echo "WARNING: [skip ci] detected - release workflow may not trigger!"
```
3. Verify version number in package.json
4. Review all changed files
5. Ensure no unintended changes included
6. Wait for required PR checks:
```bash
gh pr checks ${PR_NUMBER} --watch
```
6. **FINAL CODE REVIEW**: Release label present and no [skip ci]?
7. **FINAL CODE REVIEW**: Release label present and no [skip ci]?
### Step 12: Pre-Merge Validation
1. **Review Requirements**: Release PRs require approval
2. Monitor CI checks
3. Check no new commits to main since PR creation
4. **DEPLOYMENT READINESS**: Ready to merge?
2. Monitor CI checks - watch for update-locales
3. **CRITICAL WARNING**: If update-locales adds [skip ci], the release workflow won't trigger!
4. Check no new commits to main since PR creation
5. **DEPLOYMENT READINESS**: Ready to merge?
### Step 13: Execute Release
@@ -469,33 +459,32 @@ echo "Workflow triggered. Waiting for PR creation..."
3. **IMMEDIATELY CHECK**: Did release workflow trigger?
```bash
sleep 10
gh run list --workflow=release-draft-create.yaml --limit=1
gh run list --workflow=release.yaml --limit=1
```
4. **For Minor/Major Version Releases**: The release-branch-create workflow will automatically:
4. **For Minor/Major Version Releases**: The create-release-candidate-branch workflow will automatically:
- Create a `core/x.yy` branch for the PREVIOUS minor version
- Apply branch protection rules
- Document the feature freeze policy
```bash
# Monitor branch creation (for minor/major releases)
gh run list --workflow=release-branch-create.yaml --limit=1
gh run list --workflow=create-release-candidate-branch.yaml --limit=1
```
5. If workflow didn't trigger due to [skip ci]:
4. If workflow didn't trigger due to [skip ci]:
```bash
echo "ERROR: Release workflow didn't trigger!"
echo "Options:"
echo "1. Create patch release (e.g., 1.24.1) to trigger workflow"
echo "2. Investigate manual release options"
```
6. If workflow triggered, monitor execution:
5. If workflow triggered, monitor execution:
```bash
WORKFLOW_RUN_ID=$(gh run list --workflow=release-draft-create.yaml --limit=1 --json databaseId --jq '.[0].databaseId')
WORKFLOW_RUN_ID=$(gh run list --workflow=release.yaml --limit=1 --json databaseId --jq '.[0].databaseId')
gh run watch ${WORKFLOW_RUN_ID}
```
### Step 14: Enhance GitHub Release
1. Wait for automatic release creation:
```bash
# Wait for release to be created
while ! gh release view v${NEW_VERSION} >/dev/null 2>&1; do
@@ -505,14 +494,13 @@ echo "Workflow triggered. Waiting for PR creation..."
```
2. **Enhance the GitHub release:**
```bash
# Update release with our release notes
gh release edit v${NEW_VERSION} \
--title "🚀 ComfyUI Frontend v${NEW_VERSION}" \
--notes-file release-notes-${NEW_VERSION}.md \
--latest
# Add any additional assets if needed
# gh release upload v${NEW_VERSION} additional-assets.zip
```
@@ -525,17 +513,14 @@ echo "Workflow triggered. Waiting for PR creation..."
### Step 15: Verify Multi-Channel Distribution
1. **GitHub Release:**
```bash
gh release view v${NEW_VERSION} --json assets,body,createdAt,tagName
```
- ✅ Check release notes
- ✅ Verify dist.zip attachment
- ✅ Confirm release marked as latest (for main branch)
2. **PyPI Package:**
```bash
# Check PyPI availability (may take a few minutes)
for i in {1..10}; do
@@ -549,7 +534,6 @@ echo "Workflow triggered. Waiting for PR creation..."
```
3. **npm Package:**
```bash
# Check npm availability
for i in {1..10}; do
@@ -567,17 +551,15 @@ echo "Workflow triggered. Waiting for PR creation..."
### Step 16: Post-Release Monitoring Setup
1. **Monitor immediate release health:**
```bash
# Check for immediate issues
gh issue list --label "bug" --state open --limit 5 --json title,number,createdAt
# Monitor download metrics (if accessible)
gh release view v${NEW_VERSION} --json assets --jq '.assets[].downloadCount'
```
2. **Update documentation tracking:**
```bash
cat > post-release-checklist.md << EOF
# Post-Release Checklist for v${NEW_VERSION}
@@ -608,7 +590,6 @@ echo "Workflow triggered. Waiting for PR creation..."
```
3. **Create release summary:**
```bash
cat > release-summary-${NEW_VERSION}.md << EOF
# Release Summary: ComfyUI Frontend v${NEW_VERSION}
@@ -646,7 +627,6 @@ echo "Workflow triggered. Waiting for PR creation..."
### Step 17: Create Release Summary
1. **Create comprehensive release summary:**
```bash
cat > release-summary-${NEW_VERSION}.md << EOF
# Release Summary: ComfyUI Frontend v${NEW_VERSION}
@@ -686,7 +666,6 @@ echo "Workflow triggered. Waiting for PR creation..."
### Rollback Procedures
**Pre-Merge Rollback:**
```bash
# Close version bump PR and reset
gh pr close ${PR_NUMBER}
@@ -695,7 +674,6 @@ git clean -fd
```
**Post-Merge Rollback:**
```bash
# Create immediate patch release with reverts
git revert ${RELEASE_COMMIT}
@@ -703,7 +681,6 @@ git revert ${RELEASE_COMMIT}
```
**Emergency Procedures:**
```bash
# Document incident
cat > release-incident-${NEW_VERSION}.md << EOF
@@ -737,39 +714,31 @@ The command implements multiple quality gates:
## Common Scenarios
### Scenario 1: Regular Feature Release
```bash
/project:create-frontend-release minor
```
- Analyzes features since last release
- Generates changelog automatically
- Creates comprehensive release notes
### Scenario 2: Critical Security Patch
```bash
/project:create-frontend-release patch "Security fixes for CVE-2024-XXXX"
```
- Expedited security scanning
- Enhanced monitoring setup
### Scenario 3: Major Version with Breaking Changes
```bash
/project:create-frontend-release major
```
- Comprehensive breaking change analysis
- Migration guide generation
### Scenario 4: Pre-release Testing
```bash
/project:create-frontend-release prerelease
```
- Creates alpha/beta/rc versions
- Draft release status
- Python package specs require that prereleases use alpha/beta/rc as the preid
@@ -779,12 +748,10 @@ The command implements multiple quality gates:
When executing this release process, pay attention to these key aspects:
### Version Handling
- For pre-release versions (e.g., 1.24.0-rc.1), the next stable release should be the same version without the suffix (1.24.0)
- Never skip version numbers - follow semantic versioning strictly
### Commit History Analysis
- **ALWAYS** use `--first-parent` flag with git log to avoid including commits from merged feature branches
- Verify PR merge targets before including them in changelogs:
```bash
@@ -792,7 +759,6 @@ When executing this release process, pay attention to these key aspects:
```
### Release Workflow Triggers
- The "Release" label on the PR is **CRITICAL** - without it, PyPI/npm publishing won't occur
- Check for `[skip ci]` in commit messages before merging - this blocks the release workflow
- If you encounter `[skip ci]`, push an empty commit to override it:
@@ -801,13 +767,11 @@ When executing this release process, pay attention to these key aspects:
```
### PR Creation Details
- Version bump PRs come from `comfy-pr-bot`, not `github-actions`
- The workflow typically completes in 20-30 seconds
- Always wait for the PR to be created before trying to edit it
### Breaking Changes Detection
- Analyze changes to public-facing APIs:
- The `app` object and its methods
- The `api` module exports
@@ -816,10 +780,9 @@ When executing this release process, pay attention to these key aspects:
- Any modifications to these require marking as breaking changes
### Recovery Procedures
If the release workflow fails to trigger:
1. Create a revert PR to restore the previous version
2. Merge the revert
3. Re-run the version bump workflow
4. This approach is cleaner than creating extra version numbers

View File

@@ -3,11 +3,10 @@
This command creates patch/hotfix releases for ComfyUI Frontend by backporting fixes to stable core branches. It handles both automated backports (preferred) and manual cherry-picking (fallback).
**Process Overview:**
1. **Check automated backports first** (via labels)
2. **Skip to version bump** if backports already merged
3. **Manual cherry-picking** if automation failed
4. **Create patch release** with version bump
4. **Create patch release** with version bump
5. **Publish GitHub release** (manually uncheck "latest")
6. **Update ComfyUI requirements.txt** via PR
@@ -15,8 +14,7 @@ This command creates patch/hotfix releases for ComfyUI Frontend by backporting f
Create a hotfix release by backporting commits/PRs from main to a core branch: $ARGUMENTS
Expected format: Comma-separated list of commits or PR numbers
Examples:
Examples:
- `#1234,#5678` (PRs - preferred)
- `abc123,def456` (commit hashes)
- `#1234,abc123` (mixed)
@@ -27,7 +25,7 @@ If no arguments provided, the command will guide you through identifying commits
## Prerequisites
- Push access to repository
- GitHub CLI (`gh`) authenticated
- GitHub CLI (`gh`) authenticated
- Clean working tree
- Understanding of what fixes need backporting
@@ -38,13 +36,11 @@ If no arguments provided, the command will guide you through identifying commits
**Check if automated backports were attempted:**
1. **For each PR, check existing backport labels:**
```bash
gh pr view #1234 --json labels | jq -r '.labels[].name'
```
2. **If no backport labels exist, add them now:**
```bash
# Add backport labels (this triggers automated backports)
gh pr edit #1234 --add-label "needs-backport"
@@ -52,7 +48,6 @@ If no arguments provided, the command will guide you through identifying commits
```
3. **Check for existing backport PRs:**
```bash
# Check for backport PRs created by automation
PR_NUMBER=${ARGUMENTS%%,*} # Extract first PR number from arguments
@@ -63,22 +58,18 @@ If no arguments provided, the command will guide you through identifying commits
4. **Handle existing backport scenarios:**
**Scenario A: Automated backports already merged**
```bash
# Check if backport PRs were merged to core branches
gh pr list --search "backport-${PR_NUMBER}-to" --state merged
```
- If backport PRs are merged → Skip to Step 10 (Version Bump)
- **CONFIRMATION**: Automated backports completed, proceeding to version bump?
**Scenario B: Automated backport PRs exist but not merged**
```bash
# Show open backport PRs that need merging
gh pr list --search "backport-${PR_NUMBER}-to" --state open
```
- **ACTION REQUIRED**: Merge the existing backport PRs first
- Use: `gh pr merge [PR_NUMBER] --merge` for each backport PR
- After merging, return to this command and skip to Step 10 (Version Bump)
@@ -136,7 +127,6 @@ If no arguments provided, the command will guide you through identifying commits
### Step 6: Cherry-pick Changes
For each commit:
1. Attempt cherry-pick: `git cherry-pick <commit>`
2. If conflicts occur:
- Display conflict details
@@ -208,7 +198,6 @@ For each commit:
```
3. **CRITICAL**: Verify "Release" label is added
4. Create standardized release notes:
```bash
cat > release-notes-${NEW_VERSION}.md << 'EOF'
## ⚠️ Breaking Changes
@@ -242,14 +231,12 @@ For each commit:
**Full Changelog**: https://github.com/Comfy-Org/ComfyUI_frontend/compare/v${CURRENT_VERSION}...v${NEW_VERSION}
EOF
```
- For hotfixes, typically only populate the "Bug Fixes" section
- Include links to the cherry-picked PRs/commits
- Update the PR body with the release notes:
```bash
gh pr edit ${PR_NUMBER} --body-file release-notes-${NEW_VERSION}.md
```
5. **CONFIRMATION REQUIRED**: Release PR has "Release" label?
### Step 12: Monitor Release Process
@@ -259,7 +246,7 @@ For each commit:
3. Merge the PR: `gh pr merge --merge`
4. Monitor release workflow:
```bash
gh run list --workflow=release-draft-create.yaml --limit=1
gh run list --workflow=release.yaml --limit=1
gh run watch
```
5. Track progress:
@@ -275,7 +262,7 @@ For each commit:
2. **Find the DRAFT release** (e.g., "v1.23.5 Draft")
3. **Click "Edit release"**
4. **UNCHECK "Set as the latest release"** ⚠️ **CRITICAL**
- This prevents the hotfix from showing as "latest"
- This prevents the hotfix from showing as "latest"
- Main branch should always be "latest release"
5. **Click "Publish release"**
6. **CONFIRMATION REQUIRED**: Draft release published with "latest" unchecked?
@@ -285,7 +272,6 @@ For each commit:
**IMPORTANT**: Create PR to update ComfyUI's requirements.txt via fork:
1. **Setup fork (if needed):**
```bash
# Check if fork already exists
if gh repo view ComfyUI --json owner | jq -r '.owner.login' | grep -q "$(gh api user --jq .login)"; then
@@ -298,32 +284,30 @@ For each commit:
```
2. **Clone fork and create branch:**
```bash
# Clone your fork (or use existing clone)
GITHUB_USER=$(gh api user --jq .login)
if [ ! -d "ComfyUI-fork" ]; then
gh repo clone ${GITHUB_USER}/ComfyUI ComfyUI-fork
fi
cd ComfyUI-fork
git checkout master
git pull origin master
# Create update branch
BRANCH_NAME="update-frontend-${NEW_VERSION}"
git checkout -b ${BRANCH_NAME}
```
3. **Update requirements.txt:**
```bash
# Update the version in requirements.txt
sed -i "s/comfyui-frontend-package==[0-9].*$/comfyui-frontend-package==${NEW_VERSION}/" requirements.txt
# Verify the change
grep "comfyui-frontend-package" requirements.txt
# Commit the change
git add requirements.txt
git commit -m "Bump frontend to ${NEW_VERSION}"
@@ -337,8 +321,7 @@ For each commit:
--repo comfyanonymous/ComfyUI \
--title "Bump frontend to ${NEW_VERSION}" \
--body "$(cat <<EOF
Bump frontend to ${NEW_VERSION}
```
Bump frontend to ${NEW_VERSION}
\`\`\`
python main.py --front-end-version Comfy-Org/ComfyUI_frontend@${NEW_VERSION}
@@ -351,19 +334,15 @@ python main.py --front-end-version Comfy-Org/ComfyUI_frontend@${NEW_VERSION}
## Changes
- Fix: [Brief description of hotfixes included]
EOF
)"
```
```
EOF
)"
```
5. **Clean up:**
```bash
# Return to original directory
cd ..
# Keep fork directory for future updates
echo "Fork directory 'ComfyUI-fork' kept for future use"
```
@@ -396,7 +375,6 @@ python main.py --front-end-version Comfy-Org/ComfyUI_frontend@${NEW_VERSION}
## Safety Checks
Throughout the process:
- Always verify core branch matches ComfyUI's requirements.txt
- For PRs: Ensure using correct commits (merge vs individual)
- Check version numbers follow semantic versioning
@@ -407,7 +385,6 @@ Throughout the process:
## Rollback Procedures
If something goes wrong:
- Before push: `git reset --hard origin/core/X.Y`
- After PR creation: Close PR and start over
- After failed release: Create new patch version with fixes
@@ -427,16 +404,16 @@ If something goes wrong:
## Modern Workflow Context
**Primary Backport Method:** Automated via `needs-backport` + `X.YY` labels
**This Command Usage:**
**This Command Usage:**
- Smart path detection - skip to version bump if backports already merged
- Fallback to manual cherry-picking only when automation fails/has conflicts
**Complete Hotfix:** Includes GitHub release publishing + ComfyUI requirements.txt integration
**Complete Hotfix:** Includes GitHub release publishing + ComfyUI requirements.txt integration
## Workflow Paths
- **Path A:** Backports already merged → Skip to Step 10 (Version Bump)
- **Path B:** Backport PRs need merging → Merge them → Skip to Step 10 (Version Bump)
- **Path B:** Backport PRs need merging → Merge them → Skip to Step 10 (Version Bump)
- **Path C:** No/failed backports → Manual cherry-picking (Steps 2-9) → Version Bump (Step 10)
This process ensures a complete hotfix release with proper GitHub publishing, ComfyUI integration, and multiple safety checkpoints.
This process ensures a complete hotfix release with proper GitHub publishing, ComfyUI integration, and multiple safety checkpoints.

View File

@@ -5,7 +5,6 @@ Bootstrap the ComfyUI Frontend monorepo with all necessary dependencies and veri
## Overview
This command will:
1. Install pnpm package manager (if not present)
2. Install all project dependencies
3. Verify the project builds successfully
@@ -94,7 +93,7 @@ sleep 10
# Check if server is running
if curl -s http://localhost:5173 > /dev/null 2>&1; then
echo "✅ Development server started successfully at http://localhost:5173"
# Kill the background server
kill $SERVER_PID
wait $SERVER_PID 2>/dev/null
@@ -121,9 +120,10 @@ echo "Available commands:"
echo " pnpm dev - Start development server"
echo " pnpm build - Build for production"
echo " pnpm test:unit - Run unit tests"
echo " pnpm test:component - Run component tests"
echo " pnpm typecheck - Run TypeScript checks"
echo " pnpm lint - Run ESLint"
echo " pnpm format - Format code with oxfmt"
echo " pnpm format - Format code with Prettier"
echo ""
echo "Next steps:"
echo "1. Run 'pnpm dev' to start developing"
@@ -155,4 +155,4 @@ After running the setup, manually verify:
- Node.js >= 24
- Git repository
- Internet connection for package downloads
- Available ports (typically 5173 for dev server)
- Available ports (typically 5173 for dev server)

View File

@@ -12,10 +12,10 @@ Follow these steps systematically to verify our changes:
2. **Visual Testing Process**
- Navigate to http://localhost:5173/
- For each target page (specified in arguments or recently changed files):
- Navigate to the page using direct URL or site navigation
- Take a high-quality screenshot
- Analyze the screenshot for the specific changes we implemented
- Document any visual issues or improvements needed
* Navigate to the page using direct URL or site navigation
* Take a high-quality screenshot
* Analyze the screenshot for the specific changes we implemented
* Document any visual issues or improvements needed
3. **Quality Verification**
Check each page for:
@@ -27,7 +27,7 @@ Follow these steps systematically to verify our changes:
- Typography and readability
- Color scheme consistency
- Interactive elements (buttons, links, forms)
</instructions>
</instructions>
<examples>
Common issues to watch for:
@@ -48,11 +48,10 @@ For each page tested, provide:
4. Overall assessment of visual quality
If you find issues, be specific about:
- Exact location of the problem
- Expected vs actual behavior
- Severity level (critical, important, minor)
- Suggested fix if obvious
</reporting>
</reporting>
Remember: Take your time with each screenshot and analysis. Visual quality directly impacts user experience and our project's professional appearance.
Remember: Take your time with each screenshot and analysis. Visual quality directly impacts user experience and our project's professional appearance.

View File

@@ -1,9 +0,0 @@
{
"permissions": {
"allow": [
"Bash(pnpx vitest run --testPathPattern=\"draftCacheV2.property\")",
"Bash(pnpx vitest run \"draftCacheV2.property\")",
"Bash(node -e \"const fc = require\\(''fast-check''\\); console.log\\(Object.keys\\(fc\\).filter\\(k => k.includes\\(''string''\\)\\).join\\('', ''\\)\\)\")"
]
}
}

View File

@@ -1,84 +0,0 @@
---
name: adding-deprecation-warnings
description: 'Adds deprecation warnings for renamed or removed properties/APIs. Searches custom node ecosystem for usage, applies defineDeprecatedProperty helper, adds JSDoc. Triggers on: deprecate, deprecation warning, rename property, backward compatibility.'
---
# Adding Deprecation Warnings
Adds backward-compatible deprecation warnings for renamed or removed
properties using the `defineDeprecatedProperty` helper in
`src/lib/litegraph/src/utils/feedback.ts`.
## When to Use
- A property or API has been renamed and custom nodes still use the old name
- A property is being removed but needs a grace period
- Backward compatibility must be preserved while nudging adoption
## Steps
### 1. Search the Custom Node Ecosystem
Before implementing, assess impact by searching for usage of the
deprecated property across ComfyUI custom nodes:
```text
Use the comfy_codesearch tool to search for the old property name.
Search for both `widget.oldProp` and just `oldProp` to catch all patterns.
```
Document the usage patterns found (property access, truthiness checks,
caching to local vars, style mutation, etc.) — these all must continue
working.
### 2. Apply the Deprecation
Use `defineDeprecatedProperty` from `src/lib/litegraph/src/utils/feedback.ts`:
```typescript
import { defineDeprecatedProperty } from '@/lib/litegraph/src/utils/feedback'
/** @deprecated Use {@link obj.newProp} instead. */
defineDeprecatedProperty(
obj,
'oldProp',
'newProp',
'obj.oldProp is deprecated. Use obj.newProp instead.'
)
```
### 3. Checklist
- [ ] Ecosystem search completed — all usage patterns are compatible
- [ ] `defineDeprecatedProperty` call added after the new property is assigned
- [ ] JSDoc `@deprecated` tag added above the call for IDE support
- [ ] Warning message names both old and new property clearly
- [ ] `pnpm typecheck` passes
- [ ] `pnpm lint` passes
### 4. PR Comment
Add a PR comment summarizing the ecosystem search results: which repos
use the deprecated property, what access patterns were found, and
confirmation that all patterns are compatible with the ODP getter/setter.
## How `defineDeprecatedProperty` Works
- Creates an `Object.defineProperty` getter/setter on the target object
- Getter returns `this[currentKey]`, setter assigns `this[currentKey]`
- Both log via `warnDeprecated`, which deduplicates (once per unique
message per session via a `Set`)
- `enumerable: false` keeps the alias out of `Object.keys()` / `for...in`
/ `JSON.stringify`
- `configurable: true` allows further redefinition if needed
## Edge Cases
- **Truthiness checks** (`if (widget.oldProp)`) — works, getter fires
- **Caching to local var** (`const el = widget.oldProp`) — works, warns
once then the cached ref is used directly
- **Style/property mutation** (`widget.oldProp.style.color = 'red'`) —
works, getter returns the real object
- **Serialization** (`JSON.stringify`) — `enumerable: false` excludes it
- **Heavy access in loops** — `warnDeprecated` deduplicates, only warns
once per session regardless of call count

View File

@@ -1,336 +0,0 @@
---
name: backport-management
description: Manages cherry-pick backports across stable release branches. Discovers candidates from Slack/git, analyzes dependencies, resolves conflicts via worktree, and logs results. Use when asked to backport, cherry-pick to stable, manage release branches, do stable branch maintenance, or run a backport session.
---
# Backport Management
Cherry-pick backport management for Comfy-Org/ComfyUI_frontend stable release branches.
## Quick Start
1. **Discover** — Collect candidates from Slack bot + git log gap (`reference/discovery.md`)
2. **Analyze** — Categorize MUST/SHOULD/SKIP, check deps (`reference/analysis.md`)
3. **Human Review** — Present candidates in batches for interactive approval (see Interactive Approval Flow)
4. **Plan** — Order by dependency (leaf fixes first), group into waves per branch
5. **Execute** — Label-driven automation → worktree fallback for conflicts (`reference/execution.md`)
6. **Verify** — After each wave, verify branch integrity before proceeding
7. **Log & Report** — Generate session report (`reference/logging.md`)
## System Context
| Item | Value |
| -------------- | --------------------------------------------------------------------------- |
| Repo | `~/ComfyUI_frontend` (Comfy-Org/ComfyUI_frontend) |
| Merge strategy | Auto-merge via workflow (`--auto --squash`); `--admin` only after CI passes |
| Automation | `pr-backport.yaml` GitHub Action (label-driven, auto-merge enabled) |
| Tracking dir | `~/temp/backport-session/` |
## CI Safety Rules
**NEVER merge a backport PR without all CI checks passing.** This applies to both automation-created and manual cherry-pick PRs.
- **Automation PRs:** The `pr-backport.yaml` workflow now enables `gh pr merge --auto --squash`, so clean PRs auto-merge once CI passes. Monitor with polling (`gh pr list --base TARGET_BRANCH --state open`). Do not intervene unless CI fails.
- **Manual cherry-pick PRs:** After `gh pr create`, wait for CI before merging. Poll with `gh pr checks $PR --watch` or use a sleep+check loop. Only merge after all checks pass.
- **CI failures:** DO NOT use `--admin` to bypass failing CI. Analyze the failure, present it to the user with possible causes (test backported without implementation, missing dependency, flaky test), and let the user decide the next step.
## Branch Scope Rules
**Critical: Match PRs to the correct target branches.**
| Branch prefix | Scope | Example |
| ------------- | ------------------------------ | ------------------------------------------------- |
| `cloud/*` | Cloud-hosted ComfyUI only | Team workspaces, cloud queue, cloud-only login |
| `core/*` | Local/self-hosted ComfyUI only | Core editor, local workflows, node system |
| Both | Shared infrastructure | App mode, Firebase auth (API nodes), payment URLs |
### What Goes Where
**Both core + cloud:**
- **App mode** PRs — app mode is NOT cloud-only
- **Firebase auth** PRs — Firebase auth is on core for API nodes
- **Payment redirect** PRs — payment infrastructure shared
- **Bug fixes** touching shared components
**Cloud-only (skip for core):**
- Team workspaces
- Cloud queue virtualization
- Hide API key login
- Cloud-specific UI behind cloud feature flags
**⚠️ NEVER backport cloud-only PRs to `core/*` branches.** But do NOT assume "app mode" or "Firebase" = cloud-only. Check the actual files changed.
## ⚠️ Gotchas (Learn from Past Sessions)
### Use `gh api` for Labels — NOT `gh pr edit`
`gh pr edit --add-label` triggers Projects Classic deprecation errors. Always use:
```bash
gh api repos/Comfy-Org/ComfyUI_frontend/issues/$PR/labels \
-f "labels[]=needs-backport" -f "labels[]=TARGET_BRANCH"
```
### Automation Over-Reports Conflicts
The `pr-backport.yaml` action reports more conflicts than reality. `git cherry-pick -m 1` with git auto-merge handles many cases the automation can't. Always attempt manual cherry-pick before skipping.
### Never Skip Based on Conflict File Count
12 or 27 conflicting files can be trivial (snapshots, new files). **Categorize conflicts first**, then decide. See Conflict Triage below.
### Accept-Theirs Can Produce Broken Hybrids
When a PR **rewrites a component** (e.g., PrimeVue → Reka UI), the accept-theirs regex produces a broken mix of old and new code. The template may reference new APIs while the script still has old imports, or vice versa.
**Detection:** Content conflicts with 4+ conflict markers in a single `.vue` file, especially when imports change between component libraries.
**Fix:** Instead of accept-theirs regex, use `git show MERGE_SHA:path/to/file > path/to/file` to get the complete correct version from the merge commit on main. This bypasses the conflict entirely.
### Cherry-Picks Can Reference Missing Dependencies
When PR A on main depends on code introduced by PR B (which was merged before A), cherry-picking A brings in code that references B's additions. The cherry-pick succeeds but the branch is broken.
**Common pattern:** Composables, component files, or type definitions introduced by an earlier PR and used by the cherry-picked PR.
**Detection:** `pnpm typecheck` fails with "Cannot find module" or "is not defined" errors after cherry-pick.
**Fix:** Use `git show MERGE_SHA:path/to/missing/file > path/to/missing/file` to bring the missing files from main. Always verify with typecheck.
### Use `--no-verify` for Worktree Pushes
Husky hooks fail in worktrees (can't find lint-staged config). Always use `git push --no-verify` and `git commit --no-verify` when working in `/tmp/` worktrees.
### Automation Success Varies Wildly by Branch
In the 2026-04-06 session: core/1.42 got 18/26 auto-PRs, cloud/1.42 got only 1/25. The cloud branch has more divergence. **Always plan for manual fallback** — don't assume automation will handle most PRs.
## Conflict Triage
**Always categorize before deciding to skip. High conflict count ≠ hard conflicts.**
| Type | Symptom | Resolution |
| ---------------------------- | ------------------------------------ | --------------------------------------------------------------- |
| **Binary snapshots (PNGs)** | `.png` files in conflict list | `git checkout --theirs $FILE && git add $FILE` — always trivial |
| **Modify/delete (new file)** | PR introduces files not on target | `git add $FILE` — keep the new file |
| **Modify/delete (removed)** | Target removed files the PR modifies | `git rm $FILE` — file no longer relevant |
| **Content conflicts** | Marker-based (`<<<<<<<`) | Accept theirs via python regex (see below) |
| **Component rewrites** | 4+ markers in `.vue`, library change | Use `git show SHA:path > path` — do NOT accept-theirs |
| **Import-only conflicts** | Only import lines differ | Keep both imports if both used; remove unused after |
| **Add/add** | Both sides added same file | Accept theirs, verify no logic conflict |
| **Locale/JSON files** | i18n key additions | Accept theirs, validate JSON after |
```python
# Accept theirs for content conflicts
import re
pattern = r'<<<<<<< HEAD\n(.*?)=======\n(.*?)>>>>>>> [^\n]+\n?'
content = re.sub(pattern, r'\2', content, flags=re.DOTALL)
```
### Escalation Triggers (Flag for Human)
- **Package.json/lockfile changes** → skip on stable (transitive dep regression risk)
- **Core type definition changes** → requires human judgment
- **Business logic conflicts** (not just imports/exports) → requires domain knowledge
- **Admin-merged conflict resolutions** → get human review of the resolution before continuing the wave
## Auto-Skip Categories
Skip these without discussion:
- **Dep refresh PRs** — Risk of transitive dep regressions on stable. Cherry-pick individual CVE fixes instead.
- **CI/tooling changes** — Not user-facing
- **Test-only / lint rule changes** — Not user-facing
- **Revert pairs** — If PR A reverted by PR B, skip both. If fixed version (PR C) exists, backport only C.
- **Features not on target branch** — e.g., Painter, GLSLShader, appModeStore on core/1.40
- **Cloud-only PRs on core/\* branches** — Team workspaces, cloud queue, cloud-only login. (Note: app mode and Firebase auth are NOT cloud-only — see Branch Scope Rules)
## Wave Verification
After merging each wave of PRs to a target branch, verify branch integrity before proceeding:
```bash
# Fetch latest state of target branch
git fetch origin TARGET_BRANCH
# Quick smoke check: does the branch build?
git worktree add /tmp/verify-TARGET origin/TARGET_BRANCH
cd /tmp/verify-TARGET
source ~/.nvm/nvm.sh && nvm use 24 && pnpm install && pnpm typecheck && pnpm test:unit
git worktree remove /tmp/verify-TARGET --force
```
If typecheck or tests fail, stop and investigate before continuing. A broken branch after wave N means all subsequent waves will compound the problem.
### Fix PRs Are Normal
Expect to create 1 fix PR per branch after verification. Common issues:
1. **Component rewrite hybrids** — accept-theirs produced broken `.vue` files. Fix: overwrite with correct version from merge commit via `git show SHA:path > path`
2. **Missing dependency files** — cherry-pick brought in code referencing composables/components not on the branch. Fix: add missing files from merge commit
3. **Missing type properties** — cherry-picked code uses interface properties not yet on the branch (e.g., `key` on `ConfirmDialogOptions`). Fix: add the property to the interface
4. **Unused imports** — conflict resolution kept imports that the branch doesn't use. Fix: remove unused imports
5. **Wrong types from conflict resolution** — e.g., `{ top: number; right: number }` vs `{ top: number; left: number }`. Fix: match the return type of the actual function
Create a fix PR on a branch from the target, verify typecheck passes, then merge with `--squash --admin`.
### Never Admin-Merge Without CI
In a previous bulk session, all 69 backport PRs were merged with `gh pr merge --squash --admin`, bypassing required CI checks. This shipped 3 test failures to a release branch. **Lesson: `--admin` skips all branch protection, including required status checks.** Only use `--admin` after confirming CI has passed (e.g., `gh pr checks $PR` shows all green), or rely on auto-merge (`--auto --squash`) which waits for CI by design.
## Continuous Backporting Recommendation
Large backport sessions (50+ PRs) are expensive and error-prone. Prefer continuous backporting:
- Backport bug fixes as they merge to main (same day or next day)
- Use the automation labels immediately after merge
- Reserve session-style bulk backporting for catching up after gaps
- When a release branch is created, immediately start the continuous process
## Interactive Approval Flow
After analysis, present ALL candidates (MUST, SHOULD, and borderline) to the human for interactive review before execution. Do not write a static decisions.md — collect approvals in conversation.
### Batch Presentation
Present PRs in batches of 5-10, grouped by theme (visual bugs, interaction bugs, cloud/auth, data correctness, etc.). Use this table format:
```
# | PR | Title | Target | Rec | Context
----+--------+------------------------------------------+---------------+------+--------
1 | #12345 | fix: broken thing | core+cloud/42 | Y | Description here. Why it matters. Agent reasoning.
2 | #12346 | fix: another issue | core/42 | N | Only affects removed feature. Not on target branch.
```
Each row includes:
- PR number and title
- Target branches
- Agent recommendation: `Rec: Y` or `Rec: N` with brief reasoning
- 2-3 sentence context: what the PR does, why it matters (or doesn't)
### Human Response Format
- `Y` — approve for backport
- `N` — skip
- `?` — investigate (agent shows PR description, files changed, detailed take, then re-asks)
- Any freeform question or comment triggers discussion before moving on
- Bulk responses accepted (e.g. `1 Y, 2 Y, 3 N, 4 ?`)
### Rules
- ALL candidates are reviewed, not just MUST items
- When human responds `?`, show the PR description, files changed, and agent's detailed analysis, then re-ask for their decision
- When human asks a question about a PR, answer with context and recommendation, then wait for their decision
- Do not proceed to execution until all batches are reviewed and every candidate has a Y or N
## Quick Reference
### Label-Driven Automation (default path)
```bash
gh api repos/Comfy-Org/ComfyUI_frontend/issues/$PR/labels \
-f "labels[]=needs-backport" -f "labels[]=TARGET_BRANCH"
# Wait 3 min, check: gh pr list --base TARGET_BRANCH --state open
```
### Manual Worktree Cherry-Pick (conflict fallback)
```bash
git worktree add /tmp/backport-$BRANCH origin/$BRANCH
cd /tmp/backport-$BRANCH
# For each PR:
git fetch origin $BRANCH
git checkout -b backport-$PR-to-$BRANCH origin/$BRANCH
git cherry-pick -m 1 $MERGE_SHA
# Resolve conflicts (see Conflict Triage)
git push origin backport-$PR-to-$BRANCH --no-verify
gh pr create --base $BRANCH --head backport-$PR-to-$BRANCH \
--title "[backport $BRANCH] $TITLE (#$PR)" \
--body "Backport of #$PR. [conflict notes]"
gh pr merge $NEW_PR --squash --admin
sleep 25
```
### Efficient Batch: Test-Then-Resolve Pattern
When many PRs need manual cherry-pick (e.g., cloud branches), test all first:
```bash
cd /tmp/backport-$BRANCH
for pr in "${ORDER[@]}"; do
git checkout -b test-$pr origin/$BRANCH
if git cherry-pick -m 1 $SHA 2>/dev/null; then
echo "CLEAN: $pr"
else
echo "CONFLICT: $pr"
git cherry-pick --abort
fi
git checkout --detach HEAD
git branch -D test-$pr
done
```
Then process clean PRs in a batch loop, conflicts individually.
### PR Title Convention
```
[backport TARGET_BRANCH] Original Title (#ORIGINAL_PR)
```
## Final Deliverables (Slack-Compatible)
After execution completes, generate two files in `~/temp/backport-session/`. Both must be **Slack-compatible plain text** — no emojis, no markdown tables, no headers (`#`), no bold (`**`), no inline code. Use plain dashes, indentation, and line breaks only.
### 1. Author Accountability Report
File: `backport-author-accountability.md`
Lists all backported PRs grouped by original author (via `gh pr view $PR --json author`). Surfaces who should be self-labeling.
```
Backport Session YYYY-MM-DD -- PRs that should have been labeled by authors
- author-login
- #1234 fix: short title
- #5678 fix: another title
- other-author
- #9012 fix: some other fix
```
Authors sorted alphabetically, 4-space indent for nested items.
### 2. Slack Status Update
File: `slack-status-update.md`
A shareable summary of the session. Structure:
```
Backport session complete -- YYYY-MM-DD
[1-sentence summary: N PRs backported to which branches. All pass typecheck.]
Branches updated:
- core/X.XX: N PRs + N fix PRs (N auto, N manual)
- cloud/X.XX: N PRs + N fix PRs (N auto, N manual)
- ...
N total PRs created and merged (N backports + N fix PRs).
Notable fixes included:
- [category]: [list of fixes]
- ...
Conflict patterns encountered:
- [pattern and how it was resolved]
- ...
N authors had PRs backported. See author accountability list for details.
```
No emojis, no tables, no bold, no headers. Plain text that pastes cleanly into Slack.

View File

@@ -1,66 +0,0 @@
# Analysis & Decision Framework
## Categorization
| Category | Criteria | Action |
| -------------------- | --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| **MUST** | User-facing bug, crash, data corruption, security. Clear breakage that users will hit. | Backport (with deps if needed) |
| **SHOULD** | UX improvement, minor bug, small dep chain. No user-visible breakage if skipped, but improves experience. | Backport if clean cherry-pick; defer if conflict resolution is non-trivial |
| **SKIP** | CI/tooling, test-only, lint rules, cosmetic, dep refresh | Skip with documented reason |
| **NEEDS DISCUSSION** | Large dep chain, unclear risk/benefit, touches core types | Flag for human |
### MUST vs SHOULD Decision Guide
When unsure, ask: "If a user on this stable branch reports this issue, would we consider it a bug?"
- **Yes** → MUST. The fix addresses broken behavior.
- **No, but it's noticeably better** → SHOULD. The fix is a quality-of-life improvement.
- **No, and it's cosmetic or internal** → SKIP.
For SHOULD items with conflicts: if conflict resolution requires more than trivial accept-theirs patterns (content conflicts in business logic, not just imports), downgrade to SKIP or escalate to NEEDS DISCUSSION.
## Branch Scope Filtering
**Before categorizing, filter by branch scope:**
| Target branch | Skip if PR is... |
| ------------- | ----------------------------------------------------------------------------------------------------------------- |
| `core/*` | Cloud-only (team workspaces, cloud queue, cloud-only login). Note: app mode and Firebase auth are NOT cloud-only. |
| `cloud/*` | Local-only features not present on cloud branch |
Cloud-only PRs backported to `core/*` are wasted effort — `core/*` branches serve local/self-hosted users who never see cloud features. Check PR titles, descriptions, and files changed for cloud-specific indicators.
## Features Not on Stable Branches
Check before backporting — these don't exist on older branches:
- **Painter** (`src/extensions/core/painter.ts`) — not on core/1.40
- **GLSLShader** — not on core/1.40
- **App builder** — check per branch
- **appModeStore.ts** — not on core/1.40
## Dep Refresh PRs
Always SKIP on stable branches. Risk of transitive dependency regressions outweighs audit cleanup benefit. If a specific CVE fix is needed, cherry-pick that individual fix instead.
## Revert Pairs
If PR A is reverted by PR B:
- Skip BOTH A and B
- If a fixed version exists (PR C), backport only C
## Dependency Analysis
```bash
# Find other PRs that touched the same files
gh pr view $PR --json files --jq '.files[].path' | while read f; do
git log --oneline origin/TARGET..$MERGE_SHA -- "$f"
done
```
## Human Review Checkpoint
Use the Interactive Approval Flow (see SKILL.md) to review all candidates interactively. Do not write a static decisions.md for the human to edit — instead, present batches of 5-10 PRs with context and recommendations, and collect Y/N/? responses in conversation.
All candidates must be reviewed (MUST, SHOULD, and borderline items), not just a subset.

View File

@@ -1,42 +0,0 @@
# Discovery — Candidate Collection
## Source 1: Slack Backport-Checker Bot
Use `slackdump` skill to export `#frontend-releases` channel (C09K9TPU2G7):
```bash
slackdump export -o ~/slack-exports/frontend-releases.zip C09K9TPU2G7
```
Parse bot messages for PRs flagged "Might need backport" per release version.
## Source 2: Git Log Gap Analysis
```bash
# Count gap
git log --oneline origin/TARGET..origin/main | wc -l
# List gap commits
git log --oneline origin/TARGET..origin/main
# Check if a PR is already on target
git log --oneline origin/TARGET --grep="#PR_NUMBER"
# Check for existing backport PRs
gh pr list --base TARGET --state all --search "backport PR_NUMBER"
```
## Source 3: GitHub PR Details
```bash
# Get merge commit SHA
gh pr view $PR --json mergeCommit,title --jq '"Title: \(.title)\nMerge: \(.mergeCommit.oid)"'
# Get files changed
gh pr view $PR --json files --jq '.files[].path'
```
## Output: candidate_list.md
Table per target branch:
| PR# | Title | Category | Flagged by Bot? | Decision |

View File

@@ -1,270 +0,0 @@
# Execution Workflow
## Per-Branch Execution Order
1. Smallest gap first (validation run)
2. Medium gap next (quick win)
3. Largest gap last (main effort)
## Step 1: Label-Driven Automation (Batch)
```bash
# Add labels to all candidates for a target branch
for pr in $PR_LIST; do
gh api repos/Comfy-Org/ComfyUI_frontend/issues/$pr/labels \
-f "labels[]=needs-backport" -f "labels[]=TARGET_BRANCH" --silent
sleep 2
done
# Wait 3 minutes for automation
sleep 180
# Check which got auto-PRs (auto-merge is enabled, so clean ones will self-merge after CI)
gh pr list --base TARGET_BRANCH --state open --limit 50 --json number,title
```
> **Note:** The `pr-backport.yaml` workflow now enables `gh pr merge --auto --squash` on automation-created PRs. Clean PRs will auto-merge once CI passes — no manual merge needed for those.
## Step 2: Wait for CI & Merge Clean Auto-PRs
Most automation PRs will auto-merge once CI passes (via `--auto --squash` in the workflow). Monitor and handle failures:
```bash
# Wait for CI to complete (~45 minutes for full suite)
sleep 2700
# Check which PRs are still open (CI may have failed, or auto-merge succeeded)
STILL_OPEN_PRS=$(gh pr list --base TARGET_BRANCH --state open --limit 50 --json number --jq '.[].number')
RECENTLY_MERGED=$(gh pr list --base TARGET_BRANCH --state merged --limit 50 --json number,title,mergedAt)
# For PRs still open, check CI status
for pr in $STILL_OPEN_PRS; do
CI_FAILED=$(gh pr checks $pr --json name,state --jq '[.[] | select(.state == "FAILURE")] | length')
CI_PENDING=$(gh pr checks $pr --json name,state --jq '[.[] | select(.state == "PENDING" or .state == "QUEUED")] | length')
if [ "$CI_FAILED" != "0" ]; then
# CI failed — collect details for triage
echo "PR #$pr — CI FAILED:"
gh pr checks $pr --json name,state,link --jq '.[] | select(.state == "FAILURE") | "\(.name): \(.state)"'
elif [ "$CI_PENDING" != "0" ]; then
echo "PR #$pr — CI still running ($CI_PENDING checks pending)"
else
# All checks passed but didn't auto-merge (race condition or label issue)
gh pr merge $pr --squash --admin
sleep 3
fi
done
```
**⚠️ If CI fails: DO NOT admin-merge to bypass.** See "CI Failure Triage" below.
## Step 3: Manual Worktree for Conflicts
```bash
git fetch origin TARGET_BRANCH
git worktree add /tmp/backport-TARGET origin/TARGET_BRANCH
cd /tmp/backport-TARGET
for PR in ${CONFLICT_PRS[@]}; do
# Refresh target ref so each branch is based on current HEAD
git fetch origin TARGET_BRANCH
git checkout origin/TARGET_BRANCH
git checkout -b backport-$PR-to-TARGET origin/TARGET_BRANCH
git cherry-pick -m 1 $MERGE_SHA
# If conflict — NEVER skip based on file count alone!
# Categorize conflicts first: binary PNGs, modify/delete, content, add/add, component rewrites
# See SKILL.md Conflict Triage table for resolution per type.
# For component rewrites (4+ markers in a .vue file, library migration):
# DO NOT use accept-theirs regex — it produces broken hybrids.
# Instead, use the complete file from the merge commit:
# git show $MERGE_SHA:path/to/file > path/to/file
# For simple content conflicts, accept theirs:
# python3 -c "import re; ..."
# Resolve all conflicts, then:
git add .
GIT_EDITOR=true git cherry-pick --continue
git push origin backport-$PR-to-TARGET --no-verify
NEW_PR=$(gh pr create --base TARGET_BRANCH --head backport-$PR-to-TARGET \
--title "[backport TARGET] TITLE (#$PR)" \
--body "Backport of #$PR..." | grep -oP '\d+$')
# Wait for CI before merging — NEVER admin-merge without CI passing
echo "Waiting for CI on PR #$NEW_PR..."
gh pr checks $NEW_PR --watch --fail-fast || {
echo "⚠️ CI failed on PR #$NEW_PR — skipping merge, needs triage"
continue
}
gh pr merge $NEW_PR --squash --admin
sleep 3
done
# Cleanup
cd -
git worktree remove /tmp/backport-TARGET --force
```
**⚠️ Human review for conflict resolutions:** When admin-merging a PR where you manually resolved conflicts (especially content conflicts beyond trivial accept-theirs), pause and present the resolution diff to the human for review before merging. Trivial resolutions (binary snapshots, modify/delete, locale key additions) can proceed without review.
## Step 4: Wave Verification
After completing all PRs in a wave for a target branch:
```bash
git fetch origin TARGET_BRANCH
git worktree add /tmp/verify-TARGET origin/TARGET_BRANCH
cd /tmp/verify-TARGET
source ~/.nvm/nvm.sh && nvm use 24 && pnpm install && pnpm typecheck && pnpm test:unit
git worktree remove /tmp/verify-TARGET --force
```
If verification fails, **do not skip** — create a fix PR:
```bash
# Stay in the verify worktree
git checkout -b fix-backport-TARGET origin/TARGET_BRANCH
# Common fixes:
# 1. Component rewrite hybrids: overwrite with merge commit version
git show MERGE_SHA:path/to/Component.vue > path/to/Component.vue
# 2. Missing dependency files
git show MERGE_SHA:path/to/missing.ts > path/to/missing.ts
# 3. Missing type properties: edit the interface
# 4. Unused imports: delete the import lines
git add -A
git commit --no-verify -m "fix: resolve backport typecheck issues on TARGET"
git push origin fix-backport-TARGET --no-verify
gh pr create --base TARGET --head fix-backport-TARGET --title "fix: resolve backport typecheck issues on TARGET" --body "..."
gh pr merge $PR --squash --admin
```
Do not proceed to the next branch until typecheck passes.
## Conflict Resolution Patterns
### 1. Content Conflicts (accept theirs)
```python
import re
pattern = r'<<<<<<< HEAD\n(.*?)=======\n(.*?)>>>>>>> [^\n]+\n?'
content = re.sub(pattern, r'\2', content, flags=re.DOTALL)
```
### 2. Modify/Delete (two cases!)
```bash
# Case A: PR introduces NEW files not on target → keep them
git add $FILE
# Case B: Target REMOVED files the PR modifies → drop them
git rm $FILE
```
### 3. Binary Files (snapshots)
```bash
git checkout --theirs $FILE && git add $FILE
```
### 4. Component Rewrites (DO NOT accept-theirs)
When a PR completely rewrites a component (e.g., PrimeVue → Reka UI), accept-theirs produces
a broken hybrid with mismatched template/script sections.
```bash
# Use the complete correct file from the merge commit instead:
git show $MERGE_SHA:src/components/input/MultiSelect.vue > src/components/input/MultiSelect.vue
git show $MERGE_SHA:src/components/input/SingleSelect.vue > src/components/input/SingleSelect.vue
git add src/components/input/MultiSelect.vue src/components/input/SingleSelect.vue
```
**Detection:** 4+ conflict markers in a single `.vue` file, imports changing between component
libraries (PrimeVue → Reka UI, etc.), template structure completely different on each side.
### 5. Missing Dependencies After Cherry-Pick
Cherry-picks can succeed but leave the branch broken because the PR's code on main
references composables/components introduced by an earlier PR.
```bash
# Add the missing file from the merge commit:
git show $MERGE_SHA:src/composables/queue/useJobDetailsHover.ts > src/composables/queue/useJobDetailsHover.ts
git show $MERGE_SHA:src/components/builder/BuilderSaveDialogContent.vue > src/components/builder/BuilderSaveDialogContent.vue
```
**Detection:** `pnpm typecheck` fails with "Cannot find module" or "X is not defined" after cherry-pick succeeds cleanly.
### 6. Locale Files
Usually adding new i18n keys — accept theirs, validate JSON:
```bash
python3 -c "import json; json.load(open('src/locales/en/main.json'))" && echo "Valid"
```
## Merge Conflicts After Other Merges
When merging multiple PRs to the same branch, later PRs may conflict with earlier merges:
```bash
git fetch origin TARGET_BRANCH
git rebase origin/TARGET_BRANCH
# Resolve new conflicts
git push --force origin backport-$PR-to-TARGET
sleep 20 # Wait for GitHub to recompute merge state
# Wait for CI after rebase before merging
gh pr checks $PR --watch --fail-fast && gh pr merge $PR --squash --admin
```
## Lessons Learned
1. **Automation reports more conflicts than reality**`cherry-pick -m 1` with git auto-merge handles many "conflicts" the automation can't
2. **Never skip based on conflict file count** — 12 or 27 conflicts can be trivial (snapshots, new files). Categorize first: binary PNGs, modify/delete, content, add/add.
3. **Modify/delete goes BOTH ways** — if the PR introduces new files (not on target), `git add` them. If target deleted files the PR modifies, `git rm`.
4. **Binary snapshot PNGs** — always `git checkout --theirs && git add`. Never skip a PR just because it has many snapshot conflicts.
5. **Batch label additions need 2s delay** between API calls to avoid rate limits
6. **Merging 6+ PRs rapidly** can cause later PRs to become unmergeable — wait 20-30s for GitHub to recompute merge state
7. **appModeStore.ts, painter files, GLSLShader files** don't exist on core/1.40 — `git rm` these
8. **Always validate JSON** after resolving locale file conflicts
9. **Dep refresh PRs** — skip on stable branches. Risk of transitive dep regressions outweighs audit cleanup. Cherry-pick individual CVE fixes instead.
10. **Verify after each wave** — run `pnpm typecheck && pnpm test:unit` on the target branch after merging a batch. Catching breakage early prevents compounding errors.
11. **App mode and Firebase auth are NOT cloud-only** — they go to both core and cloud branches. Only team workspaces, cloud queue, and cloud-specific login are cloud-only.
12. **Never admin-merge without CI**`--admin` bypasses all branch protections including required status checks. A bulk session of 69 admin-merges shipped 3 test failures. Always wait for CI to pass first, or use `--auto --squash` which waits by design.
13. **Accept-theirs regex breaks component rewrites** — when a PR migrates between component libraries (PrimeVue → Reka UI), the regex produces a broken hybrid. Use `git show SHA:path > path` to get the complete correct version instead.
14. **Cherry-picks can silently bring in missing-dependency code** — if PR A references a composable introduced by PR B, cherry-picking A succeeds but typecheck fails. Always run typecheck after each wave and add missing files from the merge commit.
15. **Fix PRs are expected** — plan for 1 fix PR per branch to resolve typecheck issues from conflict resolutions. This is normal, not a failure.
16. **Use `--no-verify` in worktrees** — husky hooks fail in `/tmp/` worktrees. Always push/commit with `--no-verify`.
17. **Automation success varies by branch** — core/1.42 got 18/26 auto-PRs (69%), cloud/1.42 got 1/25 (4%). Cloud branches diverge more. Plan for manual fallback.
18. **Test-then-resolve pattern** — for branches with low automation success, run a dry-run loop to classify clean vs conflict PRs before processing. This is much faster than resolving conflicts serially.
## CI Failure Triage
When CI fails on a backport PR, present failures to the user using this template:
```markdown
### PR #XXXX — CI Failed
- **Failing check:** test / lint / typecheck
- **Error:** (summary of the failure message)
- **Likely cause:** test backported without implementation / missing dependency / flaky test / snapshot mismatch
- **Recommendation:** backport PR #YYYY first / skip this PR / rerun CI after fixing prerequisites
```
Common failure categories:
| Category | Example | Resolution |
| --------------------------- | ---------------------------------------- | ----------------------------------------- |
| Test without implementation | Test references function not on branch | Backport the implementation PR first |
| Missing dependency | Import from module not on branch | Backport the dependency PR first, or skip |
| Snapshot mismatch | Screenshot test differs | Usually safe — update snapshots on branch |
| Flaky test | Passes on retry | Re-run CI, merge if green on retry |
| Type error | Interface changed on main but not branch | May need manual adaptation |
**Never assume a failure is safe to skip.** Present all failures to the user with analysis.

View File

@@ -1,103 +0,0 @@
# Logging & Session Reports
## During Execution
Maintain `execution-log.md` with per-branch tables (this is internal, markdown tables are fine here):
```markdown
| PR# | Title | Status | Backport PR | Notes |
| ----- | ----- | ------ | ----------- | ------- |
| #XXXX | Title | merged | #YYYY | Details |
```
## Wave Verification Log
Track verification results per wave within execution-log.md:
```markdown
Wave N Verification -- TARGET_BRANCH
- PRs merged: #A, #B, #C
- Typecheck: pass / fail
- Fix PR: #YYYY (if needed)
- Issues found: (if any)
```
## Session Report Template
```markdown
# Backport Session Report
## Summary
| Branch | Candidates | Merged | Skipped | Deferred | Rate |
| ------ | ---------- | ------ | ------- | -------- | ---- |
## Deferred Items (Needs Human)
| PR# | Title | Branch | Issue |
## Conflict Resolutions Requiring Review
| PR# | Branch | Conflict Type | Resolution Summary |
## CI Failure Report
| PR# | Branch | Failing Check | Error Summary | Cause | Resolution |
| --- | ------ | ------------- | ------------- | ----- | ---------- |
## Automation Performance
| Metric | Value |
| --------------------------- | ----- |
| Auto success rate | X% |
| Manual resolution rate | X% |
| Overall clean rate | X% |
| Wave verification pass rate | X% |
## Process Recommendations
- Were there clusters of related PRs that should have been backported together?
- Any PRs that should have been backported sooner (continuous backporting candidates)?
- Feature branches that need tracking for future sessions?
```
## Final Deliverables
After all branches are complete and verified, generate these files in `~/temp/backport-session/`:
### 1. execution-log.md (internal)
Per-branch tables with PR#, title, status, backport PR#, notes. Markdown tables are fine — this is for internal tracking, not Slack.
### 2. backport-author-accountability.md (Slack-compatible)
See SKILL.md "Final Deliverables" section. Plain text, no emojis/tables/headers/bold. Authors sorted alphabetically with PRs nested under each.
### 3. slack-status-update.md (Slack-compatible)
See SKILL.md "Final Deliverables" section. Plain text summary that pastes cleanly into Slack. Includes branch counts, notable fixes, conflict patterns, author count.
## Slack Formatting Rules
Both shareable files (author accountability + status update) must follow these rules:
- No emojis (no checkmarks, no arrows, no icons)
- No markdown tables (use plain lists with dashes)
- No headers (no # or ##)
- No bold (\*_) or italic (_)
- No inline code backticks
- Use -- instead of em dash
- Use plain dashes (-) for lists with 4-space indent for nesting
- Line breaks between sections for readability
These files should paste directly into a Slack message and look clean.
## Files to Track
All in `~/temp/backport-session/`:
- `execution-plan.md` -- approved PRs with merge SHAs (input)
- `execution-log.md` -- real-time status with per-branch tables (internal)
- `backport-author-accountability.md` -- PRs grouped by author (Slack-compatible)
- `slack-status-update.md` -- session summary (Slack-compatible)

View File

@@ -1,278 +0,0 @@
---
name: reproduce-issue
description: 'Reproduce a GitHub issue by researching prerequisites, setting up the environment (custom nodes, workflows, settings), and interactively exploring ComfyUI via playwright-cli until the bug is confirmed. Then records a clean demo video.'
---
# Issue Reproduction Skill
Reproduce a reported GitHub issue against a running ComfyUI instance. This skill uses an interactive, agent-driven approach — not a static script. You will research, explore, retry, and adapt until the bug is reproduced, then record a clean demo.
## Architecture
Two videos are produced:
1. **Research video** — the full exploration session: installing deps, trying things, failing, retrying, figuring out the bug. Valuable for debugging context.
2. **Reproduce video** — a clean, minimal recording of just the reproduction steps. This is the demo you'd attach to the issue.
```
Phase 1: Research → Read issue, understand prerequisites
Phase 2: Environment → Install custom nodes, load workflows, configure settings
Phase 3: Explore → [VIDEO 1: research] Interactively try to reproduce (retries OK)
Phase 4: Record → [VIDEO 2: reproduce] Clean recording of just the minimal repro steps
Phase 5: Report → Generate a structured reproduction report
```
## Prerequisites
- ComfyUI server running (ask user for URL, default: `http://127.0.0.1:8188`)
- `playwright-cli` installed: `npm install -g @playwright/cli@latest`
- `gh` CLI (authenticated, for reading issues)
- ComfyUI backend with Python environment (for installing custom nodes)
## Phase 1: Research the Issue
1. Fetch the issue details:
```bash
gh issue view <number> --repo Comfy-Org/ComfyUI_frontend --json title,body,comments
```
2. Extract from the issue body:
- **Reproduction steps** (the exact sequence)
- **Prerequisites**: specific workflows, custom nodes, settings, models
- **Environment**: OS, browser, ComfyUI version
- **Media**: screenshots or videos showing the bug
3. Search the codebase for related code:
- Find the feature/component mentioned in the issue
- Understand how it works currently
- Identify what state the UI needs to be in
## Phase 2: Environment Setup
Set up everything the issue requires BEFORE attempting reproduction.
### Custom Nodes
If the issue mentions custom nodes:
```bash
# Find the custom node repo
# Clone into ComfyUI's custom_nodes directory
cd <comfyui_path>/custom_nodes
git clone <custom_node_repo_url>
# Install dependencies if needed
cd <custom_node_name>
pip install -r requirements.txt 2>/dev/null || true
# Restart ComfyUI server to load the new nodes
```
### Workflows
If the issue references a specific workflow:
```bash
# Download workflow JSON if a URL is provided
curl -L "<workflow_url>" -o /tmp/test-workflow.json
# Load it via the API
curl -X POST http://127.0.0.1:8188/api/workflow \
-H "Content-Type: application/json" \
-d @/tmp/test-workflow.json
```
Or load via playwright-cli:
```bash
playwright-cli goto "http://127.0.0.1:8188"
# Drag-and-drop or use File > Open to load the workflow
```
### Settings
If the issue requires specific settings:
```bash
# Use playwright-cli to open settings and change them
playwright-cli press "Control+,"
playwright-cli snapshot
# Find and modify the relevant setting
```
## Phase 3: Interactive Exploration — Research Video
Start recording the **research video** (Video 1). This captures the full exploration — mistakes, retries, dead ends — all valuable context.
```bash
# Open browser and start video recording
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
# Take a snapshot to see current state
playwright-cli snapshot
# Interact based on what you see
playwright-cli click <ref>
playwright-cli fill <ref> "text"
playwright-cli press "Control+s"
# Check results
playwright-cli snapshot
playwright-cli screenshot --filename=/tmp/qa/research-step-1.png
```
### Key Principles
- **Observe before acting**: Always `snapshot` before interacting
- **Retry and adapt**: If a step fails, try a different approach
- **Document what works**: Keep notes on which steps trigger the bug
- **Don't give up**: Try multiple approaches if the first doesn't work
- **Establish prerequisites**: Many bugs require specific UI state:
- Save a workflow first (File > Save)
- Make changes to dirty the workflow
- Open multiple tabs
- Add specific node types
- Change settings
- Resize the window
### Common ComfyUI Interactions via playwright-cli
| Action | Command |
| ------------------- | -------------------------------------------------------------- |
| Open hamburger menu | `playwright-cli click` on the C logo button |
| Navigate menu | `playwright-cli hover <ref>` then `playwright-cli click <ref>` |
| Add node | Double-click canvas → type node name → select from results |
| Connect nodes | Drag from output slot to input slot |
| Save workflow | `playwright-cli press "Control+s"` |
| Save As | Menu > File > Save As |
| Select node | Click on the node |
| Delete node | Select → `playwright-cli press "Delete"` |
| Right-click menu | `playwright-cli click <ref> --button right` |
| Keyboard shortcut | `playwright-cli press "Control+z"` |
## Phase 4: Record Clean Demo — Reproduce Video (max 5 minutes)
Once the bug is confirmed, **stop the research video** and **close the research browser**:
```bash
playwright-cli video-stop
playwright-cli close
```
Now start a **fresh browser session** for the clean reproduce video (Video 2).
**IMPORTANT constraints:**
- **Max 5 minutes** — the reproduce video must be short and focused
- **No environment setup** — server, user, custom nodes are already set up from Phase 3. Just log in and go.
- **No exploration** — you already know the exact steps. Execute them quickly and precisely.
- **Start video recording immediately**, execute steps, stop. Don't leave the recording running while thinking.
1. **Open browser and start recording**:
```bash
playwright-cli open "http://127.0.0.1:8188"
playwright-cli video-start
```
2. **Execute only the minimal reproduction steps** — no exploration, no mistakes. Just the clean sequence that demonstrates the bug. You already know exactly what works from Phase 3.
3. **Take key screenshots** at critical moments:
```bash
playwright-cli screenshot --filename=/tmp/qa/before-bug.png
# ... trigger the bug ...
playwright-cli screenshot --filename=/tmp/qa/bug-visible.png
```
4. **Stop recording and close** immediately after the bug is demonstrated:
```bash
playwright-cli video-stop
playwright-cli close
```
## Phase 5: Generate Report
Create a reproduction report at `tmp/qa/reproduce-report.md`:
```markdown
# Issue Reproduction Report
- **Issue**: <issue_url>
- **Title**: <issue_title>
- **Date**: <today>
- **Status**: Reproduced / Not Reproduced / Partially Reproduced
## Environment
- ComfyUI Server: <url>
- OS: <os>
- Custom Nodes Installed: <list or "none">
- Settings Changed: <list or "none">
## Prerequisites
List everything that had to be set up before the bug could be triggered:
1. ...
2. ...
## Reproduction Steps
Minimal steps to reproduce (the clean sequence):
1. ...
2. ...
3. ...
## Expected Behavior
<from the issue>
## Actual Behavior
<what actually happened>
## Evidence
- Research video: `research-video/video.webm` (full exploration session)
- Reproduce video: `reproduce-video/video.webm` (clean minimal repro)
- Screenshots: `before-bug.png`, `bug-visible.png`
## Root Cause Analysis (if identified)
<code pointers, hypothesis about what's going wrong>
## Notes
<any additional observations, workarounds discovered, related issues>
```
## Handling Failures
If the bug **cannot be reproduced**:
1. Document what you tried and why it didn't work
2. Check if the issue was already fixed (search git log for related commits)
3. Check if it's environment-specific (OS, browser, specific version)
4. Set report status to "Not Reproduced" with detailed notes
5. The report is still valuable — it saves others from repeating the same investigation
## CI Integration
In CI, this skill runs as a Claude Code agent with:
- `ANTHROPIC_API_KEY` for Claude
- `GEMINI_API_KEY` for initial issue analysis (optional)
- ComfyUI server pre-started in the container
- `playwright-cli` pre-installed
The CI workflow:
1. Gemini generates a reproduce guide (markdown) from the issue
2. Claude agent receives the guide and runs this skill
3. Claude explores interactively, installs dependencies, retries
4. Claude records a clean demo once reproduced
5. Video and report are uploaded as artifacts

View File

@@ -1,283 +0,0 @@
---
name: comfy-qa
description: 'Comprehensive QA of ComfyUI frontend. Navigates all routes, tests all interactive features using playwright-cli, generates a report, and submits a draft PR. Works in CI and local environments, cross-platform.'
---
# ComfyUI Frontend QA Skill
Automated quality assurance for the ComfyUI frontend. The pipeline reproduces reported bugs using Playwright E2E tests, records video evidence, and deploys reports to Cloudflare Pages.
## Architecture Overview
The QA pipeline uses a **three-phase approach**:
1. **RESEARCH** — Claude writes Playwright E2E tests to reproduce bugs (assertion-backed, no hallucination)
2. **REPRODUCE** — Deterministic replay of the research test with video recording
3. **REPORT** — Deploy results to Cloudflare Pages with badge, video, and verdict
### Key Design Decision
Earlier iterations used AI vision (Gemini) to drive a browser and judge results from video. This was abandoned after discovering **AI reviewers hallucinate** — Gemini reported "REPRODUCED" when videos showed idle screens. The current approach uses **Playwright assertions** as the source of truth: if the test passes, the bug is proven.
## Prerequisites
- Node.js 22+
- `pnpm` package manager
- `gh` CLI (authenticated)
- Playwright browsers: `npx playwright install chromium`
- Environment variables:
- `GEMINI_API_KEY` — for PR analysis and video review
- `ANTHROPIC_API_KEY` — for Claude Agent SDK (research phase)
- `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` — for report deployment
## Pipeline Scripts
| Script | Role | Model |
| --------------------------------- | ------------------------------------------------------- | ----------------------------- |
| `scripts/qa-analyze-pr.ts` | Deep PR/issue analysis → QA guide | gemini-3.1-pro-preview |
| `scripts/qa-agent.ts` | Research phase: Claude writes E2E tests | claude-sonnet-4-6 (Agent SDK) |
| `scripts/qa-record.ts` | Before/after video recording with Gemini-driven actions | gemini-3.1-pro-preview |
| `scripts/qa-reproduce.ts` | Deterministic replay with narration | gemini-3-flash-preview |
| `scripts/qa-video-review.ts` | Video comparison review | gemini-3-flash-preview |
| `scripts/qa-generate-test.ts` | Regression test generation from QA report | gemini-3-flash-preview |
| `scripts/qa-deploy-pages.sh` | Deploy to Cloudflare Pages + badge | — |
| `scripts/qa-batch.sh` | Batch-trigger QA for multiple issues | — |
| `scripts/qa-report-template.html` | Report site (light/dark, seekbar, copy badge) | — |
## Triggering QA
### Via GitHub Labels
- **`qa-changes`** — Focused QA on a PR (Linux-only, before/after comparison)
- **`qa-full`** — Full QA (3-OS matrix, after-only)
- **`qa-issue`** — Reproduce a bug from an issue
### Via Batch Script
```bash
# Trigger QA for specific issue numbers
./scripts/qa-batch.sh 10394 10238 9996
# From a triage file (top 5 Tier 1 issues)
./scripts/qa-batch.sh --from tmp/issues.md --top 5
# Preview without pushing
./scripts/qa-batch.sh --dry-run 10394
# Clean up old trigger branches
./scripts/qa-batch.sh --cleanup
```
### Via Workflow Dispatch
Go to Actions → "PR: QA" → Run workflow → choose mode (focused/full).
## CI Workflow (`.github/workflows/pr-qa.yaml`)
```
resolve-matrix → analyze-pr ──┐
├→ qa-before (main branch, worktree build)
├→ qa-after (PR branch)
└→ report (video review, deploy, comment)
```
Before/after jobs run **in parallel** on separate runners for clean isolation.
### Issue Reproduce Mode
For issues (not PRs), the pipeline:
1. Fetches the issue body and comments
2. Runs `qa-analyze-pr.ts --type issue` to generate a QA guide
3. Runs the research phase (Claude writes E2E test to reproduce)
4. Records video of the test execution
5. Posts results as a comment on the issue
## Running Locally
### Step 1: Environment Setup
```bash
# Ensure ComfyUI server is running
# Default: http://127.0.0.1:8188
# Install Playwright browsers
npx playwright install chromium
```
### Step 2: Analyze the Issue/PR
```bash
# For a PR
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides
# For an issue
pnpm exec tsx scripts/qa-analyze-pr.ts \
--pr-number 10394 \
--repo Comfy-Org/ComfyUI_frontend \
--output-dir qa-guides \
--type issue
```
### Step 3: Record Before/After
```bash
# Before (main branch)
pnpm exec tsx scripts/qa-record.ts \
--mode before \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-before \
--qa-guide qa-guides/qa-guide-1.json
# After (PR branch)
pnpm exec tsx scripts/qa-record.ts \
--mode after \
--diff /tmp/pr-diff.txt \
--output-dir /tmp/qa-after \
--qa-guide qa-guides/qa-guide-1.json
```
### Step 4: Review Videos
```bash
pnpm exec tsx scripts/qa-video-review.ts \
--artifacts-dir /tmp/qa-artifacts \
--video-file qa-session.mp4 \
--before-video qa-before-session.mp4 \
--output-dir /tmp/video-reviews \
--pr-context /tmp/pr-context.txt
```
## Research Phase Details (`qa-agent.ts`)
Claude receives:
- The issue description and comments
- A QA guide from `qa-analyze-pr.ts`
- An accessibility tree snapshot of the current UI
Claude's tools:
- **`inspect(selector?)`** — Read a11y tree to discover element selectors
- **`writeTest(code)`** — Write a Playwright `.spec.ts` file
- **`runTest()`** — Execute the test and get pass/fail + errors
- **`done(verdict, summary, evidence, testCode)`** — Finish with verdict
The test uses the project's Playwright fixtures (`comfyPageFixture`), giving access to `comfyPage.page`, `comfyPage.menu`, `comfyPage.settings`, etc.
### Verdict Logic
- **REPRODUCED** — Test passes (asserting the bug exists) → bug is proven
- **NOT_REPRODUCIBLE** — Claude exhausted attempts, test cannot pass
- **INCONCLUSIVE** — Agent timed out or encountered infrastructure issues
Auto-completion: if a test passed but `done()` was never called, the pipeline auto-completes with REPRODUCED.
## Manual QA (Fallback)
When the automated pipeline isn't suitable (e.g., visual-only bugs, complex multi-step interactions), use **playwright-cli** for manual browser interaction:
```bash
# Install
npm install -g @playwright/cli@latest
# Open browser and navigate
playwright-cli open http://127.0.0.1:8188
# Get element references
playwright-cli snapshot
# Interact
playwright-cli click e1
playwright-cli fill e2 "test text"
playwright-cli press Escape
playwright-cli screenshot --filename=f.png
```
Snapshots return element references (`e1`, `e2`, …). Always run `snapshot` after navigation to refresh refs.
## Manual QA Test Plan
When performing manual QA (either via playwright-cli or the automated pipeline), systematically test each area below.
### Application Load & Routes
| Test | Steps |
| ----------------- | ------------------------------------------------------------ |
| Root route loads | Navigate to `/` — GraphView should render with canvas |
| User select route | Navigate to `/user-select` — user selection UI should appear |
| 404 handling | Navigate to `/nonexistent` — should handle gracefully |
### Canvas & Graph View
| Test | Steps |
| ------------------------- | -------------------------------------------------------------- |
| Canvas renders | The LiteGraph canvas is visible and interactive |
| Pan canvas | Click and drag on empty canvas area |
| Zoom in/out | Use scroll wheel or Alt+=/Alt+- |
| Add node via double-click | Double-click canvas to open search, type "KSampler", select it |
| Delete node | Select a node, press Delete key |
| Connect nodes | Drag from output slot to input slot |
| Copy/Paste | Select nodes, Ctrl+C then Ctrl+V |
| Undo/Redo | Make changes, Ctrl+Z to undo, Ctrl+Y to redo |
| Context menus | Right-click node vs empty canvas — different menus |
### Sidebar Tabs
| Test | Steps |
| ----------------- | ------------------------------------- |
| Workflows tab | Press W — workflows sidebar opens |
| Node Library tab | Press N — node library opens |
| Model Library tab | Press M — model library opens |
| Tab toggle | Press same key again — sidebar closes |
| Search in sidebar | Type in search box — results filter |
### Settings Dialog
| Test | Steps |
| ---------------- | ---------------------------------------------------- |
| Open settings | Press Ctrl+, or click settings button |
| Change a setting | Toggle a boolean setting — it persists after closing |
| Search settings | Type in settings search box — results filter |
| Close settings | Press Escape or click close button |
### Execution & Queue
| Test | Steps |
| -------------- | ----------------------------------------------------- |
| Queue prompt | Load default workflow, click Queue — execution starts |
| Queue progress | Progress indicator shows during execution |
| Interrupt | Press Ctrl+Alt+Enter during execution — interrupts |
## Report Site
Deployed to Cloudflare Pages at `https://comfy-qa.pages.dev/<branch>/`.
Features:
- Light/dark theme
- Seekable video player with preload
- Copy badge button (markdown)
- Date-stamped badges (e.g., `QA0327`)
- Vertical box badge for issues and PRs
## Known Issues & Troubleshooting
See `docs/qa/TROUBLESHOOTING.md` for common failures:
- `set -euo pipefail` + grep with no match → append `|| true`
- `__name is not defined` in `page.evaluate` → use `addScriptTag`
- Cursor not visible in videos → monkey-patch `page.mouse` methods
- Agent not calling `done()` → auto-complete from passing test
## Backlog
See `docs/qa/backlog.md` for planned improvements:
- **Type B comparison**: Different commits for regression detection
- **Type C comparison**: Cross-browser testing
- **Pre-seed assets**: Upload test images before recording
- **Lazy a11y tree**: Reduce token usage with `inspect(selector)` vs full dump

View File

@@ -1,668 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Research Phase — Claude writes & debugs E2E tests to reproduce bugs
*
* Instead of driving a browser interactively, Claude:
* 1. Reads the issue + a11y snapshot of the UI
* 2. Writes a Playwright E2E test (.spec.ts) that reproduces the bug
* 3. Runs the test → reads errors → rewrites → repeats until it works
* 4. Outputs the passing test + verdict
*
* Tools:
* - inspect(selector) — read a11y tree to understand UI state
* - writeTest(code) — write a Playwright test file
* - runTest() — execute the test and get results
* - done(verdict, summary, testCode) — finish with the working test
*/
import type { Page } from '@playwright/test'
/* eslint-disable import-x/no-unresolved */
// @ts-expect-error — claude-agent-sdk has no type declarations for vue-tsc
import { query, tool, createSdkMcpServer } from '@anthropic-ai/claude-agent-sdk'
/* eslint-enable import-x/no-unresolved */
import { z } from 'zod'
import { mkdirSync, readFileSync, writeFileSync } from 'fs'
import { execSync } from 'child_process'
// ── Types ──
interface ResearchOptions {
page: Page
issueContext: string
qaGuide: string
outputDir: string
serverUrl: string
anthropicApiKey?: string
maxTurns?: number
timeBudgetMs?: number
}
export type ReproMethod = 'e2e_test' | 'video' | 'both' | 'none'
export interface ResearchResult {
verdict: 'REPRODUCED' | 'NOT_REPRODUCIBLE' | 'INCONCLUSIVE'
reproducedBy: ReproMethod
summary: string
evidence: string
testCode: string
videoScript?: string
log: Array<{
turn: number
timestampMs: number
toolName: string
toolInput: unknown
toolResult: string
}>
}
// ── Main research function ──
export async function runResearchPhase(
opts: ResearchOptions
): Promise<ResearchResult> {
const { page, issueContext, qaGuide, outputDir, serverUrl, anthropicApiKey } =
opts
const maxTurns = opts.maxTurns ?? 50
let agentDone = false
let finalVerdict: ResearchResult['verdict'] = 'INCONCLUSIVE'
let finalReproducedBy: ReproMethod = 'none'
let finalSummary = 'Agent did not complete'
let finalEvidence = ''
let finalTestCode = ''
let finalVideoScript = ''
let turnCount = 0
let lastPassedTurn = -1
const startTime = Date.now()
const researchLog: ResearchResult['log'] = []
const testDir = `${outputDir}/research`
mkdirSync(testDir, { recursive: true })
const testPath = `${testDir}/reproduce.spec.ts`
// Get initial a11y snapshot for context
let initialA11y = ''
try {
initialA11y = await page.locator('body').ariaSnapshot({ timeout: 5000 })
initialA11y = initialA11y.slice(0, 3000)
} catch {
initialA11y = '(could not capture initial a11y snapshot)'
}
// ── Tool: inspect ──
const inspectTool = tool(
'inspect',
'Read the current accessibility tree to understand UI state. Use this to discover element names, roles, and selectors for your test.',
{
selector: z
.string()
.optional()
.describe(
'Optional filter — only show elements matching this name/role. Omit for full tree.'
)
},
async (args: { selector?: string }) => {
let resultText: string
try {
const ariaText = await page
.locator('body')
.ariaSnapshot({ timeout: 5000 })
if (args.selector) {
const lines = ariaText.split('\n')
const matches = lines.filter((l: string) =>
l.toLowerCase().includes(args.selector!.toLowerCase())
)
resultText =
matches.length > 0
? `Found "${args.selector}":\n${matches.slice(0, 15).join('\n')}`
: `"${args.selector}" not found. Full tree:\n${ariaText.slice(0, 2000)}`
} else {
resultText = ariaText.slice(0, 3000)
}
} catch (e) {
resultText = `inspect failed: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'inspect',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readFixture ──
const readFixtureTool = tool(
'readFixture',
'Read a fixture or helper file from browser_tests/fixtures/ to understand the API. Use this to discover available methods on comfyPage helpers before writing your test.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/fixtures/, e.g. "helpers/CanvasHelper.ts" or "components/Topbar.ts" or "ComfyPage.ts"'
)
},
async (args: { path: string }) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/fixtures/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
resultText = `Could not read fixture: ${e instanceof Error ? e.message : e}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readFixture',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: readTest ──
const readTestTool = tool(
'readTest',
'Read an existing E2E test file from browser_tests/tests/ to learn patterns and conventions used in this project.',
{
path: z
.string()
.describe(
'Relative path within browser_tests/tests/, e.g. "workflow.spec.ts" or "subgraph.spec.ts"'
)
},
async (args: { path: string }) => {
let resultText: string
try {
const fullPath = `${projectRoot}/browser_tests/tests/${args.path}`
const content = readFileSync(fullPath, 'utf-8')
resultText = content.slice(0, 4000)
if (content.length > 4000) {
resultText += `\n\n... (truncated, ${content.length} total chars)`
}
} catch (e) {
// List available test files if the path doesn't exist
try {
const { readdirSync } = await import('fs')
const files = readdirSync(`${projectRoot}/browser_tests/tests/`)
.filter((f: string) => f.endsWith('.spec.ts'))
.slice(0, 30)
resultText = `File not found: ${args.path}\n\nAvailable test files:\n${files.join('\n')}`
} catch {
resultText = `Could not read test: ${e instanceof Error ? e.message : e}`
}
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'readTest',
toolInput: args,
toolResult: resultText.slice(0, 500)
})
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: writeTest ──
const writeTestTool = tool(
'writeTest',
'Write a Playwright E2E test file that reproduces the bug. The test should assert the broken behavior exists.',
{
code: z
.string()
.describe('Complete Playwright test file content (.spec.ts)')
},
async (args: { code: string }) => {
writeFileSync(testPath, args.code)
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'writeTest',
toolInput: { path: testPath, codeLength: args.code.length },
toolResult: `Test written to ${testPath} (${args.code.length} chars)`
})
return {
content: [
{
type: 'text' as const,
text: `Test written to ${testPath}. Use runTest() to execute it.`
}
]
}
}
)
// ── Tool: runTest ──
// Place test in browser_tests/ so Playwright config finds fixtures
const projectRoot = process.cwd()
const browserTestPath = `${projectRoot}/browser_tests/tests/qa-reproduce.spec.ts`
const runTestTool = tool(
'runTest',
'Run the Playwright test and get results. Returns stdout/stderr including assertion errors.',
{},
async () => {
turnCount++
// Copy the test to browser_tests/tests/ where Playwright expects it
const { copyFileSync } = await import('fs')
try {
copyFileSync(testPath, browserTestPath)
} catch {
// directory may not exist
mkdirSync(`${projectRoot}/browser_tests/tests`, { recursive: true })
copyFileSync(testPath, browserTestPath)
}
let resultText: string
try {
const output = execSync(
`cd "${projectRoot}" && npx playwright test browser_tests/tests/qa-reproduce.spec.ts --reporter=list --timeout=30000 --retries=0 --workers=1 2>&1`,
{
timeout: 90000,
encoding: 'utf-8',
env: {
...process.env,
COMFYUI_BASE_URL: serverUrl
}
}
)
resultText = `TEST PASSED:\n${output.slice(-1500)}`
} catch (e) {
const err = e as { stdout?: string; stderr?: string; message?: string }
const output = (err.stdout || '') + '\n' + (err.stderr || '')
resultText = `TEST FAILED:\n${output.slice(-2000)}`
}
researchLog.push({
turn: turnCount,
timestampMs: Date.now() - startTime,
toolName: 'runTest',
toolInput: { testPath },
toolResult: resultText.slice(0, 1000)
})
// Auto-save passing test code for fallback completion — but only if
// the test contains a bug-specific assertion (not just a discovery/debug test)
if (resultText.startsWith('TEST PASSED')) {
try {
const code = readFileSync(browserTestPath, 'utf-8')
const hasBugAssertion =
/expect\s*\(/.test(code) &&
!/^\s*expect\([^)]+\)\.toBeDefined\(\)/m.test(code) &&
!/^\s*expect\([^)]+\)\.toBeGreaterThan\(0\)/m.test(code) &&
!/Inspect|Find|Debug|discover/i.test(
code.match(/test\(['"`]([^'"`]+)/)?.[1] ?? ''
)
if (hasBugAssertion) {
finalTestCode = code
lastPassedTurn = turnCount
}
} catch {
// ignore
}
resultText +=
'\n\n⚠ Test PASSED — call done() now with verdict REPRODUCED and the test code. Do NOT write more tests.'
}
return { content: [{ type: 'text' as const, text: resultText }] }
}
)
// ── Tool: done ──
const doneTool = tool(
'done',
'Finish research with verdict and the final test code.',
{
verdict: z.enum(['REPRODUCED', 'NOT_REPRODUCIBLE', 'INCONCLUSIVE']),
reproducedBy: z
.enum(['e2e_test', 'video', 'both', 'none'])
.describe(
'How the bug was proven: e2e_test = Playwright assertion passed, video = visual evidence only, both = both methods, none = not reproduced'
),
summary: z.string().describe('What you found and why'),
evidence: z.string().describe('Test output that proves the verdict'),
testCode: z
.string()
.describe(
'Final Playwright test code. If REPRODUCED, this test asserts the bug exists and passes.'
),
videoScript: z
.string()
.optional()
.describe(
'Demowright video script for Phase 2 demo recording. REQUIRED when verdict is REPRODUCED. A separate test file using createVideoScript with title, segments, and outro. Do NOT include demowright imports in testCode.'
)
},
async (args: {
verdict: ResearchResult['verdict']
reproducedBy: ReproMethod
summary: string
evidence: string
testCode: string
videoScript?: string
}) => {
agentDone = true
finalVerdict = args.verdict
finalReproducedBy = args.reproducedBy
finalSummary = args.summary
finalEvidence = args.evidence
finalTestCode = args.testCode
finalVideoScript = args.videoScript ?? ''
writeFileSync(testPath, args.testCode)
if (args.videoScript) {
writeFileSync(`${outputDir}/video-script.spec.ts`, args.videoScript)
}
return {
content: [
{ type: 'text' as const, text: `Research complete: ${args.verdict}` }
]
}
}
)
// ── MCP Server ──
const server = createSdkMcpServer({
name: 'qa-research',
version: '1.0.0',
tools: [
inspectTool,
readFixtureTool,
readTestTool,
writeTestTool,
runTestTool,
doneTool
]
})
// ── System prompt ──
const systemPrompt = `You are a senior QA engineer who writes Playwright E2E tests to reproduce reported bugs.
## Your tools
- inspect(selector?) — Read the accessibility tree to understand the current UI. Use to discover selectors, element names, and UI state.
- readFixture(path) — Read fixture source code from browser_tests/fixtures/. Use to discover available methods. E.g. "helpers/CanvasHelper.ts", "components/Topbar.ts", "ComfyPage.ts"
- readTest(path) — Read an existing test from browser_tests/tests/ to learn patterns. E.g. "workflow.spec.ts". Pass any name to list available files.
- writeTest(code) — Write a Playwright test file (.spec.ts)
- runTest() — Execute the test and get results (pass/fail + errors)
- done(verdict, summary, evidence, testCode) — Finish with the final test
## Workflow
1. Read the issue description carefully
2. Use inspect() to understand the current UI state and discover element selectors
3. If unsure about the fixture API, use readFixture() to read the relevant helper source code
4. If unsure about test patterns, use readTest() to read an existing test for reference
5. Write a Playwright test that:
- Performs the exact reproduction steps from the issue
- Asserts the BROKEN behavior (the bug) — so the test PASSES when the bug exists
6. Run the test with runTest()
7. If it fails: read the error, fix the test, run again (max 5 attempts)
8. Call done() with the final verdict and test code
## Test writing guidelines
- Import the project fixture: \`import { comfyPageFixture as test } from '../fixtures/ComfyPage'\`
- Import expect: \`import { expect } from '@playwright/test'\`
- The fixture provides \`comfyPage\` which has all the helpers listed below
- If the bug IS present, the test should PASS. If the bug is fixed, the test would FAIL.
- Keep tests focused and minimal — test ONLY the reported bug
- Write ONE test, not multiple. Focus on the single clearest reproduction.
- The test file will be placed in browser_tests/tests/qa-reproduce.spec.ts
- Use \`comfyPage.nextFrame()\` after interactions that trigger UI updates
- NEVER use \`page.waitForTimeout()\` — use Locator actions and retrying assertions instead
- ALWAYS call done() when finished, even if the test passed — do not keep iterating after a passing test
- Use \`expect.poll()\` for async assertions: \`await expect.poll(() => comfyPage.nodeOps.getGraphNodesCount()).toBe(8)\`
- CRITICAL: Your assertions must be SPECIFIC TO THE BUG. A test that asserts \`expect(count).toBeGreaterThan(0)\` proves nothing — it would pass even without the bug. Instead assert the exact broken state, e.g. \`expect(clonedWidgets).toHaveLength(0)\` (missing widgets) or \`expect(zIndex).toBeLessThan(parentZIndex)\` (wrong z-order). If a test passes trivially, it's a false positive.
- NEVER write "debug", "discovery", or "inspect node types" tests. These waste turns and produce false REPRODUCED verdicts. If you need to discover node type names, use inspect() or readFixture() — not a passing test.
- If you cannot write a bug-specific assertion, call done() with verdict NOT_REPRODUCIBLE and explain why.
## ComfyPage Fixture API Reference
### Core properties
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.canvas\` — Locator for #graph-canvas
- \`comfyPage.queueButton\` — "Queue Prompt" button
- \`comfyPage.runButton\` — "Run" button (new UI)
- \`comfyPage.confirmDialog\` — ConfirmDialog (has .confirm, .delete, .overwrite, .reject locators + .click(name) method)
- \`comfyPage.nextFrame()\` — wait for next requestAnimationFrame
- \`comfyPage.setup()\` — navigate + wait for app ready (called automatically by fixture)
### Menu (comfyPage.menu)
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.triggerTopbarCommand(['File', 'Save As'])\` — navigate menu hierarchy
- \`.openTopbarMenu()\` / \`.closeTopbarMenu()\` — open/close hamburger
- \`.openSubmenu('File')\` — hover to open submenu, returns submenu Locator
- \`.getTabNames()\` — get all open workflow tab names
- \`.getActiveTabName()\` — get active tab name
- \`.getWorkflowTab(name)\` — get tab Locator
- \`.closeWorkflowTab(name)\` — close a tab
- \`.saveWorkflow(name)\` / \`.saveWorkflowAs(name)\` / \`.exportWorkflow(name)\`
- \`.switchTheme('dark' | 'light')\`
- \`comfyPage.menu.workflowsTab\` — WorkflowsSidebarTab:
- \`.open()\` / \`.close()\` — toggle workflows sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflow names
- \`comfyPage.menu.nodeLibraryTab\` — NodeLibrarySidebarTab
- \`comfyPage.menu.assetsTab\` — AssetsSidebarTab
### Canvas (comfyPage.canvasOps)
- \`.click({x, y})\` — click at position on canvas
- \`.rightClick(x, y)\` — right-click (opens context menu)
- \`.doubleClick()\` — double-click canvas (opens node search)
- \`.clickEmptySpace()\` — click known empty area
- \`.dragAndDrop(source, target)\` — drag from source to target position
- \`.pan(offset, safeSpot?)\` — pan canvas by offset
- \`.zoom(deltaY, steps?)\` — zoom via scroll wheel
- \`.resetView()\` — reset zoom/pan to default
- \`.getScale()\` / \`.setScale(n)\` — get/set canvas zoom
- \`.getNodeCenterByTitle(title)\` — get screen coords of node center
- \`.disconnectEdge()\` / \`.connectEdge()\` — default graph edge operations
### Node Operations (comfyPage.nodeOps)
- \`.getGraphNodesCount()\` — count all nodes
- \`.getSelectedGraphNodesCount()\` — count selected nodes
- \`.getNodes()\` — get all nodes
- \`.getFirstNodeRef()\` — get NodeReference for first node
- \`.getNodeRefById(id)\` — get NodeReference by ID
- \`.getNodeRefsByType(type)\` — get all nodes of a type
- \`.waitForGraphNodes(count)\` — wait until node count matches
### Settings (comfyPage.settings)
- \`.setSetting(id, value)\` — change a ComfyUI setting
- \`.getSetting(id)\` — read current setting value
### Keyboard (comfyPage.keyboard)
- \`.undo()\` / \`.redo()\` — Ctrl+Z / Ctrl+Y
- \`.bypass()\` — Ctrl+B
- \`.selectAll()\` — Ctrl+A
- \`.ctrlSend(key)\` — send Ctrl+key
### Workflow (comfyPage.workflow)
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\`
- \`.isCurrentWorkflowModified()\` — check dirty state
### Context Menu (comfyPage.contextMenu)
- \`.openFor(locator)\` — right-click locator and wait for menu
- \`.clickMenuItem(name)\` — click a menu item by name
- \`.isVisible()\` — check if context menu is showing
- \`.assertHasItems(items)\` — assert menu contains items
### Other helpers
- \`comfyPage.settingDialog\` — SettingDialog component
- \`comfyPage.searchBox\` / \`comfyPage.searchBoxV2\` — node search
- \`comfyPage.toast\` — ToastHelper (\`.visibleToasts\`)
- \`comfyPage.subgraph\` — SubgraphHelper
- \`comfyPage.vueNodes\` — VueNodeHelpers
- \`comfyPage.bottomPanel\` — BottomPanel
- \`comfyPage.clipboard\` — ClipboardHelper
- \`comfyPage.dragDrop\` — DragDropHelper
### Available fixture files (use readFixture to explore)
- ComfyPage.ts — main fixture with all helpers
- helpers/CanvasHelper.ts, NodeOperationsHelper.ts, WorkflowHelper.ts
- helpers/KeyboardHelper.ts, SettingsHelper.ts, SubgraphHelper.ts
- components/Topbar.ts, ContextMenu.ts, SettingDialog.ts, SidebarTab.ts
## Video Script (IMPORTANT — provide via done() tool)
When calling done(), provide a \`videoScript\` parameter with a SEPARATE test file that uses demowright's createVideoScript.
Do NOT put demowright imports in testCode — they won't resolve in Phase 1.
The videoScript is a complete, standalone Playwright test file for Phase 2 demo recording:
\`\`\`typescript
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
import { createVideoScript } from 'demowright/video-script'
test('Demo: Bug Title', async ({ comfyPage }) => {
// IMPORTANT: ALL setup code MUST go here BEFORE createVideoScript()
// so the title card is the FIRST thing viewers see in the video
await comfyPage.settings.setSetting('Comfy.UseNewMenu', 'Top')
await comfyPage.workflow.setupWorkflowsDirectory({})
const script = createVideoScript()
.title('Bug Title Here', { subtitle: 'Issue #NNNN', durationMs: 4000 })
.segment('Step 1: description of what we do', async (pace) => {
await pace() // narration finishes FIRST
await comfyPage.menu.topbar.saveWorkflow('name') // THEN action
await comfyPage.page.waitForTimeout(2000) // pause for viewer
})
.segment('Bug evidence: what we see proves the bug', async (pace) => {
await pace()
await comfyPage.page.waitForTimeout(5000) // hold on evidence
})
.outro({ text: 'Bug Reproduced', subtitle: 'Summary' })
await script.render(comfyPage.page)
})
\`\`\`
Key API:
- \`.title(text, {subtitle?, durationMs?})\` — title card (4s default)
- \`.segment(narrationText, async (pace) => { await pace(); ...actions... })\` — TTS narrated step
- \`.outro({text?, subtitle?, durationMs?})\` — ending card
- \`pace()\` — wait for narration audio to finish
CRITICAL TIMING: Call \`await pace()\` FIRST in each segment, BEFORE the Playwright actions.
This makes the narration play and finish, THEN the actions execute — so viewers hear what's about
to happen before it happens. Pattern:
\`\`\`typescript
.segment('Now we save the workflow as a new name', async (pace) => {
await pace() // narration finishes first
await comfyPage.menu.topbar.saveWorkflowAs('new-name') // then action happens
await comfyPage.page.waitForTimeout(2000) // pause so viewer sees the result
})
\`\`\`
IMPORTANT RULES for videoScript:
1. You MUST provide videoScript when verdict is REPRODUCED — every reproduced bug needs a narrated demo
2. ALL setup code (setSetting, setupWorkflowsDirectory) goes BEFORE createVideoScript() — title card must be first thing in video
3. Call \`await pace()\` FIRST in each segment callback, BEFORE actions
4. Add \`waitForTimeout(2000)\` after each action so viewers can see the result
5. Final evidence segment: hold for 5+ seconds
6. Reproduce the same steps as testCode but slower with clear narration
## Current UI state (accessibility tree)
${initialA11y}
${qaGuide ? `## QA Analysis Guide\n${qaGuide}\n` : ''}
## Issue to Reproduce
${issueContext}`
// ── Run the agent ──
console.warn('Starting research phase (Claude writes E2E tests)...')
try {
for await (const message of query({
prompt:
'Write a Playwright E2E test that reproduces the reported bug. Use inspect() to discover selectors, readFixture() or readTest() if you need to understand the fixture API or see existing test patterns, writeTest() to write the test, runTest() to execute it. Iterate until it works or you determine the bug cannot be reproduced.',
options: {
model: 'claude-sonnet-4-6',
systemPrompt,
...(anthropicApiKey ? { apiKey: anthropicApiKey } : {}),
maxTurns,
mcpServers: { 'qa-research': server },
allowedTools: [
'mcp__qa-research__inspect',
'mcp__qa-research__readFixture',
'mcp__qa-research__readTest',
'mcp__qa-research__writeTest',
'mcp__qa-research__runTest',
'mcp__qa-research__done'
]
}
})) {
if (message.type === 'assistant' && message.message?.content) {
for (const block of message.message.content) {
if ('text' in block && block.text) {
console.warn(` Claude: ${block.text.slice(0, 200)}`)
}
if ('name' in block) {
console.warn(
` Tool: ${block.name}(${JSON.stringify(block.input).slice(0, 100)})`
)
}
}
}
if (agentDone) break
}
} catch (e) {
const errMsg = e instanceof Error ? e.message : String(e)
console.warn(`Research error: ${errMsg}`)
// Detect billing/auth errors and surface them clearly
if (
errMsg.includes('Credit balance is too low') ||
errMsg.includes('insufficient_quota') ||
errMsg.includes('rate_limit')
) {
finalSummary = `API error: ${errMsg.slice(0, 200)}`
finalEvidence = 'Agent could not start due to API billing/auth issue'
console.warn(
'::error::Anthropic API credits exhausted — cannot run research phase'
)
}
}
// Auto-complete: if a test passed but done() was never called, use the passing test
if (!agentDone && lastPassedTurn >= 0 && finalTestCode) {
console.warn(
`Auto-completing: test passed at turn ${lastPassedTurn} but done() was not called`
)
finalVerdict = 'REPRODUCED'
finalReproducedBy = 'e2e_test'
finalSummary = `Test passed at turn ${lastPassedTurn} (auto-completed — agent did not call done())`
finalEvidence = `Test passed with exit code 0`
}
const result: ResearchResult = {
verdict: finalVerdict,
reproducedBy: finalReproducedBy,
summary: finalSummary,
evidence: finalEvidence,
testCode: finalTestCode,
videoScript: finalVideoScript || undefined,
log: researchLog
}
writeFileSync(`${testDir}/research-log.json`, JSON.stringify(result, null, 2))
console.warn(
`Research complete: ${finalVerdict} (${researchLog.length} tool calls)`
)
return result
}

View File

@@ -1,84 +0,0 @@
import { describe, expect, it } from 'vitest'
import { extractMediaUrls } from './qa-analyze-pr'
describe('extractMediaUrls', () => {
it('extracts markdown image URLs', () => {
const text = '![screenshot](https://example.com/image.png)'
expect(extractMediaUrls(text)).toEqual(['https://example.com/image.png'])
})
it('extracts multiple markdown images', () => {
const text = [
'![before](https://example.com/before.png)',
'Some text',
'![after](https://example.com/after.jpg)'
].join('\n')
expect(extractMediaUrls(text)).toEqual([
'https://example.com/before.png',
'https://example.com/after.jpg'
])
})
it('extracts raw URLs with media extensions', () => {
const text = 'Check this: https://cdn.example.com/demo.mp4 for details'
expect(extractMediaUrls(text)).toEqual(['https://cdn.example.com/demo.mp4'])
})
it('extracts GitHub user-attachments URLs', () => {
const text =
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
expect(extractMediaUrls(text)).toEqual([
'https://github.com/user-attachments/assets/abc12345-6789-0def-1234-567890abcdef'
])
})
it('extracts private-user-images URLs', () => {
const text =
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
expect(extractMediaUrls(text)).toEqual([
'https://private-user-images.githubusercontent.com/12345/abcdef-1234?jwt=token123'
])
})
it('extracts URLs with query parameters', () => {
const text = 'https://example.com/image.png?w=800&h=600'
expect(extractMediaUrls(text)).toEqual([
'https://example.com/image.png?w=800&h=600'
])
})
it('deduplicates URLs', () => {
const text = [
'![img](https://example.com/same.png)',
'![img2](https://example.com/same.png)',
'Also https://example.com/same.png'
].join('\n')
expect(extractMediaUrls(text)).toEqual(['https://example.com/same.png'])
})
it('returns empty array for empty input', () => {
expect(extractMediaUrls('')).toEqual([])
})
it('returns empty array for text with no media URLs', () => {
expect(extractMediaUrls('Just some text without any URLs')).toEqual([])
})
it('handles mixed media types', () => {
const text = [
'![screen](https://example.com/screenshot.png)',
'Video: https://example.com/demo.webm',
'![gif](https://example.com/animation.gif)'
].join('\n')
const urls = extractMediaUrls(text)
expect(urls).toContain('https://example.com/screenshot.png')
expect(urls).toContain('https://example.com/demo.webm')
expect(urls).toContain('https://example.com/animation.gif')
})
it('ignores non-http URLs in markdown', () => {
const text = '![local](./local-image.png)'
expect(extractMediaUrls(text)).toEqual([])
})
})

View File

@@ -1,799 +0,0 @@
#!/usr/bin/env tsx
/**
* QA PR Analysis Script
*
* Deeply analyzes a PR using Gemini Pro to generate targeted QA guides
* for before/after recording sessions. Fetches PR thread, extracts media,
* and produces structured test plans.
*
* Usage:
* pnpm exec tsx scripts/qa-analyze-pr.ts \
* --pr-number 10270 \
* --repo owner/repo \
* --output-dir qa-guides/ \
* [--model gemini-3.1-pro-preview]
*
* Env: GEMINI_API_KEY (required)
*/
import { execSync } from 'node:child_process'
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
// ── Types ──
interface QaGuideStep {
action: string
description: string
expected_before?: string
expected_after?: string
}
interface QaGuide {
summary: string
test_focus: string
prerequisites: string[]
steps: QaGuideStep[]
visual_checks: string[]
}
interface PrThread {
title: string
body: string
labels: string[]
issueComments: string[]
reviewComments: string[]
reviews: string[]
diff: string
}
type TargetType = 'pr' | 'issue'
interface Options {
prNumber: string
repo: string
outputDir: string
model: string
apiKey: string
mediaBudgetBytes: number
maxVideoBytes: number
type: TargetType
}
// ── CLI parsing ──
function parseArgs(): Options {
const args = process.argv.slice(2)
const opts: Partial<Options> = {
model: 'gemini-3.1-pro-preview',
apiKey: process.env.GEMINI_API_KEY || '',
mediaBudgetBytes: 20 * 1024 * 1024,
maxVideoBytes: 10 * 1024 * 1024,
type: 'pr'
}
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--pr-number':
opts.prNumber = args[++i]
break
case '--repo':
opts.repo = args[++i]
break
case '--output-dir':
opts.outputDir = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--type':
opts.type = args[++i] as TargetType
break
case '--help':
console.warn(
'Usage: qa-analyze-pr.ts --pr-number <num> --repo <owner/repo> --output-dir <path> [--model <model>] [--type pr|issue]'
)
process.exit(0)
}
}
if (!opts.prNumber || !opts.repo || !opts.outputDir) {
console.error(
'Required: --pr-number <num> --repo <owner/repo> --output-dir <path>'
)
process.exit(1)
}
if (!opts.apiKey) {
console.error('GEMINI_API_KEY environment variable is required')
process.exit(1)
}
return opts as Options
}
// ── PR thread fetching ──
function ghExec(cmd: string): string {
try {
return execSync(cmd, {
encoding: 'utf-8',
timeout: 30_000,
stdio: ['pipe', 'pipe', 'pipe']
}).trim()
} catch (err) {
console.warn(`gh command failed: ${cmd}`)
console.warn((err as Error).message)
return ''
}
}
function fetchPrThread(prNumber: string, repo: string): PrThread {
console.warn('Fetching PR thread...')
const prView = ghExec(
`gh pr view ${prNumber} --repo ${repo} --json title,body,labels`
)
const prData = prView
? JSON.parse(prView)
: { title: '', body: '', labels: [] }
const issueCommentsRaw = ghExec(
`gh api repos/${repo}/issues/${prNumber}/comments --paginate`
)
const issueComments: string[] = issueCommentsRaw
? JSON.parse(issueCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewCommentsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/comments --paginate`
)
const reviewComments: string[] = reviewCommentsRaw
? JSON.parse(reviewCommentsRaw).map((c: { body: string }) => c.body)
: []
const reviewsRaw = ghExec(
`gh api repos/${repo}/pulls/${prNumber}/reviews --paginate`
)
const reviews: string[] = reviewsRaw
? JSON.parse(reviewsRaw)
.filter((r: { body: string }) => r.body)
.map((r: { body: string }) => r.body)
: []
const diff = ghExec(`gh pr diff ${prNumber} --repo ${repo}`)
console.warn(
`PR #${prNumber}: "${prData.title}" | ` +
`${issueComments.length} issue comments, ` +
`${reviewComments.length} review comments, ` +
`${reviews.length} reviews, ` +
`diff: ${diff.length} chars`
)
return {
title: prData.title || '',
body: prData.body || '',
labels: (prData.labels || []).map((l: { name: string }) => l.name),
issueComments,
reviewComments,
reviews,
diff
}
}
interface IssueThread {
title: string
body: string
labels: string[]
comments: string[]
}
function fetchIssueThread(issueNumber: string, repo: string): IssueThread {
console.warn('Fetching issue thread...')
const issueView = ghExec(
`gh issue view ${issueNumber} --repo ${repo} --json title,body,labels`
)
const issueData = issueView
? JSON.parse(issueView)
: { title: '', body: '', labels: [] }
const commentsRaw = ghExec(
`gh api repos/${repo}/issues/${issueNumber}/comments --paginate`
)
const comments: string[] = commentsRaw
? JSON.parse(commentsRaw).map((c: { body: string }) => c.body)
: []
console.warn(
`Issue #${issueNumber}: "${issueData.title}" | ` +
`${comments.length} comments`
)
return {
title: issueData.title || '',
body: issueData.body || '',
labels: (issueData.labels || []).map((l: { name: string }) => l.name),
comments
}
}
// ── Media extraction ──
const MEDIA_EXTENSIONS = /\.(png|jpg|jpeg|gif|webp|mp4|webm|mov)$/i
const MEDIA_URL_PATTERNS = [
// Markdown images: ![alt](url)
/!\[[^\]]*\]\(([^)]+)\)/g,
// GitHub user-attachments
/https:\/\/github\.com\/user-attachments\/assets\/[a-f0-9-]+/g,
// Private user images
/https:\/\/private-user-images\.githubusercontent\.com\/[^\s)"]+/g,
// Raw URLs with media extensions (standalone or in text)
/(?<!="|=')https?:\/\/[^\s)<>"]+\.(?:png|jpg|jpeg|gif|webp|mp4|webm|mov)(?:\?[^\s)<>"]*)?/gi
]
export function extractMediaUrls(text: string): string[] {
if (!text) return []
const urls = new Set<string>()
for (const pattern of MEDIA_URL_PATTERNS) {
// Reset lastIndex for global patterns
pattern.lastIndex = 0
let match: RegExpExecArray | null
while ((match = pattern.exec(text)) !== null) {
// For markdown images, the URL is in capture group 1
const url = match[1] || match[0]
// Clean trailing markdown/html artifacts
const cleaned = url.replace(/[)>"'\s]+$/, '')
if (cleaned.startsWith('http')) {
urls.add(cleaned)
}
}
}
return [...urls]
}
// ── Media downloading ──
const ALLOWED_MEDIA_DOMAINS = [
'github.com',
'raw.githubusercontent.com',
'user-images.githubusercontent.com',
'private-user-images.githubusercontent.com',
'objects.githubusercontent.com',
'github.githubassets.com'
]
function isAllowedMediaDomain(url: string): boolean {
try {
const hostname = new URL(url).hostname
return ALLOWED_MEDIA_DOMAINS.some(
(domain) => hostname === domain || hostname.endsWith(`.${domain}`)
)
} catch {
return false
}
}
async function downloadMedia(
urls: string[],
outputDir: string,
budgetBytes: number,
maxVideoBytes: number
): Promise<Array<{ path: string; mimeType: string }>> {
const downloaded: Array<{ path: string; mimeType: string }> = []
let totalBytes = 0
const mediaDir = resolve(outputDir, 'media')
mkdirSync(mediaDir, { recursive: true })
for (const url of urls) {
if (totalBytes >= budgetBytes) {
console.warn(
`Media budget exhausted (${totalBytes} bytes), skipping rest`
)
break
}
if (!isAllowedMediaDomain(url)) {
console.warn(`Skipping non-GitHub URL: ${url.slice(0, 80)}`)
continue
}
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(15_000),
headers: { Accept: 'image/*,video/*' },
redirect: 'follow'
})
if (!response.ok) {
console.warn(`Failed to download ${url}: ${response.status}`)
continue
}
const contentLength = response.headers.get('content-length')
if (contentLength) {
const declaredSize = Number.parseInt(contentLength, 10)
if (declaredSize > budgetBytes - totalBytes) {
console.warn(
`Content-Length ${declaredSize} would exceed budget, skipping ${url}`
)
continue
}
}
const contentType = response.headers.get('content-type') || ''
const buffer = Buffer.from(await response.arrayBuffer())
// Skip oversized videos
const isVideo =
contentType.startsWith('video/') || /\.(mp4|webm|mov)$/i.test(url)
if (isVideo && buffer.length > maxVideoBytes) {
console.warn(
`Skipping large video ${url} (${(buffer.length / 1024 / 1024).toFixed(1)}MB > ${(maxVideoBytes / 1024 / 1024).toFixed(0)}MB cap)`
)
continue
}
if (totalBytes + buffer.length > budgetBytes) {
console.warn(`Would exceed budget, skipping ${url}`)
continue
}
const ext = guessExtension(url, contentType)
const filename = `media-${downloaded.length}${ext}`
const filepath = resolve(mediaDir, filename)
writeFileSync(filepath, buffer)
totalBytes += buffer.length
const mimeType = contentType.split(';')[0].trim() || guessMimeType(ext)
downloaded.push({ path: filepath, mimeType })
console.warn(
`Downloaded: ${url.slice(0, 80)}... (${(buffer.length / 1024).toFixed(0)}KB)`
)
} catch (err) {
console.warn(`Failed to download ${url}: ${(err as Error).message}`)
}
}
console.warn(
`Downloaded ${downloaded.length}/${urls.length} media files ` +
`(${(totalBytes / 1024 / 1024).toFixed(1)}MB)`
)
return downloaded
}
function guessExtension(url: string, contentType: string): string {
const urlMatch = url.match(MEDIA_EXTENSIONS)
if (urlMatch) return urlMatch[0].toLowerCase()
const typeMap: Record<string, string> = {
'image/png': '.png',
'image/jpeg': '.jpg',
'image/gif': '.gif',
'image/webp': '.webp',
'video/mp4': '.mp4',
'video/webm': '.webm'
}
return typeMap[contentType.split(';')[0]] || '.bin'
}
function guessMimeType(ext: string): string {
const map: Record<string, string> = {
'.png': 'image/png',
'.jpg': 'image/jpeg',
'.jpeg': 'image/jpeg',
'.gif': 'image/gif',
'.webp': 'image/webp',
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime'
}
return map[ext] || 'application/octet-stream'
}
// ── Gemini analysis ──
function buildIssueAnalysisPrompt(issue: IssueThread): string {
const allText = [
`# Issue: ${issue.title}`,
'',
'## Description',
issue.body,
'',
issue.comments.length > 0
? `## Comments\n${issue.comments.join('\n\n---\n\n')}`
: ''
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a bug report for ComfyUI frontend — a node-based visual workflow editor for AI image generation (Vue 3 + TypeScript).
The UI has:
- A large canvas (1280x720 viewport) showing a node graph centered at ~(640, 400)
- Nodes are boxes with input/output slots connected by wires
- A hamburger menu (top-left C logo) with File, Edit, Help submenus
- Sidebars (Workflows, Node Library, Models)
- A topbar with workflow tabs and Queue button
- The default workflow loads with these nodes (approximate center coordinates):
- Load Checkpoint (~150, 300), CLIP Text Encode x2 (~450, 250 and ~450, 450)
- Empty Latent Image (~450, 600), KSampler (~750, 350), VAE Decode (~1000, 350), Save Image (~1200, 350)
- Right-clicking ON a node shows node actions (Clone, Bypass, Convert, etc.)
- Right-clicking on EMPTY canvas shows Add Node menu — different from node context menu
Your task: Generate a DETAILED reproduction guide (8-15 steps) to trigger this bug on main.
${allText}
## Available test actions
Each step must use one of these actions:
### Menu actions
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
### Element actions (by visible text)
- "click" — clicks an element by visible text (text required)
- "rightClick" — right-clicks an element to open context menu (text required)
- "doubleClick" — double-clicks an element or coordinates (text or x,y)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required: Escape, Tab, Delete, Enter, etc.)
### Canvas actions (by coordinates — viewport is 1280x720)
- "clickCanvas" — click at coordinates (x, y required)
- "rightClickCanvas" — right-click at coordinates (x, y required)
- "doubleClick" — double-click at coordinates to open node search (x, y)
- "dragCanvas" — drag from one point to another (fromX, fromY, toX, toY)
- "scrollCanvas" — scroll wheel for zoom (x, y, deltaY: negative=zoom in, positive=zoom out)
### Utility
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Common ComfyUI interactions
- Right-click a node → context menu with Clone, Bypass, Remove, Colors, etc.
- Double-click empty canvas → opens node search dialog
- Ctrl+C / Ctrl+V → copy/paste selected nodes
- Delete key → remove selected node
- Ctrl+G → group selected nodes
- Drag from output slot to input slot → create connection
- Click a node to select it, Shift+click for multi-select
## Output format
Return a JSON object with exactly one key: "reproduce", containing:
{
"summary": "One sentence: what bug this issue reports",
"test_focus": "Specific behavior to reproduce",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "clickCanvas",
"description": "Click on first node to select it",
"expected_before": "What should happen if the bug is present"
}
],
"visual_checks": ["Specific visual evidence of the bug to look for"]
}
## Rules
- Generate 8-15 DETAILED steps that actually trigger the reported bug.
- Follow the issue's reproduction steps PRECISELY — translate them into available actions.
- Use canvas coordinates for node interactions (nodes are typically in the center area 300-900 x 200-500).
- Take screenshots BEFORE and AFTER critical actions to capture the bug state.
- Do NOT just open a menu and screenshot — actually perform the full reproduction sequence.
- Do NOT include login steps.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
function buildAnalysisPrompt(thread: PrThread): string {
const allText = [
`# PR: ${thread.title}`,
'',
'## Description',
thread.body,
'',
thread.issueComments.length > 0
? `## Issue Comments\n${thread.issueComments.join('\n\n---\n\n')}`
: '',
thread.reviewComments.length > 0
? `## Review Comments\n${thread.reviewComments.join('\n\n---\n\n')}`
: '',
thread.reviews.length > 0
? `## Reviews\n${thread.reviews.join('\n\n---\n\n')}`
: '',
'',
'## Diff (truncated)',
'```',
thread.diff.slice(0, 8000),
'```'
]
.filter(Boolean)
.join('\n')
return `You are a senior QA engineer analyzing a pull request for ComfyUI frontend (a Vue 3 + TypeScript web application for AI image generation workflows).
Your task: Generate TWO targeted QA test guides — one for BEFORE the PR (main branch) and one for AFTER (PR branch).
${allText}
## Available test actions
Each step must use one of these actions:
- "openMenu" — clicks the Comfy hamburger menu (top-left C logo)
- "hoverMenuItem" — hovers a top-level menu item to open submenu (label required)
- "clickMenuItem" — clicks an item in the visible submenu (label required)
- "fillDialog" — fills dialog input and presses Enter (text required)
- "pressKey" — presses a keyboard key (key required)
- "click" — clicks an element by visible text (text required)
- "wait" — waits briefly (ms required, max 3000)
- "screenshot" — takes a screenshot (name required)
## Output format
Return a JSON object with exactly two keys: "before" and "after", each containing:
{
"summary": "One sentence: what this PR changes",
"test_focus": "Specific behaviors to verify in this recording",
"prerequisites": ["e.g. Load default workflow"],
"steps": [
{
"action": "openMenu",
"description": "Open the main menu to check file options",
"expected_before": "Old behavior description (before key only)",
"expected_after": "New behavior description (after key only)"
}
],
"visual_checks": ["Specific visual elements to look for"]
}
## Rules
- BEFORE guide: 2-4 steps, under 15 seconds. Show OLD/missing behavior.
- AFTER guide: 3-6 steps, under 30 seconds. Prove the fix/feature works.
- Focus on the SPECIFIC behavior changed by this PR, not generic testing.
- Use information from PR description, screenshots, and comments to understand intended behavior.
- Include at least one screenshot step in each guide.
- Do NOT include login steps.
- Menu pattern: openMenu -> hoverMenuItem -> clickMenuItem or screenshot.
- Output ONLY valid JSON, no markdown fences or explanation.`
}
async function analyzeWithGemini(
thread: PrThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<{ before: QaGuide; after: QaGuide }> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildAnalysisPrompt(thread)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
// Add media as inline data
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
// Strip markdown fences if present
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
// Handle different response shapes from Gemini
let before: QaGuide
let after: QaGuide
if (Array.isArray(parsed) && parsed.length >= 2) {
// Array format: [before, after]
before = parsed[0]
after = parsed[1]
} else if (parsed.before && parsed.after) {
// Object format: { before, after }
before = parsed.before
after = parsed.after
} else {
// Try nested wrapper keys
const inner = parsed.qa_guide ?? parsed.guides ?? parsed
if (inner.before && inner.after) {
before = inner.before
after = inner.after
} else {
console.warn(
'Full response:',
JSON.stringify(parsed, null, 2).slice(0, 2000)
)
throw new Error(
`Unexpected response shape. Got keys: ${Object.keys(parsed).join(', ')}`
)
}
}
return { before, after }
}
async function analyzeIssueWithGemini(
issue: IssueThread,
media: Array<{ path: string; mimeType: string }>,
model: string,
apiKey: string
): Promise<QaGuide> {
const genAI = new GoogleGenerativeAI(apiKey)
const geminiModel = genAI.getGenerativeModel({ model })
const prompt = buildIssueAnalysisPrompt(issue)
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
for (const item of media) {
try {
const buffer = readFileSync(item.path)
parts.push({
inlineData: {
mimeType: item.mimeType,
data: buffer.toString('base64')
}
})
} catch (err) {
console.warn(
`Failed to read media ${item.path}: ${(err as Error).message}`
)
}
}
console.warn(
`Sending to ${model}: ${prompt.length} chars text, ${media.length} media files`
)
const result = await geminiModel.generateContent({
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192,
responseMimeType: 'application/json'
}
})
let text = result.response.text()
text = text
.replace(/^```(?:json)?\n?/gm, '')
.replace(/```$/gm, '')
.trim()
console.warn('Gemini response received')
console.warn('Raw response (first 500 chars):', text.slice(0, 500))
const parsed = JSON.parse(text)
const guide: QaGuide =
parsed.reproduce ?? parsed.qa_guide?.reproduce ?? parsed
return guide
}
// ── Main ──
async function main() {
const opts = parseArgs()
mkdirSync(opts.outputDir, { recursive: true })
if (opts.type === 'issue') {
await analyzeIssue(opts)
} else {
await analyzePr(opts)
}
}
async function analyzeIssue(opts: Options) {
const issue = fetchIssueThread(opts.prNumber, opts.repo)
const allText = [issue.body, ...issue.comments].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guide = await analyzeIssueWithGemini(
issue,
media,
opts.model,
opts.apiKey
)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
writeFileSync(beforePath, JSON.stringify(guide, null, 2))
console.warn(`Wrote QA guide:`)
console.warn(` Reproduce: ${beforePath}`)
}
async function analyzePr(opts: Options) {
const thread = fetchPrThread(opts.prNumber, opts.repo)
const allText = [
thread.body,
...thread.issueComments,
...thread.reviewComments,
...thread.reviews
].join('\n')
const mediaUrls = extractMediaUrls(allText)
console.warn(`Found ${mediaUrls.length} media URLs`)
const media = await downloadMedia(
mediaUrls,
opts.outputDir,
opts.mediaBudgetBytes,
opts.maxVideoBytes
)
const guides = await analyzeWithGemini(thread, media, opts.model, opts.apiKey)
const beforePath = resolve(opts.outputDir, 'qa-guide-before.json')
const afterPath = resolve(opts.outputDir, 'qa-guide-after.json')
writeFileSync(beforePath, JSON.stringify(guides.before, null, 2))
writeFileSync(afterPath, JSON.stringify(guides.after, null, 2))
console.warn(`Wrote QA guides:`)
console.warn(` Before: ${beforePath}`)
console.warn(` After: ${afterPath}`)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
if (isExecutedAsScript(import.meta.url)) {
main().catch((err) => {
console.error('PR analysis failed:', err)
process.exit(1)
})
}

View File

@@ -1,413 +0,0 @@
#!/usr/bin/env bash
# Deploy QA report to Cloudflare Pages.
# Expected env vars: CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, RAW_BRANCH,
# BEFORE_SHA, AFTER_SHA, TARGET_NUM, TARGET_TYPE, REPO, RUN_ID
# Writes outputs to GITHUB_OUTPUT: badge_status, url
set -euo pipefail
npm install -g wrangler@4.74.0 >/dev/null 2>&1
DEPLOY_DIR=$(mktemp -d)
mkdir -p "$DEPLOY_DIR"
for os in Linux macOS Windows; do
DIR="qa-artifacts/qa-report-${os}-${RUN_ID}"
for prefix in qa qa-before; do
VID="${DIR}/${prefix}-session.mp4"
if [ -f "$VID" ]; then
DEST="$DEPLOY_DIR/${prefix}-${os}.mp4"
cp "$VID" "$DEST"
echo "Found ${prefix} ${os} video ($(du -h "$VID" | cut -f1))"
fi
done
# Copy multi-pass session videos (qa-session-1, qa-session-2, etc.)
for numbered in "$DIR"/qa-session-[0-9].mp4; do
[ -f "$numbered" ] || continue
NUM=$(basename "$numbered" | sed 's/qa-session-\([0-9]\).mp4/\1/')
DEST="$DEPLOY_DIR/qa-${os}-pass${NUM}.mp4"
cp "$numbered" "$DEST"
echo "Found pass ${NUM} ${os} video ($(du -h "$numbered" | cut -f1))"
done
# Generate GIF thumbnail from after video (or first pass)
THUMB_SRC="$DEPLOY_DIR/qa-${os}.mp4"
[ ! -f "$THUMB_SRC" ] && THUMB_SRC="$DEPLOY_DIR/qa-${os}-pass1.mp4"
if [ -f "$THUMB_SRC" ]; then
ffmpeg -y -ss 10 -i "$THUMB_SRC" -t 8 \
-vf "fps=8,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=64[p];[s1][p]paletteuse=dither=bayer" \
-loop 0 "$DEPLOY_DIR/qa-${os}-thumb.gif" 2>/dev/null \
|| echo "GIF generation failed for ${os} (non-fatal)"
fi
done
# Build video cards and report sections
CARDS=""
# shellcheck disable=SC2034 # accessed via eval
ICONS_Linux="&#x1F427;" ICONS_macOS="&#x1F34E;" ICONS_Windows="&#x1FA9F;"
CARD_COUNT=0
DL_ICON="<svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4'/><polyline points='7 10 12 15 17 10'/><line x1=12 y1=15 x2=12 y2=3'/></svg>"
for os in Linux macOS Windows; do
eval "ICON=\$ICONS_${os}"
OS_LOWER=$(echo "$os" | tr '[:upper:]' '[:lower:]')
HAS_BEFORE=$([ -f "$DEPLOY_DIR/qa-before-${os}.mp4" ] && echo 1 || echo 0)
HAS_AFTER=$( { [ -f "$DEPLOY_DIR/qa-${os}.mp4" ] || [ -f "$DEPLOY_DIR/qa-${os}-pass1.mp4" ]; } && echo 1 || echo 0)
[ "$HAS_AFTER" = "0" ] && continue
# Collect all reports for this platform (single + multi-pass)
REPORT_FILES=""
REPORT_LINK=""
REPORT_HTML=""
for rpt in "video-reviews/${OS_LOWER}-qa-video-report.md" "video-reviews/${OS_LOWER}-pass"*-qa-video-report.md; do
[ -f "$rpt" ] && REPORT_FILES="${REPORT_FILES} ${rpt}"
done
if [ -n "$REPORT_FILES" ]; then
# Concatenate all reports into one combined report file
COMBINED_MD=""
for rpt in $REPORT_FILES; do
cp "$rpt" "$DEPLOY_DIR/$(basename "$rpt")"
RPT_MD=$(sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' "$rpt")
[ -n "$COMBINED_MD" ] && COMBINED_MD="${COMBINED_MD}&#10;&#10;---&#10;&#10;"
COMBINED_MD="${COMBINED_MD}${RPT_MD}"
done
FIRST_REPORT=$(echo "$REPORT_FILES" | awk '{print $1}')
FIRST_BASENAME=$(basename "$FIRST_REPORT")
REPORT_LINK="<a class=dl href=${FIRST_BASENAME}><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><path d='M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z'/><polyline points='14 2 14 8 20 8'/><line x1=16 y1=13 x2=8 y2=13/><line x1=16 y1=17 x2=8 y2=17'/></svg>Report</a>"
REPORT_HTML="<details class=report open><summary><svg width=14 height=14 viewBox='0 0 24 24' fill=none stroke=currentColor stroke-width=2><circle cx=12 cy=12 r=10/><line x1=12 y1=16 x2=12 y2=12/><line x1=12 y1=8 x2=12.01 y2=8'/></svg> AI Comparative Review</summary><div class=report-body data-md>${COMBINED_MD}</div></details>"
fi
if [ "$HAS_BEFORE" = "1" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison><div class=comp-panel><div class=comp-label>Before <span class=comp-tag>main</span></div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-before-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-before-${os}.mp4 download>${DL_ICON}Before</a></div></div><div class=comp-panel><div class=comp-label>After <span class=comp-tag>PR</span></div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}.mp4 download>${DL_ICON}After</a></div></div></div>${REPORT_HTML}</div>"
elif [ -f "$DEPLOY_DIR/qa-${os}.mp4" ]; then
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}.mp4 type=video/mp4></video></div><div class=card-body><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links><a class=dl href=qa-${os}.mp4 download>${DL_ICON}Download</a>${REPORT_LINK}</span></div>${REPORT_HTML}</div>"
else
PASS_VIDEOS=""
for pass_vid in "$DEPLOY_DIR/qa-${os}-pass"[0-9].mp4; do
[ -f "$pass_vid" ] || continue
PASS_NUM=$(basename "$pass_vid" | sed "s/qa-${os}-pass\([0-9]\).mp4/\1/")
PASS_VIDEOS="${PASS_VIDEOS}<div class=comp-panel><div class=comp-label>Pass ${PASS_NUM}</div><div class=video-wrap><video controls autoplay preload=auto><source src=qa-${os}-pass${PASS_NUM}.mp4 type=video/mp4></video></div><div class=comp-dl><a class=dl href=qa-${os}-pass${PASS_NUM}.mp4 download>${DL_ICON}Pass ${PASS_NUM}</a></div></div>"
done
CARDS="${CARDS}<div class='card reveal' style='--i:${CARD_COUNT}'><div class=card-header><span class=platform><span class=icon>${ICON}</span>${os}</span><span class=links>${REPORT_LINK}</span></div><div class=comparison>${PASS_VIDEOS}</div>${REPORT_HTML}</div>"
fi
CARD_COUNT=$((CARD_COUNT + 1))
done
# Build commit info and target link for the report header
COMMIT_HTML=""
REPO_URL="https://github.com/${REPO}"
if [ -n "${TARGET_NUM:-}" ]; then
if [ "$TARGET_TYPE" = "issue" ]; then
COMMIT_HTML="<a href=${REPO_URL}/issues/${TARGET_NUM} class=sha title='Issue'>Issue #${TARGET_NUM}</a>"
else
COMMIT_HTML="<a href=${REPO_URL}/pull/${TARGET_NUM} class=sha title='Pull Request'>PR #${TARGET_NUM}</a>"
fi
fi
if [ -n "${BEFORE_SHA:-}" ]; then
SHORT_BEFORE="${BEFORE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${BEFORE_SHA} class=sha title='main branch'>main @ ${SHORT_BEFORE}</a>"
fi
if [ -n "${AFTER_SHA:-}" ]; then
SHORT_AFTER="${AFTER_SHA:0:7}"
AFTER_LABEL="PR"
[ -n "${TARGET_NUM:-}" ] && AFTER_LABEL="#${TARGET_NUM}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${AFTER_SHA} class=sha title='PR head commit'>${AFTER_LABEL} @ ${SHORT_AFTER}</a>"
fi
if [ -n "${PIPELINE_SHA:-}" ]; then
SHORT_PIPE="${PIPELINE_SHA:0:7}"
COMMIT_HTML="${COMMIT_HTML:+${COMMIT_HTML} &middot; }<a href=${REPO_URL}/commit/${PIPELINE_SHA} class=sha title='QA pipeline version'>QA @ ${SHORT_PIPE}</a>"
fi
[ -n "$COMMIT_HTML" ] && COMMIT_HTML=" &middot; ${COMMIT_HTML}"
RUN_LINK=""
if [ -n "${RUN_URL:-}" ]; then
RUN_LINK=" &middot; <a href=\"${RUN_URL}\" class=sha title=\"GitHub Actions run\">CI Job</a>"
fi
# Timing info
DEPLOY_TIME=$(date -u '+%Y-%m-%d %H:%M UTC')
TIMING_HTML=""
if [ -n "${RUN_START_TIME:-}" ]; then
TIMING_HTML=" &middot; <span class=sha title='Pipeline timing'>${RUN_START_TIME} &rarr; ${DEPLOY_TIME}</span>"
fi
# Generate index.html from template
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
TEMPLATE="$SCRIPT_DIR/qa-report-template.html"
# Write dynamic content to temp files for safe substitution
# Cloudflare Pages _headers file — enable range requests for video seeking
cat > "$DEPLOY_DIR/_headers" <<'HEADERSEOF'
/*.mp4
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
HEADERSEOF
# Build purpose description from pr-context.txt
PURPOSE_HTML=""
if [ -f pr-context.txt ]; then
# Extract title line and first paragraph of description
PR_TITLE=$(grep -m1 '^Title:' pr-context.txt 2>/dev/null | sed 's/^Title: //' || true)
if [ "$TARGET_TYPE" = "issue" ]; then
PURPOSE_LABEL="Issue #${TARGET_NUM}"
PURPOSE_VERB="reports"
else
PURPOSE_LABEL="PR #${TARGET_NUM}"
PURPOSE_VERB="aims to"
fi
# Get first ~300 chars of description body (after "Description:" line)
PR_DESC=$(sed -n '/^Description:/,/^###/p' pr-context.txt 2>/dev/null | grep -v '^Description:\|^###' | head -5 | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
[ -z "$PR_DESC" ] && PR_DESC=$(sed -n '3,8p' pr-context.txt 2>/dev/null | sed 's/&/\&amp;/g; s/</\&lt;/g; s/>/\&gt;/g' | tr '\n' ' ' | head -c 400 || true)
# Build requirements from QA guide JSON
REQS_HTML=""
QA_GUIDE=$(ls qa-guides/qa-guide-*.json 2>/dev/null | head -1 || true)
if [ -f "$QA_GUIDE" ]; then
PREREQS=$(python3 -c "
import json, sys, html
try:
g = json.load(open(sys.argv[1]))
prereqs = g.get('prerequisites', [])
steps = g.get('steps', [])
focus = g.get('test_focus', '')
parts = []
if focus:
parts.append('<strong>Test focus:</strong> ' + html.escape(focus))
if prereqs:
parts.append('<strong>Prerequisites:</strong> ' + ', '.join(html.escape(p) for p in prereqs))
if steps:
parts.append('<strong>Steps:</strong> ' + ' → '.join(html.escape(s.get('description', str(s))) for s in steps[:6]))
if len(steps) > 6:
parts[-1] += ' → ...'
print('<br>'.join(parts))
except: pass
" "$QA_GUIDE" 2>/dev/null)
[ -n "$PREREQS" ] && REQS_HTML="<div class=purpose-reqs>${PREREQS}</div>"
fi
PURPOSE_HTML="<div class=purpose><div class=purpose-label>${PURPOSE_LABEL} ${PURPOSE_VERB}</div><strong>${PR_TITLE}</strong><br>${PR_DESC}${REQS_HTML}</div>"
fi
echo -n "$COMMIT_HTML" > "$DEPLOY_DIR/.commit_html"
echo -n "$CARDS" > "$DEPLOY_DIR/.cards_html"
echo -n "$RUN_LINK" > "$DEPLOY_DIR/.run_link"
# Badge HTML with copy button (placeholder URL filled after deploy)
echo -n '<div class="badge-bar"><img src="badge.svg" alt="QA Badge" class="badge-img"/><button class="copy-badge" title="Copy badge markdown" onclick="copyBadge()"><svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></div>' > "$DEPLOY_DIR/.badge_html"
echo -n "${TIMING_HTML:-}" > "$DEPLOY_DIR/.timing_html"
echo -n "$PURPOSE_HTML" > "$DEPLOY_DIR/.purpose_html"
python3 -c "
import sys, pathlib
d = pathlib.Path(sys.argv[1])
t = pathlib.Path(sys.argv[2]).read_text()
t = t.replace('{{COMMIT_HTML}}', (d / '.commit_html').read_text())
t = t.replace('{{CARDS}}', (d / '.cards_html').read_text())
t = t.replace('{{RUN_LINK}}', (d / '.run_link').read_text())
t = t.replace('{{BADGE_HTML}}', (d / '.badge_html').read_text())
t = t.replace('{{TIMING_HTML}}', (d / '.timing_html').read_text())
t = t.replace('{{PURPOSE_HTML}}', (d / '.purpose_html').read_text())
sys.stdout.write(t)
" "$DEPLOY_DIR" "$TEMPLATE" > "$DEPLOY_DIR/index.html"
rm -f "$DEPLOY_DIR/.commit_html" "$DEPLOY_DIR/.cards_html" "$DEPLOY_DIR/.run_link" "$DEPLOY_DIR/.badge_html" "$DEPLOY_DIR/.timing_html" "$DEPLOY_DIR/.purpose_html"
cat > "$DEPLOY_DIR/404.html" <<'ERROREOF'
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><title>404</title>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&display=swap" rel=stylesheet>
<style>:root{--bg:oklch(8% 0.02 265);--fg:oklch(45% 0.01 265);--err:oklch(62% 0.22 25)}*{margin:0;padding:0;box-sizing:border-box}body{background:var(--bg);color:var(--fg);font-family:'Inter',system-ui,sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}div{text-align:center}h1{color:var(--err);font-size:clamp(3rem,8vw,5rem);font-weight:700;letter-spacing:-.04em;margin-bottom:.5rem}p{font-size:1rem;max-width:32ch;line-height:1.5}</style>
</head><body><div><h1>404</h1><p>File not found. The QA recording may have failed or been cancelled.</p></div></body></html>
ERROREOF
# Copy research log to deploy dir if it exists
for rlog in qa-artifacts/*/research/research-log.json qa-artifacts/*/*/research/research-log.json qa-artifacts/before/*/research/research-log.json; do
if [ -f "$rlog" ]; then
cp "$rlog" "$DEPLOY_DIR/research-log.json"
echo "Found research log: $rlog"
break
fi
done
# Copy generated test code to deploy dir
for tfile in qa-artifacts/*/research/reproduce.spec.ts qa-artifacts/*/*/research/reproduce.spec.ts qa-artifacts/before/*/research/reproduce.spec.ts; do
if [ -f "$tfile" ]; then
cp "$tfile" "$DEPLOY_DIR/reproduce.spec.ts"
echo "Found test code: $tfile"
break
fi
done
# Copy video script if available
for vsfile in qa-artifacts/*/video-script.spec.ts qa-artifacts/*/*/video-script.spec.ts qa-artifacts/before/*/video-script.spec.ts; do
if [ -f "$vsfile" ]; then
cp "$vsfile" "$DEPLOY_DIR/video-script.spec.ts"
echo "Found video script: $vsfile"
break
fi
done
# Generate badge SVGs into deploy dir
# Priority: research-log.json verdict (a11y-verified) > video review verdict (AI interpretation)
REPRO_COUNT=0 INCONC_COUNT=0 NOT_REPRO_COUNT=0 TOTAL_REPORTS=0
# Try research log first (ground truth from a11y assertions)
RESEARCH_VERDICT=""
REPRO_METHOD=""
if [ -f "$DEPLOY_DIR/research-log.json" ]; then
RESEARCH_VERDICT=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('verdict',''))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
REPRO_METHOD=$(python3 -c "import json,sys; d=json.load(open(sys.argv[1])); print(d.get('reproducedBy','none'))" "$DEPLOY_DIR/research-log.json" 2>/dev/null || true)
echo "Research verdict (a11y-verified): ${RESEARCH_VERDICT:-none} (by: ${REPRO_METHOD:-none})"
if [ -n "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=1
case "$RESEARCH_VERDICT" in
REPRODUCED) REPRO_COUNT=1 ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=1 ;;
INCONCLUSIVE) INCONC_COUNT=1 ;;
esac
fi
fi
# Check video review verdicts (always, not just as fallback)
VIDEO_REPRODUCED=false
if [ -d video-reviews ]; then
for rpt in video-reviews/*-qa-video-report.md; do
[ -f "$rpt" ] || continue
VERDICT_JSON=$(grep -oP '"verdict":\s*"[A-Z_]+' "$rpt" 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
if [ -n "$VERDICT_JSON" ]; then
echo "Video review verdict: $VERDICT_JSON ($(basename "$rpt"))"
[ "$VERDICT_JSON" = "REPRODUCED" ] && VIDEO_REPRODUCED=true
# Only count video as separate report if no research log
if [ -z "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
case "$VERDICT_JSON" in
REPRODUCED) REPRO_COUNT=$((REPRO_COUNT + 1)) ;;
NOT_REPRODUCIBLE) NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1)) ;;
INCONCLUSIVE) INCONC_COUNT=$((INCONC_COUNT + 1)) ;;
esac
fi
elif [ -z "$RESEARCH_VERDICT" ]; then
TOTAL_REPORTS=$((TOTAL_REPORTS + 1))
# Fallback: grep Summary section (for older reports without ## Verdict)
SUMM=$(sed -n '/^## Summary/,/^## /p' "$rpt" 2>/dev/null | head -15)
if echo "$SUMM" | grep -iq 'INCONCLUSIVE'; then
INCONC_COUNT=$((INCONC_COUNT + 1))
elif echo "$SUMM" | grep -iq 'not reproduced\|could not reproduce\|could not be confirmed\|unable to reproduce\|fails\? to reproduce\|fails\? to perform\|was NOT\|NOT visible\|not observed\|fail.* to demonstrate\|does not demonstrate\|steps were not performed\|never.*tested\|never.*accessed\|not.* confirmed'; then
NOT_REPRO_COUNT=$((NOT_REPRO_COUNT + 1))
elif echo "$SUMM" | grep -iq 'reproduc\|confirm'; then
REPRO_COUNT=$((REPRO_COUNT + 1))
VIDEO_REPRODUCED=true
fi
fi
done
fi
# Upgrade reproduction method to "both" when E2E and video agree
if [ "$REPRO_METHOD" = "e2e_test" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
REPRO_METHOD="both"
echo "Upgraded reproducedBy to 'both' (E2E + video review agree)"
elif [ -z "$RESEARCH_VERDICT" ] && [ "$VIDEO_REPRODUCED" = "true" ]; then
REPRO_METHOD="video"
fi
FAIL_COUNT=$((TOTAL_REPORTS - REPRO_COUNT - NOT_REPRO_COUNT))
[ "$FAIL_COUNT" -lt 0 ] && FAIL_COUNT=0
echo "DEBUG verdict: repro=${REPRO_COUNT} not_repro=${NOT_REPRO_COUNT} inconc=${INCONC_COUNT} fail=${FAIL_COUNT} total=${TOTAL_REPORTS}"
# Warn on verdict mismatch between E2E and video review
if [ -n "$RESEARCH_VERDICT" ]; then
VIDEO_VERDICT=$(grep -oP '"verdict":\s*"[A-Z_]+' video-reviews/*-qa-video-report.md 2>/dev/null | tail -1 | grep -oP '[A-Z_]+$' || true)
if [ -n "$VIDEO_VERDICT" ] && [ "$RESEARCH_VERDICT" != "$VIDEO_VERDICT" ]; then
echo "⚠ Verdict mismatch: E2E=$RESEARCH_VERDICT vs Video=$VIDEO_VERDICT (E2E takes priority)"
fi
fi
echo "Verdict: ${REPRO_COUNT}${NOT_REPRO_COUNT}${FAIL_COUNT}⚠ / ${TOTAL_REPORTS}"
# Badge text:
# Single pass: "REPRODUCED" / "NOT REPRODUCIBLE" / "INCONCLUSIVE"
# Multi pass: "2✓ 0✗ 1⚠ / 3" with color based on dominant result
REPRO_RESULT="" REPRO_COLOR="#9f9f9f"
if [ "$TOTAL_REPORTS" -le 1 ]; then
# Single report — simple label
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="REPRODUCED" REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_RESULT="NOT REPRODUCIBLE" REPRO_COLOR="#9f9f9f"
elif [ "$FAIL_COUNT" -gt 0 ]; then
REPRO_RESULT="INCONCLUSIVE" REPRO_COLOR="#9f9f9f"
fi
else
# Multi pass — show breakdown: X✓ Y✗ Z⚠ / N
PARTS=""
[ "$REPRO_COUNT" -gt 0 ] && PARTS="${REPRO_COUNT}"
[ "$NOT_REPRO_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${NOT_REPRO_COUNT}"
[ "$FAIL_COUNT" -gt 0 ] && PARTS="${PARTS:+${PARTS} }${FAIL_COUNT}"
REPRO_RESULT="${PARTS} / ${TOTAL_REPORTS}"
# Color based on best outcome
if [ "$REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#2196f3"
elif [ "$NOT_REPRO_COUNT" -gt 0 ]; then
REPRO_COLOR="#9f9f9f"
fi
fi
# Badge label: #NUM QA0327 (with today's date)
QA_DATE=$(date -u '+%m%d')
BADGE_LABEL="QA${QA_DATE}"
[ -n "${TARGET_NUM:-}" ] && BADGE_LABEL="#${TARGET_NUM} QA${QA_DATE}"
# For PRs, also extract fix quality from Overall Risk section
FIX_RESULT="" FIX_COLOR="#4c1"
if [ "$TARGET_TYPE" != "issue" ]; then
# Try structured JSON risk first
ALL_RISKS=$(grep -ohP '"risk":\s*"[a-z]+' video-reviews/*.md 2>/dev/null | grep -oP '[a-z]+$' || true)
if [ -n "$ALL_RISKS" ]; then
# Use worst risk across all reports
if echo "$ALL_RISKS" | grep -q 'high'; then
FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44"
elif echo "$ALL_RISKS" | grep -q 'medium'; then
FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317"
elif echo "$ALL_RISKS" | grep -q 'low'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
else
# Fallback: grep Overall Risk section
RISK_TEXT=""
if [ -d video-reviews ]; then
RISK_TEXT=$(sed -n '/^## Overall Risk/,/^## /p' video-reviews/*.md 2>/dev/null | sed 's/\*//g' | head -20 || true)
fi
RISK_FIRST=$(echo "$RISK_TEXT" | grep -oiP '^\s*(high|medium|moderate|low|minimal|critical)' | head -1 | tr '[:upper:]' '[:lower:]' || true)
if [ -n "$RISK_FIRST" ]; then
case "$RISK_FIRST" in
*low*|*minimal*) FIX_RESULT="APPROVED" FIX_COLOR="#4c1" ;;
*medium*|*moderate*) FIX_RESULT="MINOR ISSUES" FIX_COLOR="#dfb317" ;;
*high*|*critical*) FIX_RESULT="MAJOR ISSUES" FIX_COLOR="#e05d44" ;;
esac
elif echo "$RISK_TEXT" | grep -iq 'no.*risk\|approved\|looks good'; then
FIX_RESULT="APPROVED" FIX_COLOR="#4c1"
fi
fi
fi
# Always use vertical box badge
/tmp/gen-badge-box.sh "$DEPLOY_DIR/badge.svg" "$BADGE_LABEL" \
"$REPRO_COUNT" "$NOT_REPRO_COUNT" "$FAIL_COUNT" "$TOTAL_REPORTS" \
"$FIX_RESULT" "$FIX_COLOR" "$REPRO_METHOD"
BADGE_STATUS="${REPRO_RESULT:-UNKNOWN}${FIX_RESULT:+ | Fix: ${FIX_RESULT}}"
echo "badge_status=${BADGE_STATUS:-FINISHED}" >> "$GITHUB_OUTPUT"
# Remove files exceeding Cloudflare Pages 25MB limit to prevent silent deploy failures
MAX_SIZE=$((25 * 1024 * 1024))
find "$DEPLOY_DIR" -type f -size +${MAX_SIZE}c | while read -r big_file; do
SIZE_MB=$(( $(stat -c%s "$big_file") / 1024 / 1024 ))
echo "Removing oversized file: $(basename "$big_file") (${SIZE_MB}MB > 25MB limit)"
rm "$big_file"
done
BRANCH=$(echo "$RAW_BRANCH" | sed 's/[^a-zA-Z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-//;s/-$//' | cut -c1-28)
DEPLOY_OUTPUT=$(wrangler pages deploy "$DEPLOY_DIR" \
--project-name="comfy-qa" \
--branch="$BRANCH" 2>&1) || true
echo "$DEPLOY_OUTPUT" | tail -5
URL=$(echo "$DEPLOY_OUTPUT" | grep -oE 'https://[a-zA-Z0-9.-]+\.pages\.dev\S*' | head -1 || true)
FALLBACK_URL="https://${BRANCH}.comfy-qa.pages.dev"
echo "url=${URL:-$FALLBACK_URL}" >> "$GITHUB_OUTPUT"
echo "Deployed to: ${URL:-$FALLBACK_URL}"

View File

@@ -1,208 +0,0 @@
#!/usr/bin/env tsx
/**
* Generates a Playwright regression test (.spec.ts) from a QA report + PR diff.
* Uses Gemini to produce a test that asserts UIUX behavior verified during QA.
*
* Usage:
* pnpm exec tsx scripts/qa-generate-test.ts \
* --qa-report <path> QA video review report (markdown)
* --pr-diff <path> PR diff file
* --output <path> Output .spec.ts file path
* --model <name> Gemini model (default: gemini-3-flash-preview)
*/
import { readFile, writeFile } from 'node:fs/promises'
import { basename, resolve } from 'node:path'
import { GoogleGenerativeAI } from '@google/generative-ai'
interface CliOptions {
qaReport: string
prDiff: string
output: string
model: string
}
const DEFAULTS: CliOptions = {
qaReport: '',
prDiff: '',
output: '',
model: 'gemini-3-flash-preview'
}
// ── Fixture API reference for the prompt ────────────────────────────
const FIXTURE_API = `
## ComfyUI Playwright Test Fixture API
Import pattern:
\`\`\`typescript
import { expect } from '@playwright/test'
import { comfyPageFixture as test } from '../fixtures/ComfyPage'
\`\`\`
### Available helpers on \`comfyPage\`:
- \`comfyPage.page\` — raw Playwright Page
- \`comfyPage.menu.topbar\` — Topbar helper:
- \`.getTabNames(): Promise<string[]>\` — get all open tab names
- \`.getActiveTabName(): Promise<string>\` — get active tab name
- \`.saveWorkflow(name)\` — Save via File > Save dialog
- \`.saveWorkflowAs(name)\` — Save via File > Save As dialog
- \`.exportWorkflow(name)\` — Export via File > Export dialog
- \`.triggerTopbarCommand(path: string[])\` — e.g. ['File', 'Save As']
- \`.getWorkflowTab(name)\` — get a tab locator by name
- \`.closeWorkflowTab(name)\` — close a tab
- \`.openTopbarMenu()\` — open the hamburger menu
- \`.openSubmenu(label)\` — hover to open a submenu
- \`comfyPage.menu.workflowsTab\` — Workflows sidebar:
- \`.open()\` / \`.close()\` — toggle sidebar
- \`.getTopLevelSavedWorkflowNames()\` — list saved workflows
- \`.getPersistedItem(name)\` — get a workflow item locator
- \`comfyPage.workflow\` — WorkflowHelper:
- \`.loadWorkflow(name)\` — load from browser_tests/assets/{name}.json
- \`.setupWorkflowsDirectory(structure)\` — setup test directory
- \`.deleteWorkflow(name)\` — delete a workflow
- \`.isCurrentWorkflowModified(): Promise<boolean>\` — check dirty state
- \`.getUndoQueueSize()\` / \`.getRedoQueueSize()\`
- \`comfyPage.settings.setSetting(key, value)\` — change settings
- \`comfyPage.keyboard\` — KeyboardHelper:
- \`.undo()\` / \`.redo()\` / \`.bypass()\`
- \`comfyPage.nodeOps\` — NodeOperationsHelper
- \`comfyPage.canvas\` — CanvasHelper
- \`comfyPage.contextMenu\` — ContextMenu
- \`comfyPage.toast\` — ToastHelper
- \`comfyPage.confirmDialog\` — confirmation dialog
- \`comfyPage.nextFrame()\` — wait for Vue re-render
### Test patterns:
- Use \`test.describe('Name', { tag: '@ui' }, () => { ... })\` for UI tests
- Use \`test.beforeEach\` to set up common state (settings, workflow dir)
- Use \`expect(locator).toHaveScreenshot('name.png')\` for visual assertions
- Use \`expect(locator).toBeVisible()\` / \`.toHaveText()\` for behavioral assertions
- Use \`comfyPage.workflow.setupWorkflowsDirectory({})\` to ensure clean state
`
// ── Prompt builder ──────────────────────────────────────────────────
function buildPrompt(qaReport: string, prDiff: string): string {
return `You are a Playwright test generator for the ComfyUI frontend.
Your task: Generate a single .spec.ts regression test file that asserts the UIUX behavior
described in the QA report below. The test must:
1. Use the ComfyUI Playwright fixture API (documented below)
2. Test UIUX behavior ONLY — element visibility, tab names, dialog states, workflow states
3. NOT test code implementation details
4. Be concise — only test the behavior that the PR changed
5. Follow existing test conventions (see API reference)
${FIXTURE_API}
## QA Video Review Report
${qaReport}
## PR Diff (for context on what changed)
${prDiff.slice(0, 8000)}
## Output Requirements
- Output ONLY the .spec.ts file content — no markdown fences, no explanations
- Start with imports, end with closing brace
- Use descriptive test names that explain the expected behavior
- Add screenshot assertions where visual verification matters
- Keep it focused: 2-5 test cases covering the core behavioral change
- Use \`test.beforeEach\` for common setup (settings, workflow directory)
- Tag the describe block with \`{ tag: '@ui' }\` or \`{ tag: '@workflow' }\` as appropriate
`
}
// ── Gemini call ─────────────────────────────────────────────────────
async function generateTest(
qaReport: string,
prDiff: string,
model: string
): Promise<string> {
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) throw new Error('GEMINI_API_KEY env var required')
const genAI = new GoogleGenerativeAI(apiKey)
const genModel = genAI.getGenerativeModel({ model })
const prompt = buildPrompt(qaReport, prDiff)
console.warn(`Sending prompt to ${model} (${prompt.length} chars)...`)
const result = await genModel.generateContent({
contents: [{ role: 'user', parts: [{ text: prompt }] }],
generationConfig: {
temperature: 0.2,
maxOutputTokens: 8192
}
})
const text = result.response.text()
// Strip markdown fences if model wraps output
return text
.replace(/^```(?:typescript|ts)?\n?/, '')
.replace(/\n?```$/, '')
.trim()
}
// ── CLI ─────────────────────────────────────────────────────────────
function parseArgs(): CliOptions {
const args = process.argv.slice(2)
const opts = { ...DEFAULTS }
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--qa-report':
opts.qaReport = args[++i]
break
case '--pr-diff':
opts.prDiff = args[++i]
break
case '--output':
opts.output = args[++i]
break
case '--model':
opts.model = args[++i]
break
case '--help':
console.warn(`Usage:
pnpm exec tsx scripts/qa-generate-test.ts [options]
Options:
--qa-report <path> QA video review report (markdown) [required]
--pr-diff <path> PR diff file [required]
--output <path> Output .spec.ts path [required]
--model <name> Gemini model (default: gemini-3-flash-preview)`)
process.exit(0)
}
}
if (!opts.qaReport || !opts.prDiff || !opts.output) {
console.error('Missing required args. Run with --help for usage.')
process.exit(1)
}
return opts
}
async function main() {
const opts = parseArgs()
const qaReport = await readFile(resolve(opts.qaReport), 'utf-8')
const prDiff = await readFile(resolve(opts.prDiff), 'utf-8')
console.warn(
`QA report: ${basename(opts.qaReport)} (${qaReport.length} chars)`
)
console.warn(`PR diff: ${basename(opts.prDiff)} (${prDiff.length} chars)`)
const testCode = await generateTest(qaReport, prDiff, opts.model)
const outputPath = resolve(opts.output)
await writeFile(outputPath, testCode + '\n')
console.warn(`Generated test: ${outputPath} (${testCode.length} chars)`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})

File diff suppressed because it is too large Load Diff

View File

@@ -1,158 +0,0 @@
<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>QA Session Recordings</title>
<link rel=preconnect href=https://fonts.googleapis.com><link rel=preconnect href=https://fonts.gstatic.com crossorigin><link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel=stylesheet>
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<style>
:root{--bg:oklch(97% 0.01 265);--surface:oklch(100% 0 0);--surface-up:oklch(94% 0.01 265);--fg:oklch(15% 0.02 265);--fg-muted:oklch(40% 0.01 265);--fg-dim:oklch(55% 0.01 265);--primary:oklch(50% 0.21 265);--primary-up:oklch(45% 0.21 265);--primary-glow:oklch(55% 0.15 265);--ok:oklch(45% 0.18 155);--err:oklch(50% 0.22 25);--border:oklch(85% 0.01 265);--border-faint:oklch(90% 0.01 265);--r:0.75rem;--r-lg:1rem;--ease-out:cubic-bezier(0.22,1,0.36,1);--dur-base:250ms;--dur-slow:500ms;--font:'Inter',system-ui,sans-serif;--font-mono:'JetBrains Mono',monospace}
@media(prefers-color-scheme:dark){:root{--bg:oklch(8% 0.02 265);--surface:oklch(12% 0.02 265);--surface-up:oklch(16% 0.02 265);--fg:oklch(96% 0.01 95);--fg-muted:oklch(65% 0.01 265);--fg-dim:oklch(45% 0.01 265);--primary:oklch(62% 0.21 265);--primary-up:oklch(68% 0.21 265);--primary-glow:oklch(62% 0.15 265);--ok:oklch(62% 0.18 155);--err:oklch(62% 0.22 25);--border:oklch(22% 0.02 265);--border-faint:oklch(15% 0.01 265)}}
*{margin:0;padding:0;box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font-family:var(--font);min-height:100vh;padding:clamp(1.5rem,4vw,3rem) clamp(1rem,3vw,2rem);position:relative}
@media(prefers-color-scheme:dark){body::after{content:'';position:fixed;inset:0;pointer-events:none;opacity:.03;background:url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.85' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E")}}
.container{max-width:1200px;margin:0 auto}
header{display:flex;align-items:center;gap:1rem;margin-bottom:clamp(1.5rem,4vw,3rem);padding-bottom:1.25rem;border-bottom:1px solid var(--border)}
.header-icon{width:36px;height:36px;display:grid;place-items:center;background:linear-gradient(135deg,oklch(100% 0 0/.06),oklch(100% 0 0/.02));backdrop-filter:blur(12px);border:1px solid oklch(100% 0 0/.1);border-radius:var(--r);flex-shrink:0}
.header-icon svg{color:var(--primary)}
h1{font-size:clamp(1.25rem,2.5vw,1.625rem);font-weight:700;letter-spacing:-.03em;background:linear-gradient(135deg,var(--fg),var(--fg-muted));-webkit-background-clip:text;-webkit-text-fill-color:transparent;background-clip:text}
.meta{color:var(--fg-dim);font-size:.8125rem;margin-top:.15rem;letter-spacing:.01em}
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(480px,100%),1fr));gap:1.5rem}
.card{background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg);overflow:hidden;transition:border-color var(--dur-base) var(--ease-out),box-shadow var(--dur-base) var(--ease-out),transform var(--dur-base) var(--ease-out)}
.card:hover{border-color:var(--primary);box-shadow:0 4px 16px oklch(0% 0 0/.1);transform:translateY(-2px)}
.video-wrap{position:relative;background:var(--surface);border-bottom:1px solid var(--border-faint)}
.video-wrap video{width:100%;display:block;aspect-ratio:16/9;object-fit:contain}
.card-body{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between}
.platform{display:flex;align-items:center;gap:.5rem;font-weight:600;font-size:.9375rem;letter-spacing:-.01em}
.icon{font-size:1.125rem}
.links{display:flex;gap:.5rem}
.dl{color:var(--fg-muted);text-decoration:none;font-size:.75rem;font-weight:500;display:inline-flex;align-items:center;gap:.3rem;padding:.25rem .6rem;border-radius:9999px;border:1px solid var(--border);background:oklch(100% 0 0/.03);transition:all var(--dur-base) var(--ease-out)}
.dl:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.08)}
.badge{font-size:.6875rem;font-weight:600;padding:.2rem .625rem;border-radius:9999px;text-transform:uppercase;letter-spacing:.05em}
.card-header{padding:.75rem 1rem;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid var(--border-faint)}
.comparison{display:grid;grid-template-columns:1fr 1fr;gap:0}
.comp-panel{border-right:1px solid var(--border-faint)}
.comp-panel:last-child{border-right:none}
.comp-label{padding:.4rem .75rem;font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);background:var(--surface);display:flex;align-items:center;gap:.4rem}
.comp-tag{font-size:.6rem;padding:.1rem .4rem;border-radius:9999px;font-weight:600}
.comp-panel:first-child .comp-tag{background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border:1px solid var(--border)}
.comp-panel:last-child .comp-tag{background:oklch(62% 0.18 155/.15);color:var(--ok);border:1px solid oklch(62% 0.18 155/.25)}
.comp-dl{padding:.4rem .75rem;display:flex;justify-content:center}
.report{border-top:1px solid var(--border-faint);padding:.75rem 1rem;font-size:.8125rem}
.report summary{cursor:pointer;color:var(--fg-muted);font-weight:500;display:flex;align-items:center;gap:.4rem;user-select:none;transition:color var(--dur-base) var(--ease-out)}
.report summary:hover{color:var(--fg)}
.report summary svg{flex-shrink:0;opacity:.5}
.report[open] summary{margin-bottom:.75rem;padding-bottom:.5rem;border-bottom:1px solid var(--border-faint)}
.report-body{line-height:1.7;color:oklch(80% 0.01 265);overflow-x:auto}
.report-body h1,.report-body h2{margin:1.25rem 0 .5rem;color:var(--fg);font-size:1rem;font-weight:600;letter-spacing:-.02em;border-bottom:1px solid var(--border-faint);padding-bottom:.4rem}
.report-body h3{margin:.75rem 0 .4rem;color:var(--fg);font-size:.875rem;font-weight:600}
.report-body p{margin:.4rem 0}
.report-body ul,.report-body ol{margin:.4rem 0 .4rem 1.5rem}
.report-body li{margin:.25rem 0}
.report-body code{background:var(--surface-up);padding:.125rem .375rem;border-radius:.25rem;font-size:.7rem;font-family:var(--font-mono);border:1px solid var(--border-faint)}
.report-body h3+p>code:first-child{background:oklch(62% 0.22 25/.15);color:var(--err);border-color:oklch(62% 0.22 25/.25)}
.report-body h3+p>code:nth-child(2){background:oklch(62% 0.21 265/.15);color:var(--primary-up);border-color:oklch(62% 0.21 265/.25)}
.report-body h3+p>code:nth-child(3){background:oklch(65% 0.01 265/.15);color:var(--fg-muted);border-color:var(--border)}
.report-body table{width:100%;border-collapse:collapse;margin:.75rem 0;font-size:.75rem;border:1px solid var(--border);border-radius:var(--r);overflow:hidden}
.report-body th,.report-body td{border:1px solid var(--border-faint);padding:.5rem .75rem;text-align:left;vertical-align:top;word-wrap:break-word}
.report-body th{background:var(--surface-up);color:var(--fg);font-weight:600;font-size:.6875rem;text-transform:uppercase;letter-spacing:.05em;position:sticky;top:0;white-space:nowrap}
.report-body tr:nth-child(even){background:color-mix(in oklch,var(--surface) 50%,transparent)}
.report-body tr:hover{background:color-mix(in oklch,var(--surface-up) 50%,transparent)}
.report-body strong{color:var(--fg)}
.report-body hr{border:none;border-top:1px solid var(--border-faint);margin:1rem 0}
@keyframes fade-up{from{opacity:0;transform:translateY(16px)}to{opacity:1;transform:translateY(0)}}
.reveal{animation:fade-up var(--dur-slow) var(--ease-out) both;animation-delay:calc(var(--i,0) * 120ms)}
@media(prefers-reduced-motion:reduce){.reveal{animation:none}}
@media(max-width:480px){.grid{grid-template-columns:1fr}.card-body{flex-wrap:wrap;gap:.5rem}}
.sha{color:var(--primary);text-decoration:none;font-family:var(--font-mono);font-size:.75rem;font-weight:500;padding:.1rem .4rem;border-radius:.25rem;background:oklch(62% 0.21 265/.08);border:1px solid oklch(62% 0.21 265/.15);transition:all var(--dur-base) var(--ease-out)}
.sha:hover{background:oklch(62% 0.21 265/.15);border-color:var(--primary)}
.badge-bar{display:flex;align-items:center;gap:.5rem;margin-bottom:1rem}
.badge-img{height:20px;display:block}
.copy-badge{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);padding:.3rem .4rem;border-radius:var(--r);cursor:pointer;display:inline-flex;align-items:center;transition:all var(--dur-base) var(--ease-out)}
.copy-badge:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.copy-badge.copied{color:var(--ok);border-color:var(--ok)}
.vseek{width:100%;padding:0 .75rem;background:var(--surface);border-top:1px solid var(--border-faint);position:relative;height:24px;display:flex;align-items:center}
.vseek input[type=range]{-webkit-appearance:none;appearance:none;width:100%;height:4px;background:var(--border);border-radius:2px;outline:none;cursor:pointer;position:relative;z-index:2}
.vseek input[type=range]::-webkit-slider-thumb{-webkit-appearance:none;width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg);box-shadow:0 0 4px oklch(0% 0 0/.3)}
.vseek input[type=range]::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:var(--primary);cursor:pointer;border:2px solid var(--bg)}
.vseek .vbuf{position:absolute;left:.75rem;right:.75rem;height:4px;border-radius:2px;pointer-events:none;top:50%;transform:translateY(-50%)}
.vseek .vbuf-bar{height:100%;background:oklch(62% 0.21 265/.25);border-radius:2px;transition:width 200ms linear}
.vctrl{display:flex;align-items:center;gap:.375rem;padding:.5rem .75rem;background:var(--surface);border-top:1px solid var(--border-faint);flex-wrap:wrap}
.vctrl button{background:oklch(100% 0 0/.06);border:1px solid var(--border);color:var(--fg-muted);font-size:.6875rem;font-weight:600;font-family:var(--font-mono);padding:.25rem .5rem;border-radius:.25rem;cursor:pointer;transition:all var(--dur-base) var(--ease-out);white-space:nowrap}
.vctrl button:hover{color:var(--primary-up);border-color:var(--primary);background:oklch(62% 0.21 265/.1)}
.vctrl button.active{color:var(--primary);border-color:var(--primary);background:oklch(62% 0.21 265/.15)}
.vctrl .vtime{font-family:var(--font-mono);font-size:.6875rem;color:var(--fg-dim);min-width:10ch;text-align:center}
.vctrl .vsep{width:1px;height:1rem;background:var(--border);flex-shrink:0}
.vctrl .vhint{font-size:.6rem;color:var(--fg-dim);margin-left:auto}
.purpose{background:linear-gradient(135deg,oklch(100% 0 0/.04),oklch(100% 0 0/.02));border:1px solid oklch(100% 0 0/.08);border-radius:var(--r-lg);padding:1rem 1.25rem;margin-bottom:1.5rem;font-size:.85rem;line-height:1.7;color:oklch(80% 0.01 265)}
.purpose strong{color:var(--fg);font-weight:600}
.purpose .purpose-label{font-size:.7rem;font-weight:600;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-muted);margin-bottom:.4rem}
.purpose .purpose-reqs{margin-top:.75rem;padding-top:.75rem;border-top:1px solid oklch(100% 0 0/.06);font-size:.8rem;color:oklch(70% 0.01 265);line-height:1.8}
</style></head><body><div class=container>
<header><div class=header-icon><svg width=20 height=20 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2 stroke-linecap=round stroke-linejoin=round><polygon points="23 7 16 12 23 17 23 7"/><rect x=1 y=5 width=15 height=14 rx=2 ry=2/></svg></div><div><h1>QA Session Recordings</h1><div class=meta>ComfyUI Frontend &middot; Automated QA{{COMMIT_HTML}}{{RUN_LINK}}{{TIMING_HTML}}</div>{{BADGE_HTML}}</div></header>
{{PURPOSE_HTML}}<div class=grid>{{CARDS}}</div>
<div id=research-section style="margin-top:2rem"></div>
</div><script>
// Load research-log.json and reproduce.spec.ts if available
(async()=>{
const sec=document.getElementById('research-section');
try{
const [logRes,testRes,vsRes]=await Promise.allSettled([fetch('research-log.json'),fetch('reproduce.spec.ts'),fetch('video-script.spec.ts')]);
let html='';
if(logRes.status==='fulfilled'&&logRes.value.ok){
const log=await logRes.value.json();
html+=`<details style="margin-bottom:1.5rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Research Log &mdash; ${log.verdict||'?'} (${log.toolCalls||'?'} tool calls, ${((log.elapsedMs||0)/1000).toFixed(1)}s)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${JSON.stringify(log,null,2)}</pre></div></details>`;
}
if(testRes.status==='fulfilled'&&testRes.value.ok){
const code=await testRes.value.text();
html+=`<details><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">E2E Test Code (reproduce.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${code.replace(/</g,'&lt;').replace(/>/g,'&gt;')}</pre></div></details>`;
}
if(vsRes.status==='fulfilled'&&vsRes.value.ok){
const vsCode=await vsRes.value.text();
html+=`<details style="margin-top:1rem"><summary style="cursor:pointer;font-weight:600;font-size:1rem;padding:.75rem 1rem;background:var(--surface);border:1px solid var(--border);border-radius:var(--r-lg)">Video Script (video-script.spec.ts)</summary><div style="padding:1rem;background:var(--surface);border:1px solid var(--border);border-top:0;border-radius:0 0 var(--r-lg) var(--r-lg);overflow:auto;max-height:600px"><pre style="font-family:var(--font-mono);font-size:.8rem;line-height:1.6;white-space:pre-wrap">${vsCode.replace(/</g,'&lt;').replace(/>/g,'&gt;')}</pre></div></details>`;
}
if(html)sec.innerHTML=html;
}catch(e){console.warn('research load failed',e)}
})();
</script><script>
function copyBadge(){const u=location.href.replace(/\/[^/]*$/,'/');const b=u+'badge.svg';const md='[![QA Badge]('+b+')]('+u+')';navigator.clipboard.writeText(md).then(()=>{const btn=document.querySelector('.copy-badge');btn.classList.add('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><polyline points="20 6 9 17 4 12"/></svg>';setTimeout(()=>{btn.classList.remove('copied');btn.innerHTML='<svg width=14 height=14 viewBox="0 0 24 24" fill=none stroke=currentColor stroke-width=2><rect x=9 y=9 width=13 height=13 rx=2/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>'},2000)})}
document.querySelectorAll('[data-md]').forEach(el=>{const t=el.textContent;el.removeAttribute('data-md');el.innerHTML=marked.parse(t)});
const FPS=30,FT=1/FPS,SPEEDS=[0.1,0.25,0.5,1,1.5,2];
document.querySelectorAll('.video-wrap video').forEach(v=>{
v.playbackRate=0.5;
const c=document.createElement('div');c.className='vctrl';
const btn=(label,fn)=>{const b=document.createElement('button');b.textContent=label;b.onclick=fn;c.appendChild(b);return b};
const sep=()=>{const s=document.createElement('div');s.className='vsep';c.appendChild(s)};
const time=document.createElement('span');time.className='vtime';time.textContent='0:00.000';
btn('\u23EE',()=>{v.currentTime=0});
btn('\u25C0\u25C0',()=>{v.currentTime=Math.max(0,v.currentTime-FT*10)});
btn('\u25C0',()=>{v.pause();v.currentTime=Math.max(0,v.currentTime-FT)});
const playBtn=btn('\u25B6',()=>{v.paused?v.play():v.pause()});
btn('\u25B6\u25B6',()=>{v.pause();v.currentTime+=FT});
btn('\u25B6\u25B6\u25B6',()=>{v.currentTime+=FT*10});
sep();
const spdBtns=SPEEDS.map(s=>{const b=btn(s+'x',()=>{v.playbackRate=s;spdBtns.forEach(x=>x.classList.remove('active'));b.classList.add('active')});if(s===0.5)b.classList.add('active');return b});
sep();c.appendChild(time);
const hint=document.createElement('span');hint.className='vhint';hint.textContent='\u2190\u2192 frame \u2022 space play';c.appendChild(hint);
// Custom seekbar — works even without server range request support
const seekWrap=document.createElement('div');seekWrap.className='vseek';
const seekBar=document.createElement('input');seekBar.type='range';seekBar.min=0;seekBar.max=1000;seekBar.value=0;seekBar.step=1;
const bufWrap=document.createElement('div');bufWrap.className='vbuf';
const bufBar=document.createElement('div');bufBar.className='vbuf-bar';bufBar.style.width='0%';
bufWrap.appendChild(bufBar);seekWrap.appendChild(bufWrap);seekWrap.appendChild(seekBar);
let seeking=false;
seekBar.oninput=()=>{seeking=true;if(v.duration){v.currentTime=v.duration*(seekBar.value/1000)}};
seekBar.onchange=()=>{seeking=false};
v.closest('.video-wrap').after(seekWrap);
seekWrap.after(c);
v.ontimeupdate=()=>{
const m=Math.floor(v.currentTime/60),s=Math.floor(v.currentTime%60),ms=Math.floor((v.currentTime%1)*1000);
time.textContent=m+':'+(s<10?'0':'')+s+'.'+String(ms).padStart(3,'0');
if(!seeking&&v.duration){seekBar.value=Math.round((v.currentTime/v.duration)*1000)}
};
v.onprogress=v.onloadeddata=()=>{if(v.buffered.length&&v.duration){bufBar.style.width=(v.buffered.end(v.buffered.length-1)/v.duration*100)+'%'}};
v.onplay=()=>{playBtn.textContent='\u23F8'};v.onpause=()=>{playBtn.textContent='\u25B6'};
v.parentElement.addEventListener('keydown',e=>{
if(e.key==='ArrowLeft'){e.preventDefault();v.pause();v.currentTime=Math.max(0,v.currentTime-FT)}
if(e.key==='ArrowRight'){e.preventDefault();v.pause();v.currentTime+=FT}
if(e.key===' '){e.preventDefault();v.paused?v.play():v.pause()}
});
v.parentElement.setAttribute('tabindex','0');
});
</script></body></html>

View File

@@ -1,253 +0,0 @@
#!/usr/bin/env tsx
/**
* QA Reproduce Phase — Deterministic replay of research plan with narration
*
* Takes a reproduction plan from the research phase and replays it:
* 1. Execute each action deterministically (no AI decisions)
* 2. Capture a11y snapshot before/after each action
* 3. Gemini describes what visually changed (narration for humans)
* 4. Output: narration-log.json with full evidence chain
*/
import type { Page } from '@playwright/test'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { mkdirSync, writeFileSync } from 'fs'
import type { ActionResult } from './qa-record.js'
// ── Types ──
interface ReproductionStep {
action: Record<string, unknown> & { action: string }
expectedAssertion: string
}
interface NarrationEntry {
step: number
action: string
params: Record<string, unknown>
result: ActionResult
a11yBefore: unknown
a11yAfter: unknown
assertionExpected: string
assertionPassed: boolean
assertionActual: string
geminiNarration: string
timestampMs: number
}
export interface NarrationLog {
entries: NarrationEntry[]
allAssertionsPassed: boolean
}
interface ReproduceOptions {
page: Page
plan: ReproductionStep[]
geminiApiKey: string
outputDir: string
}
// ── A11y helpers ──
interface A11yNode {
role: string
name: string
value?: string
checked?: boolean
disabled?: boolean
expanded?: boolean
children?: A11yNode[]
}
function searchA11y(node: A11yNode | null, selector: string): A11yNode | null {
if (!node) return null
const sel = selector.toLowerCase()
if (
node.name?.toLowerCase().includes(sel) ||
node.role?.toLowerCase().includes(sel)
) {
return node
}
if (node.children) {
for (const child of node.children) {
const found = searchA11y(child, selector)
if (found) return found
}
}
return null
}
function summarizeA11y(node: A11yNode | null): string {
if (!node) return 'null'
const parts = [`role=${node.role}`, `name="${node.name}"`]
if (node.value !== undefined) parts.push(`value="${node.value}"`)
if (node.checked !== undefined) parts.push(`checked=${node.checked}`)
if (node.disabled) parts.push('disabled')
if (node.expanded !== undefined) parts.push(`expanded=${node.expanded}`)
return `{${parts.join(', ')}}`
}
// ── Subtitle overlay ──
async function showSubtitle(page: Page, text: string, step: number) {
const encoded = encodeURIComponent(
text.slice(0, 120).replace(/'/g, "\\'").replace(/\n/g, ' ')
)
await page.addScriptTag({
content: `(function(){
var id='qa-subtitle';
var el=document.getElementById(id);
if(!el){
el=document.createElement('div');
el.id=id;
Object.assign(el.style,{position:'fixed',bottom:'32px',left:'50%',transform:'translateX(-50%)',zIndex:'2147483646',maxWidth:'90%',padding:'6px 14px',borderRadius:'6px',background:'rgba(0,0,0,0.8)',color:'rgba(255,255,255,0.95)',fontSize:'12px',fontFamily:'system-ui,sans-serif',fontWeight:'400',lineHeight:'1.4',pointerEvents:'none',textAlign:'center',whiteSpace:'normal'});
document.body.appendChild(el);
}
el.textContent='['+${step}+'] '+decodeURIComponent('${encoded}');
})()`
})
}
// ── Gemini visual narration ──
async function geminiDescribe(
page: Page,
geminiApiKey: string,
focus: string
): Promise<string> {
try {
const screenshot = await page.screenshot({ type: 'jpeg', quality: 70 })
const genAI = new GoogleGenerativeAI(geminiApiKey)
const model = genAI.getGenerativeModel({ model: 'gemini-3-flash-preview' })
const result = await model.generateContent([
{
text: `Describe in 1-2 sentences what you see on this ComfyUI screen. Focus on: ${focus}. Be factual — only describe what is visible.`
},
{
inlineData: {
mimeType: 'image/jpeg',
data: screenshot.toString('base64')
}
}
])
return result.response.text().trim()
} catch (e) {
return `(Gemini narration failed: ${e instanceof Error ? e.message.slice(0, 50) : e})`
}
}
// ── Main reproduce function ──
export async function runReproducePhase(
opts: ReproduceOptions
): Promise<NarrationLog> {
const { page, plan, geminiApiKey, outputDir } = opts
const { executeAction } = await import('./qa-record.js')
const narrationDir = `${outputDir}/narration`
mkdirSync(narrationDir, { recursive: true })
const entries: NarrationEntry[] = []
const startMs = Date.now()
console.warn(`Reproduce phase: replaying ${plan.length} steps...`)
for (let i = 0; i < plan.length; i++) {
const step = plan[i]
const actionObj = step.action
const elapsed = Date.now() - startMs
// Show subtitle
await showSubtitle(page, `Step ${i + 1}: ${actionObj.action}`, i + 1)
console.warn(` [${i + 1}/${plan.length}] ${actionObj.action}`)
// Capture a11y BEFORE
const a11yBefore = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Execute action
const result = await executeAction(
page,
actionObj as Parameters<typeof executeAction>[1],
outputDir
)
await new Promise((r) => setTimeout(r, 500))
// Capture a11y AFTER
const a11yAfter = await page
.locator('body')
.ariaSnapshot({ timeout: 3000 })
.catch(() => null)
// Check assertion
let assertionPassed = false
let assertionActual = ''
if (step.expectedAssertion) {
// Parse the expected assertion — e.g. "Settings dialog: visible" or "tab count: 2"
const parts = step.expectedAssertion.split(':').map((s) => s.trim())
const selectorName = parts[0]
const expectedState = parts.slice(1).join(':').trim()
const found = searchA11y(a11yAfter as A11yNode | null, selectorName)
assertionActual = found ? summarizeA11y(found) : 'NOT FOUND'
if (expectedState === 'visible' || expectedState === 'exists') {
assertionPassed = found !== null
} else if (expectedState === 'hidden' || expectedState === 'gone') {
assertionPassed = found === null
} else {
// Generic: check if the actual state contains the expected text
assertionPassed = assertionActual
.toLowerCase()
.includes(expectedState.toLowerCase())
}
console.warn(
` Assertion: "${step.expectedAssertion}" → ${assertionPassed ? '✓ PASS' : '✗ FAIL'} (actual: ${assertionActual})`
)
}
// Gemini narration (visual description for humans)
const geminiNarration = await geminiDescribe(
page,
geminiApiKey,
`What changed after ${actionObj.action}?`
)
entries.push({
step: i + 1,
action: actionObj.action,
params: actionObj,
result,
a11yBefore,
a11yAfter,
assertionExpected: step.expectedAssertion,
assertionPassed,
assertionActual,
geminiNarration,
timestampMs: elapsed
})
}
// Final screenshot
await page.screenshot({ path: `${outputDir}/reproduce-final.png` })
const log: NarrationLog = {
entries,
allAssertionsPassed: entries.every((e) => e.assertionPassed)
}
writeFileSync(
`${narrationDir}/narration-log.json`,
JSON.stringify(log, null, 2)
)
console.warn(
`Reproduce phase complete: ${entries.filter((e) => e.assertionPassed).length}/${entries.length} assertions passed`
)
return log
}

View File

@@ -1,150 +0,0 @@
import { describe, expect, it } from 'vitest'
import {
extractPlatformFromArtifactDirName,
pickLatestVideosByPlatform,
selectVideoCandidateByFile
} from './qa-video-review'
describe('extractPlatformFromArtifactDirName', () => {
it('extracts and normalizes known qa artifact directory names', () => {
expect(
extractPlatformFromArtifactDirName('qa-report-Windows-22818315023')
).toBe('windows')
expect(
extractPlatformFromArtifactDirName('qa-report-macOS-22818315023')
).toBe('macos')
expect(
extractPlatformFromArtifactDirName('qa-report-Linux-22818315023')
).toBe('linux')
})
it('falls back to slugifying unknown directory names', () => {
expect(extractPlatformFromArtifactDirName('custom platform run')).toBe(
'custom-platform-run'
)
})
})
describe('pickLatestVideosByPlatform', () => {
it('keeps only the latest candidate per platform', () => {
const selected = pickLatestVideosByPlatform([
{
platformName: 'windows',
videoPath: '/tmp/windows-old.mp4',
mtimeMs: 100
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
},
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
}
])
expect(selected).toEqual([
{
platformName: 'linux',
videoPath: '/tmp/linux.mp4',
mtimeMs: 150
},
{
platformName: 'windows',
videoPath: '/tmp/windows-new.mp4',
mtimeMs: 200
}
])
})
})
describe('selectVideoCandidateByFile', () => {
it('selects a single candidate by artifacts-relative path', () => {
const selected = selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-Linux-1/qa-session.mp4'
}
)
expect(selected).toEqual({
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
})
})
it('throws when basename matches multiple videos', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
},
{
platformName: 'linux',
videoPath: '/tmp/qa-artifacts/qa-report-Linux-1/qa-session.mp4',
mtimeMs: 200
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-session.mp4'
}
)
).toThrow('matched 2 videos')
})
it('throws when there is no matching video', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: 'qa-report-macOS-1/qa-session.mp4'
}
)
).toThrow('No video matched')
})
it('throws when video file is missing', () => {
expect(() =>
selectVideoCandidateByFile(
[
{
platformName: 'windows',
videoPath: '/tmp/qa-artifacts/qa-report-Windows-1/qa-session.mp4',
mtimeMs: 100
}
],
{
artifactsDir: '/tmp/qa-artifacts',
videoFile: ' '
}
)
).toThrow('--video-file is required')
})
})

View File

@@ -1,771 +0,0 @@
#!/usr/bin/env tsx
import { mkdir, readFile, stat, writeFile } from 'node:fs/promises'
import { basename, dirname, extname, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { GoogleGenerativeAI } from '@google/generative-ai'
import { globSync } from 'glob'
interface CliOptions {
artifactsDir: string
videoFile: string
beforeVideo: string
outputDir: string
model: string
requestTimeoutMs: number
dryRun: boolean
prContext: string
targetUrl: string
passLabel: string
}
interface VideoCandidate {
platformName: string
videoPath: string
mtimeMs: number
}
const DEFAULT_OPTIONS: CliOptions = {
artifactsDir: './tmp/qa-artifacts',
videoFile: '',
beforeVideo: '',
outputDir: './tmp',
model: 'gemini-3-flash-preview',
requestTimeoutMs: 300_000,
dryRun: false,
prContext: '',
targetUrl: '',
passLabel: ''
}
const USAGE = `Usage:
pnpm exec tsx scripts/qa-video-review.ts [options]
Options:
--artifacts-dir <path> Artifacts root directory
(default: ./tmp/qa-artifacts)
--video-file <name-or-path> Video file to analyze (required)
(supports basename or relative/absolute path)
--before-video <path> Before video (main branch) for comparison
When provided, sends both videos to Gemini
for comparative before/after analysis
--output-dir <path> Output directory for markdown reports
(default: ./tmp)
--model <name> Gemini model
(default: gemini-3-flash-preview)
--request-timeout-ms <n> Request timeout in milliseconds
(default: 300000)
--pr-context <file> File with PR context (title, body, diff)
for PR-aware review
--target-url <url> Issue or PR URL to include in the report
--pass-label <label> Label for multi-pass reports (e.g. pass1)
Output becomes {platform}-{label}-qa-video-report.md
--dry-run Discover videos and output targets only
--help Show this help text
Environment:
GEMINI_API_KEY Required unless --dry-run
`
function parsePositiveInteger(rawValue: string, flagName: string): number {
const parsedValue = Number.parseInt(rawValue, 10)
if (!Number.isInteger(parsedValue) || parsedValue <= 0) {
throw new Error(`Invalid value for ${flagName}: "${rawValue}"`)
}
return parsedValue
}
function parseCliOptions(args: string[]): CliOptions {
const options: CliOptions = { ...DEFAULT_OPTIONS }
for (let index = 0; index < args.length; index += 1) {
const argument = args[index]
const nextValue = args[index + 1]
const requireValue = (flagName: string): string => {
if (!nextValue || nextValue.startsWith('--')) {
throw new Error(`Missing value for ${flagName}`)
}
index += 1
return nextValue
}
if (argument === '--help') {
process.stdout.write(USAGE)
process.exit(0)
}
if (argument === '--artifacts-dir') {
options.artifactsDir = requireValue(argument)
continue
}
if (argument === '--video-file') {
options.videoFile = requireValue(argument)
continue
}
if (argument === '--output-dir') {
options.outputDir = requireValue(argument)
continue
}
if (argument === '--model') {
options.model = requireValue(argument)
continue
}
if (argument === '--request-timeout-ms') {
options.requestTimeoutMs = parsePositiveInteger(
requireValue(argument),
argument
)
continue
}
if (argument === '--before-video') {
options.beforeVideo = requireValue(argument)
continue
}
if (argument === '--pr-context') {
options.prContext = requireValue(argument)
continue
}
if (argument === '--target-url') {
options.targetUrl = requireValue(argument)
continue
}
if (argument === '--pass-label') {
options.passLabel = requireValue(argument)
continue
}
if (argument === '--dry-run') {
options.dryRun = true
continue
}
throw new Error(`Unknown argument: ${argument}`)
}
return options
}
function normalizePlatformName(value: string): string {
const slug = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
return slug.length > 0 ? slug : 'unknown-platform'
}
export function extractPlatformFromArtifactDirName(dirName: string): string {
const matchedValue = dirName.match(/^qa-report-(.+?)(?:-\d+)?$/i)?.[1]
return normalizePlatformName(matchedValue ?? dirName)
}
function extractPlatformFromVideoPath(videoPath: string): string {
const artifactDirName = basename(dirname(videoPath))
return extractPlatformFromArtifactDirName(artifactDirName)
}
export function pickLatestVideosByPlatform(
candidates: VideoCandidate[]
): VideoCandidate[] {
const latestByPlatform = new Map<string, VideoCandidate>()
for (const candidate of candidates) {
const current = latestByPlatform.get(candidate.platformName)
if (!current || candidate.mtimeMs > current.mtimeMs) {
latestByPlatform.set(candidate.platformName, candidate)
}
}
return [...latestByPlatform.values()].sort((a, b) =>
a.platformName.localeCompare(b.platformName)
)
}
function toProjectRelativePath(targetPath: string): string {
const relativePath = relative(process.cwd(), targetPath)
if (relativePath.startsWith('.')) {
return relativePath
}
return `./${relativePath}`
}
function errorToString(error: unknown): string {
return error instanceof Error ? error.message : String(error)
}
function normalizePathForMatch(value: string): string {
return value.replaceAll('\\', '/').replace(/^\.\/+/, '')
}
export function selectVideoCandidateByFile(
candidates: VideoCandidate[],
options: { artifactsDir: string; videoFile: string }
): VideoCandidate {
const requestedValue = options.videoFile.trim()
if (requestedValue.length === 0) {
throw new Error('--video-file is required')
}
const artifactsRoot = resolve(options.artifactsDir)
const requestedAbsolutePath = resolve(requestedValue)
const requestedPathKey = normalizePathForMatch(requestedValue)
const matches = candidates.filter((candidate) => {
const candidateAbsolutePath = resolve(candidate.videoPath)
if (candidateAbsolutePath === requestedAbsolutePath) {
return true
}
const candidateBaseName = basename(candidate.videoPath)
if (candidateBaseName === requestedValue) {
return true
}
const relativeToCwd = normalizePathForMatch(
relative(process.cwd(), candidateAbsolutePath)
)
if (relativeToCwd === requestedPathKey) {
return true
}
const relativeToArtifacts = normalizePathForMatch(
relative(artifactsRoot, candidateAbsolutePath)
)
return relativeToArtifacts === requestedPathKey
})
if (matches.length === 1) {
return matches[0]
}
if (matches.length === 0) {
const availableVideos = candidates.map((candidate) =>
toProjectRelativePath(candidate.videoPath)
)
throw new Error(
[
`No video matched --video-file "${options.videoFile}".`,
'Available videos:',
...availableVideos.map((videoPath) => `- ${videoPath}`)
].join('\n')
)
}
throw new Error(
[
`--video-file "${options.videoFile}" matched ${matches.length} videos.`,
'Please pass a more specific path.',
...matches.map((match) => `- ${toProjectRelativePath(match.videoPath)}`)
].join('\n')
)
}
async function collectVideoCandidates(
artifactsDir: string
): Promise<VideoCandidate[]> {
const absoluteArtifactsDir = resolve(artifactsDir)
const videoPaths = globSync('**/qa-session{,-[0-9]}.mp4', {
cwd: absoluteArtifactsDir,
absolute: true,
nodir: true
}).sort()
const candidates = await Promise.all(
videoPaths.map(async (videoPath) => {
const videoStat = await stat(videoPath)
return {
platformName: extractPlatformFromVideoPath(videoPath),
videoPath,
mtimeMs: videoStat.mtimeMs
}
})
)
return candidates
}
function getMimeType(filePath: string): string {
const ext = extname(filePath).toLowerCase()
const mimeMap: Record<string, string> = {
'.mp4': 'video/mp4',
'.webm': 'video/webm',
'.mov': 'video/quicktime',
'.avi': 'video/x-msvideo',
'.mkv': 'video/x-matroska',
'.m4v': 'video/mp4'
}
return mimeMap[ext] || 'video/mp4'
}
function buildReviewPrompt(options: {
platformName: string
videoPath: string
prContext: string
isComparative: boolean
}): string {
const { platformName, videoPath, prContext, isComparative } = options
if (isComparative) {
return buildComparativePrompt(platformName, videoPath, prContext)
}
return buildSingleVideoPrompt(platformName, videoPath, prContext)
}
function buildComparativePrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer performing a BEFORE/AFTER comparison review.',
'',
'You are given TWO videos:',
'- **Video 1 (BEFORE)**: The main branch BEFORE the PR. This shows the OLD behavior.',
'- **Video 2 (AFTER)**: The PR branch AFTER the changes. This shows the NEW behavior.',
'',
'Both videos show the same test steps executed on different code versions.',
''
]
if (prContext) {
lines.push('## PR Context', prContext, '')
}
lines.push(
'## Your Task',
`Platform: "${platformName}". After video: ${toProjectRelativePath(videoPath)}.`,
'',
'1. **BEFORE video**: Does it demonstrate the old behavior or bug that the PR aims to fix?',
' Describe what you observe — this establishes the baseline.',
'2. **AFTER video**: Does it prove the PR fix works? Is the intended new behavior visible?',
'3. **Comparison**: What specifically changed between before and after?',
'4. **Regressions**: Did the PR introduce any new problems visible in the AFTER video',
' that were NOT present in the BEFORE video?',
'',
'Note: Brief black frames during page transitions are NORMAL.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible differences. Avoid speculation.',
'',
'Return markdown with these sections exactly:',
'## Summary',
'(What the PR changes, whether BEFORE confirms the old behavior, whether AFTER proves the fix)',
'',
'## Behavior Changes',
'Summarize ALL behavioral differences as a markdown TABLE:',
'| Behavior | Before (main) | After (PR) | Verdict |',
'',
'- **Behavior**: short name for the behavior (e.g. "Save shortcut label", "Menu hover style")',
'- **Before (main)**: how it works/looks in the BEFORE video',
'- **After (PR)**: how it works/looks in the AFTER video',
'- **Verdict**: `Fixed`, `Improved`, `Changed`, `Regression`, or `No Change`',
'',
'One row per distinct behavior. Include both changed AND unchanged key behaviors',
'that were tested, so reviewers can confirm nothing was missed.',
'',
'## Timeline Comparison',
'Present a chronological frame-by-frame comparison as a markdown TABLE:',
'| Time | Type | Severity | Before (main) | After (PR) |',
'',
'- **Time**: timestamp or range from the videos (e.g. `0:05-0:08`)',
'- **Type**: category such as `Visual`, `Behavior`, `Layout`, `Text`, `Animation`, `Menu`, `State`',
'- **Severity**: `None` (neutral change), `Fixed` (bug resolved), `Regression`, `Minor`, `Major`',
'- **Before (main)**: what is observed in the BEFORE video at that time',
'- **After (PR)**: what is observed in the AFTER video at that time',
'',
'Include one row per distinct observable difference. If behavior is identical at a timestamp,',
'omit that row. Focus on meaningful differences, not narrating every frame.',
'',
'## Confirmed Issues',
'For each issue, use this exact format:',
'',
'### [Short issue title]',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'',
'[Description — specify whether it appears in BEFORE, AFTER, or both]',
'',
'**Evidence:** [What you observed at the given timestamp in which video]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'(Assess whether the PR achieves its goal based on the before/after comparison)',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high", "confidence": "high" | "medium" | "low"}',
'- REPRODUCED: the before video confirms the old behavior and the after video shows the fix working',
'- NOT_REPRODUCIBLE: the before video does not show the reported bug',
'- INCONCLUSIVE: the videos do not adequately demonstrate the behavior change'
)
return lines.filter(Boolean).join('\n')
}
function buildSingleVideoPrompt(
platformName: string,
videoPath: string,
prContext: string
): string {
const lines = [
'You are a senior QA engineer reviewing a UI test session recording.',
'',
'## ANTI-HALLUCINATION RULES (READ FIRST)',
'- Describe ONLY what you can directly observe in the video frames',
'- NEVER infer or assume what "must have happened" between frames',
'- If a step is not visible in the video, say "NOT SHOWN" — do not guess',
'- Your job is to be a CAMERA — report facts, not interpretations',
''
]
const isIssueContext =
prContext &&
/^### Issue #|^Title:.*\bbug\b|^This video attempts to reproduce/im.test(
prContext
)
if (prContext) {
lines.push(
'## Phase 1: Blind Observation (describe what you SEE and HEAR)',
'First, describe every UI interaction chronologically WITHOUT knowing the expected outcome:',
'- What elements does the user click/hover/type?',
'- What dialogs/menus open and close?',
'- What keyboard indicators appear? (look for subtitle overlays)',
'- What is the BEFORE state and AFTER state of each action?',
'- **Audio**: Does the video have a TTS narration audio track? If yes, transcribe what the voice says. This narration describes the bug being reproduced.',
'',
'## Phase 2: Compare against expected behavior',
'Now compare your observations against the context below.',
'Only claim a match if your Phase 1 observations EXPLICITLY support it.',
''
)
if (isIssueContext) {
lines.push(
'## Issue Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video perform the reproduction steps described in the issue?',
'2. Did your Phase 1 observations show the reported bug behavior?',
'3. If the steps were not performed or the bug was not visible, say INCONCLUSIVE.',
''
)
} else {
lines.push(
'## PR Context',
prContext,
'',
'## Comparison Questions',
'1. Did the video test the specific behavior the PR changes?',
'2. Did your Phase 1 observations show the expected before/after difference?',
'3. If the test was incomplete or inconclusive, say so honestly.',
''
)
}
}
lines.push(
`Review this QA session video for platform "${platformName}".`,
`Source video: ${toProjectRelativePath(videoPath)}.`,
'The video shows the full test session — analyze it chronologically.',
'Focus on UI regressions, broken states, visual glitches, unreadable text, missing labels/i18n, and clear workflow failures.',
'Note: Brief black frames during page transitions are NORMAL and should NOT be reported as issues.',
'Note: Small cyan/purple dashed labels prefixed with "QA:" are annotations placed by the automated test script — they are NOT part of the application UI. Do not treat them as bugs or evidence.',
'Report only concrete, visible problems and avoid speculation.',
'If confidence is low, mark it explicitly.',
'',
'Return markdown with these sections exactly:',
'## Summary',
isIssueContext
? '(Explain what bug was reported and whether the video confirms it is reproducible)'
: prContext
? '(Explain what the PR intended and whether the video confirms it works)'
: '',
'## Confirmed Issues',
'For each confirmed issue, use this exact format (one block per issue):',
'',
'### [Short issue title]',
'`HIGH` `01:03` `Confidence: High`',
'',
'[Description of the issue — what went wrong and what was expected]',
'',
'**Evidence:** [What you observed in the video at the given timestamp]',
'',
'**Suggested Fix:** [Actionable recommendation]',
'',
'---',
'',
'The first line after the heading MUST be exactly three backtick-wrapped labels:',
'`SEVERITY` `TIMESTAMP` `Confidence: LEVEL`',
'Do NOT use a table for issues — use the block format above.',
'## Possible Issues (Needs Human Verification)',
'## Overall Risk',
'',
'## Narration',
'If the video contains a TTS audio narration track, transcribe it here.',
'If there is no audio or the video is silent, write "No narration detected."',
'',
'## Verdict',
'End your report with this EXACT JSON block (no markdown fence):',
'{"verdict": "REPRODUCED" | "NOT_REPRODUCIBLE" | "INCONCLUSIVE", "risk": "low" | "medium" | "high" | null, "confidence": "high" | "medium" | "low", "narrationDetected": true | false}',
'- REPRODUCED: the bug/behavior is clearly visible in the video',
'- NOT_REPRODUCIBLE: the steps were performed correctly but the bug was not observed',
'- INCONCLUSIVE: the reproduction steps were not performed or the video is insufficient',
'- narrationDetected: true if you heard TTS voice narration in the video, false if silent'
)
return lines.filter(Boolean).join('\n')
}
const MAX_VIDEO_BYTES = 100 * 1024 * 1024
async function readVideoFile(videoPath: string): Promise<Buffer> {
const fileStat = await stat(videoPath)
if (fileStat.size > MAX_VIDEO_BYTES) {
throw new Error(
`Video ${basename(videoPath)} is ${formatBytes(fileStat.size)}, exceeds ${formatBytes(MAX_VIDEO_BYTES)} limit`
)
}
return readFile(videoPath)
}
async function requestGeminiReview(options: {
apiKey: string
model: string
platformName: string
videoPath: string
beforeVideoPath: string
timeoutMs: number
prContext: string
}): Promise<string> {
const genAI = new GoogleGenerativeAI(options.apiKey)
const model = genAI.getGenerativeModel({ model: options.model })
const isComparative = options.beforeVideoPath.length > 0
const prompt = buildReviewPrompt({
platformName: options.platformName,
videoPath: options.videoPath,
prContext: options.prContext,
isComparative
})
const parts: Array<
{ text: string } | { inlineData: { mimeType: string; data: string } }
> = [{ text: prompt }]
if (isComparative) {
const beforeBuffer = await readVideoFile(options.beforeVideoPath)
parts.push(
{ text: 'Video 1 — BEFORE (main branch):' },
{
inlineData: {
mimeType: getMimeType(options.beforeVideoPath),
data: beforeBuffer.toString('base64')
}
}
)
}
const afterBuffer = await readVideoFile(options.videoPath)
if (isComparative) {
parts.push({ text: 'Video 2 — AFTER (PR branch):' })
}
parts.push({
inlineData: {
mimeType: getMimeType(options.videoPath),
data: afterBuffer.toString('base64')
}
})
const result = await model.generateContent(parts, {
timeout: options.timeoutMs
})
const response = result.response
const text = response.text()
if (!text || text.trim().length === 0) {
throw new Error('Gemini API returned no output text')
}
return text.trim()
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
function buildReportMarkdown(input: {
platformName: string
model: string
videoPath: string
videoSizeBytes: number
beforeVideoPath?: string
beforeVideoSizeBytes?: number
reviewText: string
targetUrl?: string
}): string {
const headerLines = [
`# ${input.platformName} QA Video Report`,
'',
`- Generated at: ${new Date().toISOString()}`,
`- Model: \`${input.model}\``
]
if (input.targetUrl) {
headerLines.push(`- Target: ${input.targetUrl}`)
}
if (input.beforeVideoPath) {
headerLines.push(
`- Before video: \`${toProjectRelativePath(input.beforeVideoPath)}\` (${formatBytes(input.beforeVideoSizeBytes ?? 0)})`,
`- After video: \`${toProjectRelativePath(input.videoPath)}\` (${formatBytes(input.videoSizeBytes)})`,
'- Mode: **Comparative (before/after)**'
)
} else {
headerLines.push(
`- Source video: \`${toProjectRelativePath(input.videoPath)}\``,
`- Video size: ${formatBytes(input.videoSizeBytes)}`
)
}
headerLines.push('', '## AI Review', '')
return `${headerLines.join('\n')}${input.reviewText.trim()}\n`
}
async function reviewVideo(
video: VideoCandidate,
options: CliOptions,
apiKey: string
): Promise<void> {
let prContext = ''
if (options.prContext) {
try {
prContext = await readFile(options.prContext, 'utf-8')
process.stdout.write(
`[${video.platformName}] Loaded PR context from ${options.prContext}\n`
)
} catch {
process.stdout.write(
`[${video.platformName}] Warning: Could not read PR context file ${options.prContext}\n`
)
}
}
const beforeVideoPath = options.beforeVideo
? resolve(options.beforeVideo)
: ''
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
process.stdout.write(
`[${video.platformName}] Before video: ${toProjectRelativePath(beforeVideoPath)} (${formatBytes(beforeStat.size)})\n`
)
}
process.stdout.write(
`[${video.platformName}] Sending ${beforeVideoPath ? '2 videos (comparative)' : 'video'} to ${options.model}\n`
)
const reviewText = await requestGeminiReview({
apiKey,
model: options.model,
platformName: video.platformName,
videoPath: video.videoPath,
beforeVideoPath,
timeoutMs: options.requestTimeoutMs,
prContext
})
const videoStat = await stat(video.videoPath)
const passSegment = options.passLabel ? `-${options.passLabel}` : ''
const outputPath = resolve(
options.outputDir,
`${video.platformName}${passSegment}-qa-video-report.md`
)
const reportInput: Parameters<typeof buildReportMarkdown>[0] = {
platformName: video.platformName,
model: options.model,
videoPath: video.videoPath,
videoSizeBytes: videoStat.size,
reviewText,
targetUrl: options.targetUrl || undefined
}
if (beforeVideoPath) {
const beforeStat = await stat(beforeVideoPath)
reportInput.beforeVideoPath = beforeVideoPath
reportInput.beforeVideoSizeBytes = beforeStat.size
}
const reportMarkdown = buildReportMarkdown(reportInput)
await mkdir(dirname(outputPath), { recursive: true })
await writeFile(outputPath, reportMarkdown, 'utf-8')
process.stdout.write(
`[${video.platformName}] Wrote ${toProjectRelativePath(outputPath)}\n`
)
}
function isExecutedAsScript(metaUrl: string): boolean {
const modulePath = fileURLToPath(metaUrl)
const scriptPath = process.argv[1] ? resolve(process.argv[1]) : ''
return modulePath === scriptPath
}
async function main(): Promise<void> {
const options = parseCliOptions(process.argv.slice(2))
const candidates = await collectVideoCandidates(options.artifactsDir)
if (candidates.length === 0) {
process.stdout.write(
`No qa-session.mp4 files found under ${toProjectRelativePath(resolve(options.artifactsDir))}\n`
)
return
}
const selectedVideo = selectVideoCandidateByFile(candidates, {
artifactsDir: options.artifactsDir,
videoFile: options.videoFile
})
process.stdout.write(
`Selected ${selectedVideo.platformName}: ${toProjectRelativePath(selectedVideo.videoPath)}\n`
)
if (options.dryRun) {
process.stdout.write('\nDry run mode enabled, no API calls were made.\n')
return
}
const apiKey = process.env.GEMINI_API_KEY
if (!apiKey) {
throw new Error('GEMINI_API_KEY is required unless --dry-run is set')
}
await reviewVideo(selectedVideo, options, apiKey)
}
if (isExecutedAsScript(import.meta.url)) {
void main().catch((error: unknown) => {
const message = errorToString(error)
process.stderr.write(`qa-video-review failed: ${message}\n`)
process.exit(1)
})
}

View File

@@ -1,513 +0,0 @@
#!/usr/bin/env tsx
/**
* QA CLI — simplified entry point for local & CI QA runs
*
* Usage:
* pnpm qa 10253 # auto-detects issue vs PR
* pnpm qa https://github.com/.../pull/10270
* pnpm qa 10270 -t base # test PR base (reproduce bug)
* pnpm qa 10270 -t both # test base + head
* pnpm qa --uncommitted # test local uncommitted changes
*
* Automatically loads .env.local / .env for GEMINI_API_KEY, ANTHROPIC_API_KEY.
* Results are written to .comfy-qa/<number>/ by default.
*/
import { parseArgs } from 'node:util'
import { config } from 'dotenv'
import { existsSync, mkdirSync, writeFileSync } from 'fs'
import { dirname, resolve } from 'path'
import { execSync, spawn, spawnSync } from 'child_process'
import { fileURLToPath } from 'url'
// ── Constants ──
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const RECORD_SCRIPT = resolve(SCRIPT_DIR, 'qa-record.ts')
const DEFAULT_REPO = 'Comfy-Org/ComfyUI_frontend'
const VALID_TARGETS = ['head', 'base', 'both'] as const
const CLOUD_FALLBACK_URL = 'https://testcloud.comfy.org/'
type PrTarget = (typeof VALID_TARGETS)[number]
type TargetType = 'issue' | 'pr'
// ── Load .env.local / .env ──
for (const f of ['.env.local', '.env']) {
if (existsSync(f)) {
config({ path: f })
break
}
}
// ── Parse CLI ──
const { values, positionals } = tryParseArgs()
if (values.help) {
printUsage()
process.exit(0)
}
const serverUrl =
values.url || process.env.DEV_SERVER_COMFYUI_URL || 'http://127.0.0.1:8188'
const prTarget = values.target as PrTarget
if (!VALID_TARGETS.includes(prTarget)) {
console.error(
`Invalid --target "${prTarget}". Must be one of: ${VALID_TARGETS.join(', ')}`
)
process.exit(1)
}
// ── Ensure server is reachable (may fall back to cloud) ──
const resolvedServerUrl = await ensureServer(serverUrl)
// ── Dispatch by mode ──
if (values.uncommitted) {
runUncommitted()
} else {
const input = positionals[0]
if (!input) {
printUsage()
process.exit(1)
}
runTarget(input)
}
// ── Mode: uncommitted changes ──
function runUncommitted(): never {
const diff = shell('git diff && git diff --staged')
if (!diff.trim()) {
console.error('No uncommitted changes found')
process.exit(1)
}
const outputDir = resolveOutputDir('.comfy-qa/local')
const diffFile = writeTmpFile(outputDir, 'uncommitted.diff', diff)
logHeader({ label: 'uncommitted changes', outputDir })
const code = runQaRecord('after', diffFile, outputDir)
exit(code, outputDir)
}
// ── Mode: issue or PR by number/URL ──
function runTarget(input: string): never {
const { targetType, number, repo } = resolveTarget(input)
const outputDir = resolveOutputDir(`.comfy-qa/${number}`)
logHeader({
label: `${targetType} #${number} (${repo})`,
outputDir,
extra: targetType === 'pr' ? `Target: ${prTarget}` : undefined
})
const diffFile =
targetType === 'issue'
? fetchIssue(number, repo, outputDir)
: fetchPR(number, repo, outputDir)
let exitCode: number
if (targetType === 'issue') {
exitCode = runQaRecord('reproduce', diffFile, outputDir)
} else if (prTarget === 'both') {
exitCode = runPrBoth(diffFile, outputDir)
} else if (prTarget === 'base') {
exitCode = runQaRecord('before', diffFile, outputDir)
} else {
exitCode = runQaRecord('after', diffFile, outputDir)
}
exit(exitCode, outputDir)
}
// ── PR both phases ──
function runPrBoth(diffFile: string, outputDir: string): number {
console.warn('\n=== Phase 1: Reproduce bug on base ===')
const baseDir = resolve(outputDir, 'base')
mkdirSync(baseDir, { recursive: true })
const baseCode = runQaRecord('before', diffFile, baseDir)
if (baseCode !== 0) {
console.warn('Base phase failed, continuing to head...')
}
console.warn('\n=== Phase 2: Demonstrate fix on head ===')
const headDir = resolve(outputDir, 'head')
mkdirSync(headDir, { recursive: true })
return runQaRecord('after', diffFile, headDir)
}
// ── Target resolution ──
function resolveTarget(input: string): {
targetType: TargetType
number: string
repo: string
} {
const urlMatch = input.match(
/github\.com\/([^/]+\/[^/]+)\/(issues|pull)\/(\d+)/
)
if (urlMatch) {
return {
repo: urlMatch[1],
targetType: urlMatch[2] === 'pull' ? 'pr' : 'issue',
number: urlMatch[3]
}
}
if (/^\d+$/.test(input)) {
return {
repo: DEFAULT_REPO,
targetType: detectType(input, DEFAULT_REPO),
number: input
}
}
console.error(`Cannot parse target: ${input}`)
console.error('Expected a GitHub URL or issue/PR number')
printUsage()
process.exit(1)
}
function detectType(number: string, repo: string): TargetType {
try {
const result = execSync(
`gh api repos/${repo}/issues/${number} --jq 'has("pull_request")'`,
{ encoding: 'utf-8', timeout: 15000, stdio: ['pipe', 'pipe', 'pipe'] }
)
return result.trim() === 'true' ? 'pr' : 'issue'
} catch {
return 'issue'
}
}
// ── Data fetching ──
function fetchIssue(number: string, repo: string, outputDir: string): string {
console.warn(`Fetching issue #${number}...`)
const body = shell(
`gh issue view ${number} --repo ${repo} --json title,body,labels --jq '"Title: " + .title + "\\n\\nLabels: " + ([.labels[].name] | join(", ")) + "\\n\\n" + .body'`
)
return writeTmpFile(outputDir, `issue-${number}.txt`, body)
}
function fetchPR(number: string, repo: string, outputDir: string): string {
console.warn(`Fetching PR #${number}...`)
const prJson = shell(
`gh pr view ${number} --repo ${repo} --json title,body,baseRefName,headRefName,baseRefOid,headRefOid`
)
const pr = JSON.parse(prJson) as {
title: string
body: string
baseRefName: string
headRefName: string
baseRefOid: string
headRefOid: string
}
console.warn(` Base: ${pr.baseRefName} (${pr.baseRefOid.slice(0, 8)})`)
console.warn(` Head: ${pr.headRefName} (${pr.headRefOid.slice(0, 8)})`)
let diff = ''
try {
diff = shell(`gh pr diff ${number} --repo ${repo}`)
} catch {
console.warn('Could not fetch PR diff')
}
writeTmpFile(
outputDir,
'refs.json',
JSON.stringify(
{
base: { ref: pr.baseRefName, sha: pr.baseRefOid },
head: { ref: pr.headRefName, sha: pr.headRefOid }
},
null,
2
)
)
return writeTmpFile(
outputDir,
`pr-${number}.txt`,
`Title: ${pr.title}\n\n${pr.body}\n\n--- DIFF ---\n\n${diff}`
)
}
// ── QA record runner ──
function runQaRecord(
mode: string,
diffFile: string,
outputDir: string
): number {
console.warn(`\nStarting QA ${mode} mode...\n`)
const r = spawnSync(
'pnpm',
[
'exec',
'tsx',
RECORD_SCRIPT,
'--mode',
mode,
'--diff',
diffFile,
'--output-dir',
outputDir,
'--url',
resolvedServerUrl
],
{ stdio: 'inherit', env: process.env }
)
return r.status ?? 1
}
// ── Server management ──
async function ensureServer(url: string): Promise<string> {
if (await isReachable(url)) {
console.warn(`Server OK: ${url}`)
return url
}
console.warn(`Server not reachable at ${url}, attempting auto-start...`)
const port = new URL(url).port || '8188'
// Strategy 1: comfy-cli (pip install comfy-cli)
try {
execSync('which comfy', { stdio: 'pipe' })
console.warn('Starting ComfyUI via comfy-cli...')
const proc = spawn(
'comfy',
['launch', '--background', '--', '--cpu', '--port', port],
{
stdio: 'ignore',
detached: true
}
)
proc.unref()
await waitForServer(url, 120000)
return url
} catch {
// comfy-cli not available
}
// Strategy 2: python main.py from TEST_COMFYUI_DIR or .comfy-qa/ComfyUI
const comfyDir = findComfyUIDir()
if (comfyDir) {
console.warn(`Starting ComfyUI from ${comfyDir}...`)
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
cwd: comfyDir,
stdio: 'ignore',
detached: true
})
proc.unref()
await waitForServer(url, 120000)
return url
}
// Strategy 3: clone ComfyUI and start
const cloneDir = resolve('.comfy-qa/ComfyUI')
if (!existsSync(resolve(cloneDir, 'main.py'))) {
console.warn('No ComfyUI installation found, cloning...')
try {
execSync(
`git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git "${cloneDir}"`,
{ stdio: 'inherit', timeout: 120000 }
)
console.warn('Installing ComfyUI dependencies...')
execSync('pip install -r requirements.txt', {
cwd: cloneDir,
stdio: 'inherit',
timeout: 300000
})
} catch (err) {
console.warn(
`Clone/install failed: ${err instanceof Error ? err.message : err}`
)
}
}
if (existsSync(resolve(cloneDir, 'main.py'))) {
console.warn(`Starting ComfyUI from ${cloneDir}...`)
const proc = spawn('python', ['main.py', '--cpu', '--port', port], {
cwd: cloneDir,
stdio: 'ignore',
detached: true
})
proc.unref()
await waitForServer(url, 120000)
return url
}
// Strategy 4: fallback to testcloud
console.warn(`Local server failed. Falling back to ${CLOUD_FALLBACK_URL}`)
if (await isReachable(CLOUD_FALLBACK_URL)) {
console.warn(`Cloud server OK: ${CLOUD_FALLBACK_URL}`)
return CLOUD_FALLBACK_URL
}
console.error(`
No ComfyUI server available. Tried:
1. ${url} (not reachable)
2. comfy-cli (not installed)
3. Local ComfyUI installation (not found)
4. ${CLOUD_FALLBACK_URL} (not reachable)
Install: pip install comfy-cli && comfy install && comfy launch --cpu
`)
process.exit(1)
}
function findComfyUIDir(): string | undefined {
const candidates = [
process.env.TEST_COMFYUI_DIR,
resolve('.comfy-qa/ComfyUI'),
'/home/ComfyUI'
]
return candidates.find((d) => d && existsSync(resolve(d, 'main.py')))
}
async function isReachable(url: string): Promise<boolean> {
try {
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), 5000)
const res = await fetch(url, { signal: controller.signal })
clearTimeout(timeout)
return res.ok || res.status === 200 || res.status === 304
} catch {
return false
}
}
async function waitForServer(url: string, timeoutMs: number): Promise<void> {
const start = Date.now()
while (Date.now() - start < timeoutMs) {
if (await isReachable(url)) {
console.warn('Server is ready')
return
}
await new Promise((r) => setTimeout(r, 2000))
}
console.error(`Server did not start within ${timeoutMs / 1000}s`)
process.exit(1)
}
// ── Utilities ──
function shell(cmd: string): string {
return execSync(cmd, { encoding: 'utf-8', timeout: 30000 })
}
function writeTmpFile(
outputDir: string,
filename: string,
content: string
): string {
const tmpDir = resolve(outputDir, '.tmp')
mkdirSync(tmpDir, { recursive: true })
const filePath = resolve(tmpDir, filename)
writeFileSync(filePath, content)
return filePath
}
function resolveOutputDir(defaultPath: string): string {
const dir = values.output ? resolve(values.output) : resolve(defaultPath)
mkdirSync(dir, { recursive: true })
return dir
}
function logHeader(opts: { label: string; outputDir: string; extra?: string }) {
console.warn(`QA target: ${opts.label}`)
console.warn(`Output: ${opts.outputDir}`)
console.warn(`Server: ${resolvedServerUrl}`)
if (values.ref) console.warn(`Ref: ${values.ref}`)
if (opts.extra) console.warn(opts.extra)
}
function exit(code: number, outputDir: string): never {
console.warn('\n=== QA Complete ===')
console.warn(`Results: ${outputDir}`)
try {
console.warn(shell(`ls -la "${outputDir}"`))
} catch {
// not critical
}
process.exit(code)
}
function tryParseArgs() {
try {
const parsed = parseArgs({
args: process.argv.slice(2),
options: {
target: { type: 'string', short: 't', default: 'head' },
uncommitted: { type: 'boolean', default: false },
url: { type: 'string', default: '' },
ref: { type: 'string', default: '' },
output: { type: 'string', short: 'o', default: '' },
help: { type: 'boolean', short: 'h', default: false }
},
allowPositionals: true,
strict: true
})
return {
values: parsed.values as {
target: string
uncommitted: boolean
url: string
ref: string
output: string
help: boolean
},
positionals: parsed.positionals
}
} catch (err) {
console.error(`Error: ${err instanceof Error ? err.message : err}\n`)
printUsage()
process.exit(1)
}
}
function printUsage() {
console.warn(`
QA CLI — Reproduce issues & test PRs for ComfyUI frontend
Usage:
pnpm qa <number|url> [options]
pnpm qa --uncommitted
Targets:
10253 Number (auto-detects issue vs PR via gh CLI)
https://github.com/Comfy-Org/ComfyUI_frontend/issues/10253
https://github.com/Comfy-Org/ComfyUI_frontend/pull/10270
Options:
-t, --target <head|base|both>
For PRs: which ref to test (default: head)
head — test the fix (PR head)
base — reproduce the bug (PR base)
both — base then head
--uncommitted Test local uncommitted changes
--url <url> ComfyUI server URL (default: from .env or http://127.0.0.1:8188)
--ref <ref> Git ref to test against
-o, --output <dir> Override output directory (default: .comfy-qa/<number>)
-h, --help Show this help
Environment (auto-loaded from .env.local or .env):
GEMINI_API_KEY Required — used for PR analysis, video review, TTS
ANTHROPIC_API_KEY Optional locally — Claude Agent SDK auto-detects Claude Code session
Examples:
pnpm qa 10253 # reproduce an issue
pnpm qa 10270 # test PR head (the fix)
pnpm qa 10270 -t base # reproduce bug on PR base
pnpm qa 10270 -t both # test base + head
pnpm qa --uncommitted # test local changes
`)
}

View File

@@ -1,99 +0,0 @@
---
name: contain-audit
description: 'Detect DOM elements where CSS contain:layout+style would improve rendering performance. Runs a Playwright-based audit on a large workflow, scores candidates by subtree size and sizing constraints, measures performance impact, and generates a ranked report.'
---
# CSS Containment Audit
Automatically finds DOM elements where adding `contain: layout style` would reduce browser recalculation overhead.
## What It Does
1. Loads a large workflow (245 nodes) in a real browser
2. Walks the DOM tree and scores every element as a containment candidate
3. For each high-scoring candidate, applies `contain: layout style` via JavaScript
4. Measures rendering performance (style recalcs, layouts, task duration) before and after
5. Takes before/after screenshots to detect visual breakage
6. Generates a ranked report with actionable recommendations
## When to Use
- After adding new Vue components to the node rendering pipeline
- When investigating rendering performance on large workflows
- Before and after refactoring node DOM structure
- As part of periodic performance audits
## How to Run
```bash
# Start the dev server first
pnpm dev &
# Run the audit (uses the @audit tag, not included in normal CI runs)
pnpm exec playwright test browser_tests/tests/containAudit.spec.ts --project=audit
# View the HTML report
pnpm exec playwright show-report
```
## How to Read Results
The audit outputs a table to the console:
```text
CSS Containment Audit Results
=======================================================
Rank | Selector | Subtree | Score | DRecalcs | DLayouts | Visual
1 | [data-testid="node-inner-wrap"] | 18 | 72 | -34% | -12% | OK
2 | .node-body | 12 | 48 | -8% | -3% | OK
3 | .node-header | 4 | 16 | +1% | 0% | OK
```
- **Subtree**: Number of descendant elements (higher = more to skip)
- **Score**: Composite heuristic score (subtree size x sizing constraint bonus)
- **DRecalcs / DLayouts**: Change in style recalcs / layout counts vs baseline (negative = improvement)
- **Visual**: OK if no pixel change, DIFF if screenshot differs (may include subpixel noise — verify manually)
## Candidate Scoring
An element is a good containment candidate when:
1. **Large subtree** -- many descendants that the browser can skip recalculating
2. **Externally constrained size** -- width/height determined by CSS variables, flex, or explicit values (not by content)
3. **No existing containment** -- `contain` is not already applied
4. **Not a leaf** -- has at least a few child elements
Elements that should NOT get containment:
- Elements whose children overflow visually beyond bounds (e.g., absolute-positioned overlays with negative inset)
- Elements whose height is determined by content and affects sibling layout
- Very small subtrees (overhead of containment context outweighs benefit)
## Limitations
- Cannot fully guarantee `contain` safety -- visual review of screenshots is required
- Performance measurements have natural variance; run multiple times for confidence
- Only tests idle and pan scenarios; widget interactions may differ
- The audit modifies styles at runtime via JS, which doesn't account for Tailwind purging or build-time optimizations
## Example PR
[#9946 — fix: add CSS contain:layout contain:style to node inner wrapper](https://github.com/Comfy-Org/ComfyUI_frontend/pull/9946)
This PR added `contain-layout contain-style` to the node inner wrapper div in `LGraphNode.vue`. The audit tool would have flagged this element as a high-scoring candidate because:
- **Large subtree** (18+ descendants: header, slots, widgets, content, badges)
- **Externally constrained size** (`w-(--node-width)`, `flex-1` — dimensions set by CSS variables and flex parent)
- **Natural isolation boundary** between frequently-changing content (widgets) and infrequently-changing overlays (selection outlines, borders)
The actual change was a single line: adding `'contain-layout contain-style'` to the inner wrapper's class list at `src/renderer/extensions/vueNodes/components/LGraphNode.vue:79`.
## Reference
| Resource | Path |
| ----------------- | ------------------------------------------------------- |
| Audit test | `browser_tests/tests/containAudit.spec.ts` |
| PerformanceHelper | `browser_tests/fixtures/helpers/PerformanceHelper.ts` |
| Perf tests | `browser_tests/tests/performance.spec.ts` |
| Large workflow | `browser_tests/assets/large-graph-workflow.json` |
| Example PR | https://github.com/Comfy-Org/ComfyUI_frontend/pull/9946 |

View File

@@ -1,246 +0,0 @@
---
name: hardening-flaky-e2e-tests
description: 'Diagnoses and fixes flaky Playwright e2e tests by replacing race-prone patterns with retry-safe alternatives. Use when triaging CI flakes, hardening spec files, fixing timing races, or asked to stabilize browser tests. Triggers on: flaky, flake, harden, stabilize, race condition in e2e, intermittent failure.'
---
# Hardening Flaky E2E Tests
Fix flaky Playwright specs by identifying race-prone patterns and replacing them with retry-safe alternatives. This skill covers diagnosis, pattern matching, and mechanical transforms — not writing new tests (see `writing-playwright-tests` for that).
## Workflow
### 1. Gather CI Evidence
```bash
gh run list --workflow=ci-test.yaml --limit=5
gh run download <run-id> -n playwright-report
```
- Open `report.json` and search for `"status": "flaky"` entries.
- Collect file paths, test titles, and error messages.
- Do NOT trust green checks alone — flaky tests that passed on retry still need fixing.
- Use `error-context.md`, traces, and page snapshots before editing code.
- Pull the newest run after each push instead of assuming the flaky set is unchanged.
### 2. Classify the Flake
Read the failing assertion and match it against the pattern table. Most flakes fall into one of these categories:
| # | Pattern | Signature in Code | Fix |
| --- | ------------------------------------- | --------------------------------------------------------- | ---------------------------------------------------------------- |
| 1 | **Snapshot-then-assert** | `expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
| 2 | **Immediate count** | `const n = await loc.count(); expect(n).toBe(3)` | `await expect(loc).toHaveCount(3)` |
| 3 | **nextFrame after menu click** | `clickMenuItem(x); nextFrame()` | `clickMenuItem(x); contextMenu.waitForHidden()` |
| 4 | **Tight poll timeout** | `expect.poll(..., { timeout: 250 })` | ≥2000 ms; prefer default 5000 ms |
| 5 | **Immediate evaluate after mutation** | `setSetting(k, v); expect(await evaluate()).toBe(x)` | `await expect.poll(() => evaluate()).toBe(x)` |
| 6 | **Screenshot without readiness** | `loadWorkflow(); nextFrame(); toHaveScreenshot()` | `waitForNodes()` or poll state first |
| 7 | **Non-deterministic node order** | `getNodeRefsByType('X')[0]` with >1 match | `getNodeRefById(id)` or guard `toHaveLength(1)` |
| 8 | **Fake readiness helper** | Helper clicks but doesn't assert state | Remove; poll the actual value |
| 9 | **Immediate graph state after drop** | `expect(await getLinkCount()).toBe(1)` | `await expect.poll(() => getLinkCount()).toBe(1)` |
| 10 | **Immediate boundingBox/layout read** | `const box = await loc.boundingBox(); expect(box!.width)` | `await expect.poll(() => loc.boundingBox().then(b => b?.width))` |
### 3. Apply the Transform
#### Rule: Choose the Smallest Correct Assertion
- **Locator state** → use built-in retrying assertions: `toBeVisible()`, `toHaveText()`, `toHaveCount()`, `toHaveClass()`
- **Single async value** → `expect.poll(() => asyncFn()).toBe(expected)`
- **Multiple assertions that must settle together** → `expect(async () => { ... }).toPass()`
- **Never** use `waitForTimeout()` to hide a race.
```typescript
// ✅ Single value — use expect.poll
await expect
.poll(() => comfyPage.page.evaluate(() => window.app!.graph.links.length))
.toBe(3)
// ✅ Locator count — use toHaveCount
await expect(comfyPage.page.locator('.dom-widget')).toHaveCount(2)
// ✅ Multiple conditions — use toPass
await expect(async () => {
expect(await node1.getValue()).toBe('foo')
expect(await node2.getValue()).toBe('bar')
}).toPass({ timeout: 5000 })
```
#### Rule: Wait for the Real Readiness Boundary
Visible is not always ready. Prefer user-facing assertions when possible; poll internal state only when there is no UI surface to assert on.
Common readiness boundaries:
| After this action... | Wait for... |
| -------------------------------------- | ------------------------------------------------------------ |
| Canvas interaction (drag, click node) | `await comfyPage.nextFrame()` |
| Menu item click | `await contextMenu.waitForHidden()` |
| Workflow load | `await comfyPage.workflow.loadWorkflow(...)` (built-in wait) |
| Settings write | Poll the setting value with `expect.poll()` |
| Node pin/bypass/collapse toggle | `await expect.poll(() => nodeRef.isPinned()).toBe(true)` |
| Graph mutation (add/remove node, link) | Poll link/node count |
| Clipboard write | Poll pasted value |
| Screenshot | Ensure nodes are rendered: `waitForNodes()` or poll state |
#### Rule: Expose Locators for Retrying Assertions
When a helper returns a count via `await loc.count()`, callers can't use `toHaveCount()`. Expose the underlying `Locator` as a getter so callers choose between:
```typescript
// Helper exposes locator
get domWidgets(): Locator {
return this.page.locator('.dom-widget')
}
// Caller uses retrying assertion
await expect(comfyPage.domWidgets).toHaveCount(2)
```
Replace count methods with locator getters so callers can use retrying assertions directly.
#### Rule: Fix Check-then-Act Races in Helpers
```typescript
// ❌ Race: count can change between check and waitFor
const count = await locator.count()
if (count > 0) {
await locator.waitFor({ state: 'hidden' })
}
// ✅ Direct: waitFor handles both cases
await locator.waitFor({ state: 'hidden' })
```
#### Rule: Remove force:true from Clicks
`force: true` bypasses actionability checks, hiding real animation/visibility races. Remove it and fix the underlying timing issue.
```typescript
// ❌ Hides the race
await closeButton.click({ force: true })
// ✅ Surfaces the real issue — fix with proper wait
await closeButton.click()
await dialog.waitForHidden()
```
#### Rule: Handle Non-deterministic Element Order
When `getNodeRefsByType` returns multiple nodes, the order is not guaranteed. Don't use index `[0]` blindly.
```typescript
// ❌ Assumes order
const node = (await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode'))[0]
// ✅ Find by ID or proximity
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
let target = nodes[0]
for (const n of nodes) {
const pos = await n.getPosition()
if (Math.abs(pos.y - expectedY) < minDist) target = n
}
```
Or guard the assumption:
```typescript
const nodes = await comfyPage.nodeOps.getNodeRefsByType('CLIPTextEncode')
expect(nodes).toHaveLength(1)
const node = nodes[0]
```
#### Rule: Use toPass for Timing-sensitive Dismiss Guards
Some UI elements (e.g. LiteGraph's graphdialog) have built-in dismiss delays. Retry the entire dismiss action:
```typescript
// ✅ Retry click+assert together
await expect(async () => {
await comfyPage.canvas.click({ position: { x: 10, y: 10 } })
await expect(dialog).toBeHidden({ timeout: 500 })
}).toPass({ timeout: 5000 })
```
### 4. Keep Changes Narrow
- Shared helpers should drive setup to a stable boundary.
- Do not encode one-spec timing assumptions into generic helpers.
- If a race only matters to one spec, prefer a local wait in that spec.
- If a helper fails before the real test begins, remove or relax the brittle precondition and let downstream UI interaction prove readiness.
### 5. Verify Narrowly
```bash
# Targeted rerun with repetition
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts --repeat-each 10
# Single test by line number (avoids grep quoting issues on Windows)
pnpm test:browser:local -- browser_tests/tests/myFile.spec.ts:42
```
- Use `--repeat-each 10` for targeted flake verification (use 20 for single test cases).
- Verify with the smallest command that exercises the flaky path.
### 6. Watch CI E2E Runs
After pushing, use `gh` to monitor the E2E workflow:
```bash
# Find the run for the current branch
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1
# Watch it live (blocks until complete, streams logs)
gh run watch <run-id>
# One-liner: find and watch the latest E2E run for the current branch
gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId" | xargs gh run watch
```
On Windows (PowerShell):
```powershell
# One-liner equivalent
gh run watch (gh run list --workflow="CI: Tests E2E" --branch=$(git branch --show-current) --limit=1 --json databaseId --jq ".[0].databaseId")
```
After the run completes:
```bash
# Download the Playwright report artifact
gh run download <run-id> -n playwright-report
# View the run summary in browser
gh run view <run-id> --web
```
Also watch the unit test workflow in parallel if you changed helpers:
```bash
gh run list --workflow="CI: Tests Unit" --branch=$(git branch --show-current) --limit=1
```
### 7. Pre-merge Checklist
Before merging a flaky-test fix, confirm:
- [ ] The latest CI artifact was inspected directly
- [ ] The root cause is stated as a race or readiness mismatch
- [ ] The fix waits on the real readiness boundary
- [ ] The assertion primitive matches the job (poll vs toHaveCount vs toPass)
- [ ] The fix stays local unless a shared helper truly owns the race
- [ ] Local verification uses a targeted rerun
- [ ] No behavioral changes to the test — only timing/retry strategy updated
## Local Noise — Do Not Fix
These are local distractions, not CI root causes:
- Missing local input fixture files required by the test path
- Missing local models directory
- Teardown `EPERM` while restoring the local browser-test user data directory
- Local screenshot baseline differences on Windows
Rules:
- First confirm whether it blocks the exact flaky path under investigation.
- Do not commit temporary local assets used only for verification.
- Do not commit local screenshot baselines.

View File

@@ -1,82 +0,0 @@
---
name: layer-audit
description: 'Detect violations of the layered architecture import rules (base -> platform -> workbench -> renderer). Runs ESLint with the import-x/no-restricted-paths rule and generates a grouped report.'
---
# Layer Architecture Audit
Finds imports that violate the layered architecture boundary rules enforced by `import-x/no-restricted-paths` in `eslint.config.ts`.
## Layer Hierarchy (bottom to top)
```
renderer (top -- can import from all lower layers)
^
workbench
^
platform
^
base (bottom -- cannot import from any upper layer)
```
Each layer may only import from layers below it.
## How to Run
```bash
# Run ESLint filtering for just the layer boundary rule violations
pnpm lint 2>&1 | grep 'import-x/no-restricted-paths' -B1 | head -200
```
To get a full structured report, run:
```bash
# Collect all violations from base/, platform/, workbench/ layers
pnpm eslint src/base/ src/platform/ src/workbench/ --no-error-on-unmatched-pattern --rule '{"import-x/no-restricted-paths": "warn"}' --format compact 2>&1 | grep 'no-restricted-paths' | sort
```
## How to Read Results
Each violation line shows:
- The **file** containing the bad import
- The **import path** crossing the boundary
- The **message** identifying which layer pair is violated
### Grouping by Layer Pair
After collecting violations, group them by the layer pair pattern:
| Layer pair | Meaning |
| --------------------- | ----------------------------------- |
| base -> platform | base/ importing from platform/ |
| base -> workbench | base/ importing from workbench/ |
| base -> renderer | base/ importing from renderer/ |
| platform -> workbench | platform/ importing from workbench/ |
| platform -> renderer | platform/ importing from renderer/ |
| workbench -> renderer | workbench/ importing from renderer/ |
## When to Use
- Before creating a PR that adds imports between `src/base/`, `src/platform/`, `src/workbench/`, or `src/renderer/`
- When auditing the codebase to find and plan migration of existing violations
- After moving files between layers to verify no new violations were introduced
## Fixing Violations
Common strategies to resolve a layer violation:
1. **Move the import target down** -- if the imported module doesn't depend on upper-layer concepts, move it to a lower layer
2. **Introduce an interface** -- define an interface/type in the lower layer and implement it in the upper layer via dependency injection or a registration pattern
3. **Move the importing file up** -- if the file logically belongs in a higher layer, relocate it
4. **Extract shared logic** -- pull the shared functionality into `base/` or a shared utility
## Reference
| Resource | Path |
| ------------------------------- | ------------------ |
| ESLint config (rule definition) | `eslint.config.ts` |
| Base layer | `src/base/` |
| Platform layer | `src/platform/` |
| Workbench layer | `src/workbench/` |
| Renderer layer | `src/renderer/` |

View File

@@ -1,179 +0,0 @@
---
name: perf-fix-with-proof
description: 'Ships performance fixes with CI-proven improvement using stacked PRs. PR1 adds a @perf test (establishes baseline on main), PR2 adds the fix (CI shows delta). Use when implementing a perf optimization and wanting to prove it in CI.'
---
# Performance Fix with Proof
Ships perf fixes as two stacked PRs so CI automatically proves the improvement.
## Why Two PRs
The `ci-perf-report.yaml` workflow compares PR metrics against the **base branch baseline**. If you add a new `@perf` test in the same PR as the fix, that test doesn't exist on main yet — no baseline, no delta, no proof. Stacking solves this:
1. **PR1 (test-only)** — adds the `@perf` test that exercises the bottleneck. Merges to main. CI runs it on main → baseline established.
2. **PR2 (fix)** — adds the optimization. CI runs the same test → compares against PR1's baseline → delta shows improvement.
## Workflow
### Step 1: Create the test branch
```bash
git worktree add <worktree-path> -b perf/test-<name> origin/main
```
### Step 2: Write the `@perf` test
Add a test to `browser_tests/tests/performance.spec.ts` (or a new file with `@perf` tag). The test should stress the specific bottleneck.
**Test structure:**
```typescript
test('<descriptive name>', async ({ comfyPage }) => {
// 1. Load a workflow that exercises the bottleneck
await comfyPage.workflow.loadWorkflow('<workflow>')
// 2. Start measuring
await comfyPage.perf.startMeasuring()
// 3. Perform the action that triggers the bottleneck (at scale)
for (let i = 0; i < N; i++) {
// ... stress the hot path ...
await comfyPage.nextFrame()
}
// 4. Stop measuring and record
const m = await comfyPage.perf.stopMeasuring('<metric-name>')
recordMeasurement(m)
console.log(`<name>: ${m.styleRecalcs} recalcs, ${m.layouts} layouts`)
})
```
**Available metrics** (from `PerformanceHelper`):
- `m.styleRecalcs` / `m.styleRecalcDurationMs` — style recalculation count and time
- `m.layouts` / `m.layoutDurationMs` — forced layout count and time
- `m.taskDurationMs` — total main-thread JS execution time
- `m.heapDeltaBytes` — memory pressure delta
**Key helpers** (from `ComfyPage`):
- `comfyPage.perf.startMeasuring()` / `.stopMeasuring(name)` — CDP metrics capture
- `comfyPage.nextFrame()` — wait one animation frame
- `comfyPage.workflow.loadWorkflow(name)` — load a test workflow from `browser_tests/assets/`
- `comfyPage.canvas` — the canvas locator
- `comfyPage.page.mouse.move(x, y)` — mouse interaction
### Step 3: Add test workflow asset (if needed)
If the bottleneck needs a specific workflow (e.g., 50+ nodes, many DOM widgets), add it to `browser_tests/assets/`. Keep it minimal — only the structure needed to trigger the bottleneck.
### Step 4: Verify locally
```bash
pnpm exec playwright test --project=performance --grep "<test name>"
```
Confirm the test runs and produces reasonable metric values.
### Step 5: Create PR1 (test-only)
```bash
pnpm typecheck:browser
pnpm lint
git add browser_tests/
git commit -m "test: add perf test for <bottleneck description>"
git push -u origin perf/test-<name>
gh pr create --title "test: add perf test for <bottleneck>" \
--body "Adds a @perf test to establish a baseline for <bottleneck>.
This is PR 1 of 2. The fix will follow in a separate PR once this baseline is established on main.
## What
Adds \`<test-name>\` to the performance test suite measuring <metric> during <action>.
## Why
Needed to prove the improvement from the upcoming fix for backlog item #<N>." \
--base main
```
### Step 6: Get PR1 merged
Once PR1 merges, CI runs the test on main → baseline artifact saved.
### Step 7: Create PR2 (fix) on top of main
```bash
git worktree add <worktree-path> -b perf/fix-<name> origin/main
```
Implement the fix. The `@perf` test from PR1 is now on main and will run automatically. CI will:
1. Run the test on the PR branch
2. Download the baseline from main (which includes PR1's test results)
3. Post a PR comment showing the delta
### Step 8: Verify the improvement shows in CI
The `ci-perf-report.yaml` posts a comment like:
```markdown
## ⚡ Performance Report
| Metric | Baseline | PR (n=3) | Δ | Sig |
| --------------------- | -------- | -------- | ---- | --- |
| <name>: style recalcs | 450 | 12 | -97% | 🟢 |
```
If Δ is negative for the target metric, the fix is proven.
## Test Design Guidelines
1. **Stress the specific bottleneck** — don't measure everything, isolate the hot path
2. **Use enough iterations** — the test should run long enough that the metric difference is clear (100+ frames for idle tests, 50+ interactions for event tests)
3. **Keep it deterministic** — avoid timing-dependent assertions; measure counts not durations when possible
4. **Match the backlog entry** — reference the backlog item number in the test name or PR description
## Examples
**Testing DOM widget reactive mutations (backlog #8):**
```typescript
test('DOM widget positioning recalculations', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('default')
await comfyPage.perf.startMeasuring()
// Idle for 120 frames — DOM widgets update position every frame
for (let i = 0; i < 120; i++) {
await comfyPage.nextFrame()
}
const m = await comfyPage.perf.stopMeasuring('dom-widget-idle')
recordMeasurement(m)
})
```
**Testing measureText caching (backlog #4):**
```typescript
test('canvas text rendering with many nodes', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('large-workflow-50-nodes')
await comfyPage.perf.startMeasuring()
for (let i = 0; i < 60; i++) {
await comfyPage.nextFrame()
}
const m = await comfyPage.perf.stopMeasuring('text-rendering-50-nodes')
recordMeasurement(m)
})
```
## Reference
| Resource | Path |
| ----------------- | ----------------------------------------------------- |
| Perf test file | `browser_tests/tests/performance.spec.ts` |
| PerformanceHelper | `browser_tests/fixtures/helpers/PerformanceHelper.ts` |
| Perf reporter | `browser_tests/helpers/perfReporter.ts` |
| CI workflow | `.github/workflows/ci-perf-report.yaml` |
| Report generator | `scripts/perf-report.ts` |
| Stats utilities | `scripts/perf-stats.ts` |
| Backlog | `docs/perf/BACKLOG.md` (local only, not committed) |
| Playbook | `docs/perf/PLAYBOOK.md` (local only, not committed) |

View File

@@ -1,221 +0,0 @@
---
name: red-green-fix
description: 'Bug fix workflow that proves test validity with a red-then-green CI sequence. Commits a failing test first (CI red), then the minimal fix (CI green). Use when fixing a bug, writing a regression test, or when asked to prove a fix works.'
---
# Red-Green Fix
Fixes bugs as two commits so CI automatically proves the test catches the bug.
## Why Two Commits
If you commit the test and fix together, the test always passes — reviewers cannot tell whether the test actually detects the bug or is a no-op. Splitting into two commits creates a verifiable CI trail:
1. **Commit 1 (test-only)** — adds a test that exercises the bug. CI runs it → test fails → red X.
2. **Commit 2 (fix)** — adds the minimal fix. CI runs the same test → test passes → green check.
The red-then-green sequence in the commit history proves the test is valid.
## Input
The user provides a bug description as an argument. If no description is given, ask the user to describe the bug before proceeding.
Bug description: $ARGUMENTS
## Step 0 — Setup
Create an isolated branch from main:
```bash
git fetch origin main
git checkout -b fix/<bug-name> origin/main
```
## Step 1 — Red: Failing Test Only
Write a test that reproduces the bug. **Do NOT write any fix code.**
### Choosing the Test Framework
| Bug type | Framework | File location |
| --------------------------------- | ---------- | ------------------------------- |
| Logic, utils, stores, composables | Vitest | `src/**/*.test.ts` (colocated) |
| UI interaction, canvas, workflows | Playwright | `browser_tests/tests/*.spec.ts` |
For Playwright tests, follow the `/writing-playwright-tests` skill for patterns, fixtures, and tags.
### Rules
- The test MUST fail against the current codebase (this is the whole point)
- Do NOT modify any source code outside of test files
- Do NOT include any fix, workaround, or behavioral change
- Do NOT add unrelated tests or refactor existing tests
- Keep the test minimal — only what is needed to reproduce the bug
- Avoid common anti-patterns — see `reference/testing-anti-patterns.md`
### Vitest Example
```typescript
// src/utils/pathUtil.test.ts
import { describe, expect, it } from 'vitest'
import { resolveModelPath } from './pathUtil'
describe('resolveModelPath', () => {
it('handles absolute paths from folder_paths API', () => {
const result = resolveModelPath(
'/absolute/models',
'/absolute/models/checkpoints'
)
expect(result).toBe('/absolute/models/checkpoints')
})
})
```
### Playwright Example
```typescript
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test.describe('Model Download', { tag: ['@smoke'] }, () => {
test('downloads model when path is absolute', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('missing-model')
const downloadBtn = comfyPage.page.getByTestId('download-model-button')
await downloadBtn.click()
await expect(comfyPage.page.getByText('Download complete')).toBeVisible()
})
})
```
### Verify Locally First
Run the test locally before pushing to confirm it fails for the right reason:
```bash
# Vitest
pnpm test:unit -- <test-file>
# Playwright
pnpm test:browser:local -- --grep "<test name>"
```
If the test passes locally, it does not reproduce the bug — revisit your test before pushing.
### Quality Checks and Commit
```bash
pnpm typecheck
pnpm lint
pnpm format:check
git add <test-files-only>
git commit -m "test: add failing test for <concise bug description>"
git push -u origin HEAD
```
### Verify CI Failure
```bash
gh run list --branch $(git branch --show-current) --limit 1
```
**STOP HERE.** Inform the user of the CI status and wait for confirmation before proceeding to Step 2.
- If CI passes: the test does not catch the bug. Revisit the test.
- If CI fails for unrelated reasons: investigate and fix the test setup, not the bug.
- If CI fails because the test correctly catches the bug: proceed to Step 2.
## Step 2 — Green: Minimal Fix
Write the minimum code change needed to make the failing test pass.
### Rules
- Do NOT modify, weaken, or delete the test from Step 1 — it is immutable. If the test needs changes, restart from Step 1 and re-prove the red.
- Do NOT add new tests (tests were finalized in Step 1)
- Do NOT refactor, clean up, or make "drive-by" improvements
- Do NOT modify code unrelated to the bug
- The fix should be the smallest correct change
### Quality Checks and Commit
```bash
pnpm typecheck
pnpm lint
pnpm format
git add <fix-files-only>
git commit -m "fix: <concise bug description>"
git push
```
### Verify CI Pass
```bash
gh run list --branch $(git branch --show-current) --limit 1
```
- If CI passes: the fix is verified. Proceed to PR creation.
- If CI fails: investigate and fix. Do NOT change the test from Step 1.
## Step 3 — Open Pull Request
```bash
gh pr create --title "fix: <description>" --body "$(cat <<'EOF'
## Summary
<Brief explanation of the bug and root cause>
- Fixes #<issue-number>
## Red-Green Verification
| Commit | CI Status | Purpose |
|--------|-----------|---------|
| `test: ...` | :red_circle: Red | Proves the test catches the bug |
| `fix: ...` | :green_circle: Green | Proves the fix resolves the bug |
## Test Plan
- [ ] CI red on test-only commit
- [ ] CI green on fix commit
- [ ] Added/updated E2E regression under `browser_tests/` or explained why not applicable
- [ ] Manual verification (if applicable)
EOF
)"
```
## Gotchas
### CI fails on test commit for unrelated reasons
Lint, typecheck, or other tests may fail — not just your new test. Check the CI logs carefully. If the failure is unrelated, fix it in a separate commit before the `test:` commit so the red X is clearly attributable to your test.
### Test passes when it should fail
The bug may only manifest under specific conditions (e.g., Windows paths, external model directories, certain workflow structures). Make sure your test setup matches the actual bug scenario. Check that you're not accidentally testing the happy path.
### Flaky Playwright tests
If your e2e test is intermittent, it doesn't prove anything. Use retrying assertions (`toBeVisible`, `toHaveText`) instead of `waitForTimeout`. See the `/writing-playwright-tests` skill for anti-patterns.
### Pre-existing CI failures on main
If main itself is red, branch from the last green commit or fix the pre-existing failure first. A red-green proof is meaningless if the baseline is already red.
## Reference
| Resource | Path |
| --------------------- | -------------------------------------------------- |
| Unit test framework | Vitest (`src/**/*.test.ts`) |
| E2E test framework | Playwright (`browser_tests/tests/*.spec.ts`) |
| E2E fixtures | `browser_tests/fixtures/` |
| E2E assets | `browser_tests/assets/` |
| Playwright skill | `.claude/skills/writing-playwright-tests/SKILL.md` |
| Unit CI | `.github/workflows/ci-tests-unit.yaml` |
| E2E CI | `.github/workflows/ci-tests-e2e.yaml` |
| Lint CI | `.github/workflows/ci-lint-format.yaml` |
| Testing anti-patterns | `reference/testing-anti-patterns.md` |
| Related skill | `.claude/skills/perf-fix-with-proof/SKILL.md` |

View File

@@ -1,214 +0,0 @@
# Testing Anti-Patterns for Red-Green Fixes
Common mistakes that undermine the red-green proof. Avoid these when writing the test commit (Step 1).
## Testing Implementation Details
Test observable behavior, not internal state.
**Bad** — coupling to internals:
```typescript
it('uses cache internally', () => {
const service = new UserService()
service.getUser(1)
expect(service._cache.has(1)).toBe(true) // Implementation detail
})
```
**Good** — testing through the public interface:
```typescript
it('returns same user on repeated calls', async () => {
const service = new UserService()
const user1 = await service.getUser(1)
const user2 = await service.getUser(1)
expect(user1).toBe(user2) // Behavior, not implementation
})
```
Why this matters for red-green: if your test is coupled to internals, a valid fix that changes the implementation may break the test — even though the bug is fixed. The green commit should only require changing source code, not rewriting the test.
## Assertion-Free Tests
Every test must assert something meaningful. A test without assertions always passes — it cannot produce the red X needed in Step 1.
**Bad**:
```typescript
it('processes the download', () => {
processDownload('/models/checkpoints', 'model.safetensors')
// No expect()!
})
```
**Good**:
```typescript
it('processes the download to correct path', () => {
const result = processDownload('/models/checkpoints', 'model.safetensors')
expect(result.savePath).toBe('/models/checkpoints/model.safetensors')
})
```
## Over-Mocking
Mock only system boundaries (network, filesystem, Electron APIs). If you mock the module under test, you are testing your mocks — the test will not detect the real bug.
**Bad** — mocking everything:
```typescript
vi.mock('./pathResolver')
vi.mock('./validator')
vi.mock('./downloader')
it('downloads model', () => {
// This only tests that mocks were called, not that the bug exists
})
```
**Good** — mock only the boundary:
```typescript
vi.mock('./electronAPI') // Boundary: Electron IPC
it('resolves absolute path correctly', () => {
const result = resolveModelPath('/root/models', '/root/models/checkpoints')
expect(result).toBe('/root/models/checkpoints')
})
```
See also: [Don't Mock What You Don't Own](https://hynek.me/articles/what-to-mock-in-5-mins/)
## Giant Tests
A test that covers the entire flow makes it hard to pinpoint which part catches the bug. Keep it focused — one concept per test.
**Bad**:
```typescript
it('full model download flow', async () => {
// 80 lines: load workflow, open dialog, select model,
// click download, verify path, check progress, confirm completion
})
```
**Good**:
```typescript
it('resolves absolute savePath without nesting under modelsDirectory', () => {
const result = getLocalSavePath(
'/models',
'/models/checkpoints',
'file.safetensors'
)
expect(result).toBe('/models/checkpoints/file.safetensors')
})
```
If the bug is in path resolution, test path resolution — not the entire download flow.
## Test Duplication
Duplicated test code hides what actually differs between cases. Use parameterized tests.
**Bad**:
```typescript
it('resolves checkpoints path', () => {
expect(resolve('/models', '/models/checkpoints', 'a.safetensors')).toBe(
'/models/checkpoints/a.safetensors'
)
})
it('resolves loras path', () => {
expect(resolve('/models', '/models/loras', 'b.safetensors')).toBe(
'/models/loras/b.safetensors'
)
})
```
**Good**:
```typescript
it.each([
['/models/checkpoints', 'a.safetensors', '/models/checkpoints/a.safetensors'],
['/models/loras', 'b.safetensors', '/models/loras/b.safetensors']
])('resolves %s/%s to %s', (dir, file, expected) => {
expect(resolve('/models', dir, file)).toBe(expected)
})
```
## Flaky Tests
A flaky test cannot prove anything — it may show red for reasons unrelated to the bug, or green despite the bug still existing.
**Common causes in this codebase:**
| Cause | Fix |
| -------------------------------------- | --------------------------------------- |
| Missing `nextFrame()` after canvas ops | Add `await comfyPage.nextFrame()` |
| `waitForTimeout` instead of assertions | Use `toBeVisible()`, `toHaveText()` |
| Shared state between tests | Isolate with `afterEach` / `beforeEach` |
| Timing-dependent logic | Use `expect.poll()` or `toPass()` |
## Gaming the Red-Green Process
These are ways the red-green proof gets invalidated during Step 2 (the fix commit). The test from Step 1 is immutable — if any of these happen, restart from Step 1.
**Weakening the assertion to make it pass:**
```typescript
// Step 1 (red) — strict assertion
expect(result).toBe('/external/drive/models/checkpoints/file.safetensors')
// Step 2 (green) — weakened to pass without a real fix
expect(result).toBeDefined() // This proves nothing
```
**Updating snapshots to bless the bug:**
```bash
# Instead of fixing the code, just updating the snapshot to match buggy output
pnpm test:unit -- --update
```
If a snapshot needs updating, the fix should change the code behavior, not the expected output.
**Adding mocks in Step 2 that hide the failure:**
```typescript
// Step 2 adds a mock that didn't exist in Step 1
vi.mock('./pathResolver', () => ({
resolve: () => '/expected/path' // Hardcoded to pass
}))
```
Step 2 should only change source code — not test infrastructure.
## Testing the Happy Path Only
The red-green pattern specifically requires the test to exercise the **broken path**. If you only test the case that already works, the test will pass (green) on Step 1 — defeating the purpose.
**Bad** — testing the default case that works:
```typescript
it('downloads to default models directory', () => {
// This already works — it won't produce a red X
const result = resolve('/models', 'checkpoints', 'file.safetensors')
expect(result).toBe('/models/checkpoints/file.safetensors')
})
```
**Good** — testing the case that is actually broken:
```typescript
it('downloads to external models directory configured via extra_model_paths', () => {
// This is the broken case — absolute path from folder_paths API
const result = resolve(
'/models',
'/external/drive/models/checkpoints',
'file.safetensors'
)
expect(result).toBe('/external/drive/models/checkpoints/file.safetensors')
})
```

View File

@@ -1,83 +0,0 @@
---
name: regenerating-screenshots
description: 'Creates a PR to regenerate Playwright screenshot expectations. Use when screenshot tests are failing on main or PRs due to stale golden images. Triggers on: regen screenshots, regenerate screenshots, update expectations, fix screenshot tests.'
---
# Regenerating Playwright Screenshot Expectations
Automates the process of triggering the `PR: Update Playwright Expectations`
GitHub Action by creating a labeled PR from `origin/main`.
## Steps
1. **Fetch latest main**
```bash
git fetch origin main
```
2. **Create a timestamped branch** from `origin/main`
Format: `regen-screenshots/YYYY-MM-DDTHH` (hour resolution, local time)
```bash
git checkout -b regen-screenshots/<datetime> origin/main
```
3. **Create an empty commit**
```bash
git commit --allow-empty -m "test: regenerate screenshot expectations"
```
4. **Push the branch**
```bash
git push origin regen-screenshots/<datetime>
```
5. **Generate a poem** about regenerating screenshots. Be creative — a
new, unique poem every time. Short (48 lines). Can be funny, wistful,
epic, haiku-style, limerick, sonnet fragment — vary the form.
6. **Create the PR** with the poem as the body (no label yet).
Write the poem to a temp file and use `--body-file`:
```bash
# Write poem to temp file
# Create PR:
gh pr create \
--base main \
--head regen-screenshots/<datetime> \
--title "test: regenerate screenshot expectations" \
--body-file <temp-file>
```
7. **Add the label** as a separate step to trigger the GitHub Action.
The `labeled` event only fires when a label is added after PR
creation, not when applied during creation via `--label`.
Use the GitHub API directly (`gh pr edit --add-label` fails due to
deprecated Projects Classic GraphQL errors):
```bash
gh api repos/{owner}/{repo}/issues/<pr-number>/labels \
-f "labels[]=New Browser Test Expectations" --method POST
```
8. **Report the result** to the user:
- PR URL
- Branch name
- Note that the GitHub Action will run automatically and commit
updated screenshots to the branch.
## Notes
- The `New Browser Test Expectations` label triggers the
`pr-update-playwright-expectations.yaml` workflow.
- The workflow runs Playwright with `--update-snapshots`, commits results
back to the PR branch, then removes the label.
- This is fire-and-forget — no need to wait for or monitor the Action.
- Always return to the original branch/worktree state after pushing.

View File

@@ -1,361 +0,0 @@
---
name: ticket-intake
description: 'Parse ticket URL (Notion or GitHub), extract all data, initialize pipeline run. Use when starting work on a new ticket or when asked to pick up a ticket.'
---
# Ticket Intake
Parses a ticket URL from supported sources (Notion or GitHub), extracts all relevant information, and creates a ticket in the pipeline API.
> **🚨 CRITICAL REQUIREMENT**: This skill MUST register the ticket in the Pipeline API and update the source (Notion/GitHub). If these steps are skipped, the entire pipeline breaks. See [Mandatory API Calls](#mandatory-api-calls-execute-all-three) below.
## Supported Sources
| Source | URL Pattern | Provider File |
| ------ | --------------------------------------------------- | --------------------- |
| Notion | `https://notion.so/...` `https://www.notion.so/...` | `providers/notion.md` |
| GitHub | `https://github.com/{owner}/{repo}/issues/{n}` | `providers/github.md` |
## Quick Start
When given a ticket URL:
1. **Detect source type** from URL pattern
2. **Load provider-specific logic** from `providers/` directory
3. Fetch ticket content via appropriate API
4. Extract and normalize properties to common schema
5. **Register ticket in pipeline API** ← MANDATORY
6. **Update source** (Notion status / GitHub comment) ← MANDATORY
7. **Run verification script** to confirm API registration
8. Output summary and handoff to `research-orchestrator`
## Configuration
Uses the **production API** by default. No configuration needed for read operations.
**Defaults (no setup required):**
- API URL: `https://api-gateway-856475788601.us-central1.run.app`
- Read-only endpoints at `/public/*` require no authentication
**For write operations** (transitions, creating tickets), set:
```bash
export PIPELINE_API_KEY="..." # Get from GCP Secret Manager or ask admin
```
**Optional (for local working artifacts):**
```bash
PIPELINE_DIR="${PIPELINE_DIR:-$HOME/repos/ticket-to-pr-pipeline}"
```
## Mandatory API Calls (Execute ALL Three)
**⚠️ These three API calls are the ENTIRE POINT of this skill. Without them, the ticket is invisible to the pipeline, downstream skills will fail, and Notion status won't update.**
**You MUST make these HTTP requests.** Use `curl` from bash — do not just read this as documentation.
### Call 1: Create Ticket
```bash
API_URL="${PIPELINE_API_URL:-https://api-gateway-856475788601.us-central1.run.app}"
API_KEY="${PIPELINE_API_KEY}"
curl -s -X POST "${API_URL}/v1/tickets" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"notion_page_id": "NOTION_PAGE_UUID_HERE",
"title": "TICKET_TITLE_HERE",
"source": "notion",
"metadata": {
"description": "DESCRIPTION_HERE",
"priority": "High",
"labels": [],
"acceptanceCriteria": []
}
}'
```
Save the returned `id` — you need it for the next two calls.
### Call 2: Transition to RESEARCH
```bash
TICKET_ID="id-from-step-1"
curl -s -X POST "${API_URL}/v1/tickets/${TICKET_ID}/transition" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"to_state": "RESEARCH",
"reason": "Intake complete, starting research"
}'
```
### Call 3: Queue Source Update
```bash
curl -s -X POST "${API_URL}/v1/sync/queue" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-H "X-Agent-ID: ${AGENT_ID:-amp-agent}" \
-d '{
"ticket_id": "TICKET_ID_HERE",
"action": "update_status",
"payload": { "status": "In Progress" },
"priority": "normal"
}'
```
> **Note:** The action MUST be `"update_status"` (not `"UPDATE_NOTION_STATUS"`). Valid actions: `update_status`, `update_pr_url`, `mark_done`.
### TypeScript Equivalent (if using pipeline client)
```typescript
import { PipelineClient } from '@pipeline/client'
const client = new PipelineClient({
apiUrl:
process.env.PIPELINE_API_URL ||
'https://api-gateway-856475788601.us-central1.run.app',
agentId: process.env.AGENT_ID!
})
const ticket = await client.createTicket({
notion_page_id: pageId,
title: ticketTitle,
source: 'notion',
metadata: { description, priority, labels, acceptanceCriteria }
})
await client.transitionState(
ticket.id,
'RESEARCH',
'Intake complete, starting research'
)
await client.queueSync(ticket.id, 'update_status', { status: 'In Progress' })
```
## Workflow
### Step 1: Detect Source Type
Parse the URL to determine source:
```javascript
if (url.includes('notion.so')) {
source = 'notion'
// Load providers/notion.md
} else if (url.match(/github\.com\/[^\/]+\/[^\/]+\/issues\/\d+/)) {
source = 'github'
// Load providers/github.md
} else {
// Error: Unsupported URL format
}
```
### Step 2: Load Provider and Fetch Data
Read the appropriate provider file for source-specific instructions:
- **Notion**: `providers/notion.md` - Uses Notion MCP, handles Slack links
- **GitHub**: `providers/github.md` - Uses `gh` CLI, handles Dosu comments
Follow the provider's instructions for:
- Fetching content
- Extracting properties
- **Updating the source** (Notion status → "In Progress", Assignee → pipeline owner)
### Step 3: Normalize to Common Schema
All providers must extract normalized ticket data following `schema.md`:
```json
{
"id": "abc12345",
"url": "https://...",
"source": "notion | github",
"title": "Ticket title",
"description": "Full description",
"status": "Not Started",
"assignee": "username",
"priority": "High",
"area": "UI",
"labels": ["bug", "frontend"],
"acceptanceCriteria": ["Criterion 1", "Criterion 2"],
"fetchedAt": "2024-01-15T10:30:00Z"
}
```
### Step 4: Register Ticket in Pipeline API (MANDATORY — DO NOT SKIP)
**Execute all three API calls from [Mandatory API Calls](#mandatory-api-calls-execute-all-three) above.**
This is not optional. This is not documentation. You MUST make these HTTP requests right now.
1. `createTicket()` → save the returned ticket ID
2. `transitionState(id, 'RESEARCH')` → confirm state changed
3. `queueSync(id, 'update_status', { status: 'In Progress' })` → confirm queued
**If any call fails**, retry once. If it still fails, report the error prominently — do NOT silently continue.
### Step 5: Run Verification Script
After making the API calls, run the verification script to confirm everything worked:
```bash
bash scripts/verify-intake.sh TICKET_ID_OR_NOTION_PAGE_ID
```
**If the script is not available locally**, verify manually via the public API:
```bash
curl -s "${API_URL}/public/tickets/${TICKET_ID}" | jq '{id, state, title, notion_page_id}'
```
Expected output:
```json
{
"id": "...",
"state": "RESEARCH",
"title": "...",
"notion_page_id": "..."
}
```
**If `state` is not `RESEARCH`, go back to Step 4 and complete the missing calls.**
### Step 6: Output Summary and Handoff
Print a clear summary:
```markdown
## Ticket Intake Complete
**Source:** Notion | GitHub
**Title:** [Ticket title]
**ID:** abc12345
**Status:** In Progress (queued)
**Priority:** High
**Area:** UI
### Description
[Brief description or first 200 chars]
### Acceptance Criteria
- [ ] Criterion 1
- [ ] Criterion 2
### Links
- **Ticket:** [Original URL]
- **Slack:** [Slack thread content fetched via slackdump] (Notion only)
### Pipeline
- **API Ticket ID:** abc12345
- **State:** RESEARCH
- **Verified:** ✅ (via verify-intake.sh or public API)
```
**After printing the summary, immediately handoff** to continue the pipeline. Use the `handoff` tool with all necessary context (ticket ID, source, title, description, slack context if any):
> **Handoff goal:** "Continue pipeline for ticket {ID} ({title}). Ticket is in RESEARCH state. Load skill: `research-orchestrator` to begin research phase. Ticket data: source={source}, notion_page_id={pageId}, priority={priority}. {slack context summary if available}"
**Do NOT wait for human approval to proceed.** The intake phase is complete — handoff immediately.
## Error Handling
### Unsupported URL
```
❌ Unsupported ticket URL format.
Supported formats:
- Notion: https://notion.so/... or https://www.notion.so/...
- GitHub: https://github.com/{owner}/{repo}/issues/{number}
Received: [provided URL]
```
### Provider-Specific Errors
See individual provider files for source-specific error handling:
- `providers/notion.md` - Authentication, page not found
- `providers/github.md` - Auth, rate limits, issue not found
### Missing Properties
Continue with available data and note what's missing:
```
⚠️ Some properties unavailable:
- Priority: not found (using default: Medium)
- Area: not found
Proceeding with available data...
```
### API Call Failures
```
❌ Pipeline API call failed: {method} {endpoint}
Status: {status}
Error: {message}
Retrying once...
❌ Retry also failed. INTAKE IS INCOMPLETE.
The ticket was NOT registered in the pipeline.
Downstream skills will not work until this is fixed.
```
## Notes
- This skill focuses ONLY on intake — it does not do research
- Slack thread content is fetched automatically via the `slackdump` skill — no manual copy-paste needed
- ALL API calls (createTicket, transitionState, queueSync) are MANDATORY — never skip them
- The `queueSync` action must be `"update_status"`, NOT `"UPDATE_NOTION_STATUS"`
- Pipeline state is tracked via the API, not local files
- Working artifacts (research-report.md, plan.md) can be saved locally to `$PIPELINE_DIR/runs/{ticket-id}/`
- The `source` field in the ticket determines which research strategies to use
## API Client Reference
### Available Methods
| Method | Description |
| ----------------------------------------------------------- | ------------------------------------------------------------------- |
| `createTicket({ notion_page_id, title, source, metadata })` | Create a new ticket in the API |
| `getTicket(id)` | Retrieve a ticket by ID |
| `findByNotionId(notionPageId)` | Look up a ticket by its Notion page ID |
| `listTickets({ state, agent_id, limit, offset })` | List tickets with optional filters |
| `transitionState(id, state, reason)` | Move ticket to a new state (e.g., `'RESEARCH'`) |
| `setPRCreated(id, prUrl)` | Mark ticket as having a PR created |
| `queueSync(id, action, payload)` | Queue a sync action (`update_status`, `update_pr_url`, `mark_done`) |
| `registerBranch(id, branch, repo)` | Register working branch for automatic PR detection |
### Error Handling
```typescript
import { PipelineClient, PipelineAPIError } from '@pipeline/client';
try {
await client.createTicket({ ... });
} catch (error) {
if (error instanceof PipelineAPIError) {
console.error(`API Error ${error.status}: ${error.message}`);
}
throw error;
}
```

View File

@@ -1,194 +0,0 @@
# GitHub Provider - Ticket Intake
Provider-specific logic for ingesting tickets from GitHub Issues.
## URL Pattern
```
https://github.com/{owner}/{repo}/issues/{number}
https://www.github.com/{owner}/{repo}/issues/{number}
```
Extract: `owner`, `repo`, `issue_number` from URL.
## Prerequisites
- `gh` CLI authenticated (`gh auth status`)
- Access to the repository
## Fetch Issue Content
Use `gh` CLI to fetch issue details:
```bash
# Get issue details in JSON
gh issue view {number} --repo {owner}/{repo} --json title,body,state,labels,assignees,milestone,author,createdAt,comments,linkedPRs
# Get comments separately if needed
gh issue view {number} --repo {owner}/{repo} --comments
```
## Extract Ticket Data
Map GitHub issue fields to normalized ticket data (stored via API):
| GitHub Field | ticket.json Field | Notes |
| ------------ | ----------------- | -------------------------- |
| title | title | Direct mapping |
| body | description | Issue body/description |
| state | status | Map: open → "Not Started" |
| labels | labels | Array of label names |
| assignees | assignee | First assignee login |
| author | author | Issue author login |
| milestone | milestone | Milestone title if present |
| comments | comments | Array of comment objects |
| linkedPRs | linkedPRs | PRs linked to this issue |
### Priority Mapping
Infer priority from labels:
- `priority:critical`, `P0` → "Critical"
- `priority:high`, `P1` → "High"
- `priority:medium`, `P2` → "Medium"
- `priority:low`, `P3` → "Low"
- No priority label → "Medium" (default)
### Area Mapping
Infer area from labels:
- `area:ui`, `frontend`, `component:*` → "UI"
- `area:api`, `backend` → "API"
- `area:docs`, `documentation` → "Docs"
- `bug`, `fix` → "Bug"
- `enhancement`, `feature` → "Feature"
## Update Source
**For GitHub issues, update is optional but recommended.**
Add a comment to indicate work has started:
```bash
gh issue comment {number} --repo {owner}/{repo} --body "🤖 Pipeline started processing this issue."
```
Optionally assign to self:
```bash
gh issue edit {number} --repo {owner}/{repo} --add-assignee @me
```
Log any updates via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
githubWrites: [
...(ticket.metadata?.githubWrites || []),
{
action: 'comment',
issueNumber: 123,
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
## GitHub-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "github",
"githubOwner": "Comfy-Org",
"githubRepo": "ComfyUI_frontend",
"githubIssueNumber": 123,
"githubIssueUrl": "https://github.com/Comfy-Org/ComfyUI_frontend/issues/123",
"labels": ["bug", "area:ui", "priority:high"],
"linkedPRs": [456, 789],
"dosuComment": "..." // Extracted Dosu bot analysis if present
}
```
## Dosu Bot Detection
Many repositories use Dosu bot for automated issue analysis. Check comments for Dosu:
```bash
gh issue view {number} --repo {owner}/{repo} --comments | grep -A 100 "dosu"
```
Look for comments from:
- `dosu[bot]`
- `dosu-bot`
Extract Dosu analysis which typically includes:
- Root cause analysis
- Suggested files to modify
- Related issues/PRs
- Potential solutions
Store in ticket data via API:
```json
{
"dosuComment": {
"found": true,
"analysis": "...",
"suggestedFiles": ["src/file1.ts", "src/file2.ts"],
"relatedIssues": [100, 101]
}
}
```
## Extract Linked Issues/PRs
Parse issue body and comments for references:
- `#123` → Issue or PR reference
- `fixes #123`, `closes #123` → Linked issue
- `https://github.com/.../issues/123` → Full URL reference
Store in ticket data via API for research phase:
```json
{
"referencedIssues": [100, 101, 102],
"referencedPRs": [200, 201]
}
```
## Error Handling
### Authentication Error
```
⚠️ GitHub CLI not authenticated.
Run: gh auth login
```
### Issue Not Found
```
❌ GitHub issue not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this repository
- Run: gh auth status
```
### Rate Limiting
```
⚠️ GitHub API rate limited.
Wait a few minutes and try again.
Check status: gh api rate_limit
```

View File

@@ -1,202 +0,0 @@
# Notion Provider - Ticket Intake
Provider-specific logic for ingesting tickets from Notion.
## URL Pattern
```
https://www.notion.so/workspace/Page-Title-abc123def456...
https://notion.so/Page-Title-abc123def456...
https://www.notion.so/abc123def456...
```
Page ID is the 32-character hex string (with or without hyphens).
## Prerequisites
- Notion MCP connected and authenticated
- If not setup: `claude mcp add --transport http notion https://mcp.notion.com/mcp`
- Authenticate via `/mcp` command if prompted
## Fetch Ticket Content
Use `Notion:notion-fetch` with the page URL or ID:
```
Fetch the full page content including all properties
```
## Extract Ticket Data
Extract these properties (names may vary):
| Property | Expected Name | Type |
| ------------- | ------------------------- | ------------ |
| Title | Name / Title | Title |
| Status | Status | Select |
| Assignee | Assignee / Assigned To | Person |
| Description | - | Page content |
| Slack Link | Slack Link / Slack Thread | URL |
| GitHub PR | GitHub PR / PR Link | URL |
| Priority | Priority | Select |
| Area | Area / Category | Select |
| Related Tasks | Related Tasks | Relation |
**If properties are missing**: Note what's unavailable and continue with available data.
## Update Source (REQUIRED)
**⚠️ DO NOT SKIP THIS STEP. This is a required action, not optional.**
**⚠️ Notion Write Safety rules apply (see `$PIPELINE_DIR/docs/notion-write-safety.md` for full reference):**
- **Whitelist**: Only `Status`, `GitHub PR`, and `Assignee` fields may be written
- **Valid transitions**: Not Started → In Progress, In Progress → In Review, In Review → Done
- **Logging**: Every write attempt MUST be logged with timestamp, field, value, previous value, skill name, and success status
Use `Notion:notion-update-page` to update the ticket:
1. **Status**: Set to "In Progress" (only valid from "Not Started")
2. **Assignee**: Assign to pipeline owner (Notion ID: `175d872b-594c-81d4-ba5a-0002911c5966`)
```json
{
"page_id": "{page_id_from_ticket}",
"command": "update_properties",
"properties": {
"Status": "In Progress",
"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"
}
}
```
**After the update succeeds**, log the write via the Pipeline API:
```typescript
await client.updateTicket(ticketId, {
metadata: {
...ticket.metadata,
notionWrites: [
...(ticket.metadata?.notionWrites || []),
{
field: 'Status',
value: 'In Progress',
previousValue: 'Not Started',
at: new Date().toISOString(),
skill: 'ticket-intake',
success: true
}
]
}
})
```
If update fails, log with `success: false` and continue.
## Notion-Specific Ticket Fields
Store via API using `client.createTicket()`:
```json
{
"source": "notion",
"notionPageId": "abc123def456...",
"slackLink": "https://slack.com/...",
"relatedTasks": ["page-id-1", "page-id-2"]
}
```
## Slack Thread Handling
If a Slack link exists, use the `slackdump` skill to fetch the thread content programmatically.
### Slack URL Conversion
Notion stores Slack links in `slackMessage://` format:
```
slackMessage://comfy-organization.slack.com/CHANNEL_ID/THREAD_TS/MESSAGE_TS
```
Convert to browser-clickable format:
```
https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS_NO_DOT
```
**Example:**
- Input: `slackMessage://comfy-organization.slack.com/C075ANWQ8KS/1766022478.450909/1764772881.854829`
- Output: `https://comfy-organization.slack.com/archives/C075ANWQ8KS/p1764772881854829`
(Remove the dot from the last timestamp and prefix with `p`)
### Fetching Thread Content
Load the `slackdump` skill and use the **export-thread** workflow:
```bash
# Export thread by URL
slackdump dump "https://comfy-organization.slack.com/archives/CHANNEL_ID/pMESSAGE_TS"
# Or by colon notation (channel_id:thread_ts)
slackdump dump CHANNEL_ID:THREAD_TS
```
Save the thread content to `$RUN_DIR/slack-context.md` and include it in the ticket metadata.
> **No manual action required.** The slackdump CLI handles authentication via stored credentials at `~/.cache/slackdump/comfy-organization.bin`.
## Database Reference: Comfy Tasks
The "Comfy Tasks" database has these properties (verify via `notion-search`):
- **Status values**: Not Started, In Progress, In Review, Done
- **Team assignment**: "Frontend Team" for unassigned tickets
- **Filtering note**: Team filtering in Notion may have quirks - handle gracefully
### Pipeline Owner Details
When assigning tickets, use these identifiers:
| Platform | Identifier |
| --------------- | -------------------------------------- |
| Notion User ID | `175d872b-594c-81d4-ba5a-0002911c5966` |
| Notion Name | Christian Byrne |
| Notion Email | cbyrne@comfy.org |
| Slack User ID | U087MJCDHHC |
| GitHub Username | christian-byrne |
**To update Assignee**, use the Notion User ID (not name):
```
properties: {"Assignee": "175d872b-594c-81d4-ba5a-0002911c5966"}
```
### Finding Active Tickets
To list your active tickets:
```
Use Notion:notion-search for "Comfy Tasks"
Filter by Assignee = current user OR Team = "Frontend Team"
```
## Error Handling
### Authentication Error
```
⚠️ Notion authentication required.
Run: claude mcp add --transport http notion https://mcp.notion.com/mcp
Then authenticate via /mcp command.
```
### Page Not Found
```
❌ Notion page not found or inaccessible.
- Check the URL is correct
- Ensure you have access to this page
- Try re-authenticating via /mcp
```

View File

@@ -1,81 +0,0 @@
# Ticket Schema
Common schema for normalized ticket data across all sources. This data is stored and retrieved via the Pipeline API, not local files.
## Ticket Data Schema
```json
{
// Required fields (all sources)
"id": "string", // Unique identifier (short form)
"url": "string", // Original URL
"source": "notion | github", // Source type
"title": "string", // Ticket title
"description": "string", // Full description/body
"fetchedAt": "ISO8601", // When ticket was fetched
// Common optional fields
"status": "string", // Current status
"assignee": "string", // Assigned user
"priority": "string", // Priority level
"area": "string", // Category/area
"labels": ["string"], // Tags/labels
"acceptanceCriteria": ["string"] // List of AC items
// Source-specific fields (see providers)
// Notion: notionPageId, slackLink, relatedTasks, notionWrites
// GitHub: githubOwner, githubRepo, githubIssueNumber, linkedPRs, dosuComment, referencedIssues
}
```
## Ticket State Schema (via API)
State is managed via the Pipeline API using `client.transitionState()`:
```json
{
"ticketId": "string",
"state": "intake | research | planning | implementation | pr_created | done | failed",
"stateChangedAt": "ISO8601",
// Timestamps tracked by API
"createdAt": "ISO8601",
"updatedAt": "ISO8601"
}
```
## Priority Normalization
All sources should normalize to these values:
| Normalized | Description |
| ---------- | ------------------------- |
| Critical | Production down, security |
| High | Blocking work, urgent |
| Medium | Normal priority (default) |
| Low | Nice to have, backlog |
## Status Normalization
Pipeline tracks these statuses internally:
| Status | Description |
| -------------- | ---------------------------- |
| research | Gathering context |
| planning | Creating implementation plan |
| implementation | Writing code |
| review | Code review in progress |
| qa | Quality assurance |
| done | PR merged or completed |
## ID Generation
IDs are generated by the API when creating tickets. For reference:
- **Notion**: First 8 characters of page ID
- **GitHub**: `gh-{owner}-{repo}-{issue_number}` (sanitized)
Examples:
- Notion: `abc12345`
- GitHub: `gh-comfy-org-frontend-123`

View File

@@ -1,203 +0,0 @@
---
name: writing-playwright-tests
description: 'Writes Playwright e2e tests for ComfyUI_frontend. Use when creating, modifying, or debugging browser tests. Triggers on: playwright, e2e test, browser test, spec file.'
---
# Writing Playwright Tests for ComfyUI_frontend
## Golden Rules
1. **ALWAYS look at existing tests first.** Search `browser_tests/tests/` for similar patterns before writing new tests.
2. **ALWAYS read the fixture code.** The APIs are in `browser_tests/fixtures/` - read them directly instead of guessing.
3. **Use premade JSON workflow assets** instead of building workflows programmatically.
- Assets live in `browser_tests/assets/`
- Load with `await comfyPage.workflow.loadWorkflow('feature/my_workflow')`
- Create new assets by starting with `browser_tests/assets/default.json` and manually editing the JSON to match your desired graph state
## Vue Nodes vs LiteGraph: Decision Guide
Choose based on **what you're testing**, not personal preference:
| Testing... | Use | Why |
| ---------------------------------------------- | -------------------------------- | ---------------------------------------- |
| Vue-rendered node UI, DOM widgets, CSS states | `comfyPage.vueNodes.*` | Nodes are DOM elements, use locators |
| Canvas interactions, connections, legacy nodes | `comfyPage.nodeOps.*` | Canvas-based, use coordinates/references |
| Both in same test | Pick primary, minimize switching | Avoid confusion |
**Vue Nodes requires explicit opt-in:**
```typescript
await comfyPage.settings.setSetting('Comfy.VueNodes.Enabled', true)
await comfyPage.vueNodes.waitForNodes()
```
**Vue Node state uses CSS classes:**
```typescript
const BYPASS_CLASS = /before:bg-bypass\/60/
await expect(node).toHaveClass(BYPASS_CLASS)
```
## Common Issues
These are frequent causes of flaky tests - check them first, but investigate if they don't apply:
| Symptom | Common Cause | Typical Fix |
| ----------------------------------- | ------------------------- | -------------------------------------------------------------------------------------- |
| Test passes locally, fails in CI | Missing nextFrame() | Add `await comfyPage.nextFrame()` after canvas ops (not needed after `loadWorkflow()`) |
| Keyboard shortcuts don't work | Missing focus | Add `await comfyPage.canvas.click()` first |
| Double-click doesn't trigger | Timing too fast | Add `{ delay: 5 }` option |
| Elements end up in wrong position | Drag animation incomplete | Use `{ steps: 10 }` not `{ steps: 1 }` |
| Widget value wrong after drag-drop | Upload incomplete | Add `{ waitForUpload: true }` |
| Test fails when run with others | Test pollution | Add `afterEach` with `resetView()` |
| Local screenshots don't match CI | Platform differences | Screenshots are Linux-only, use PR label |
| `subtree intercepts pointer events` | Canvas overlay (z-999) | Use `dispatchEvent` on the DOM element to bypass overlay |
| Context menu empty / wrong items | Node not selected | Select node first: `vueNodes.selectNode()` or `nodeRef.click('title')` |
| `navigateIntoSubgraph` timeout | Node too small in asset | Use node size `[400, 200]` minimum in test asset JSON |
## Test Tags
Add appropriate tags to every test:
| Tag | When to Use |
| ------------- | ----------------------------------------- |
| `@smoke` | Quick essential tests |
| `@slow` | Tests > 10 seconds |
| `@screenshot` | Visual regression tests |
| `@canvas` | Canvas interactions |
| `@node` | Node-related |
| `@widget` | Widget-related |
| `@mobile` | Mobile viewport (runs on Pixel 5 project) |
| `@2x` | HiDPI tests (runs on 2x scale project) |
```typescript
test.describe('Feature', { tag: ['@screenshot', '@canvas'] }, () => {
```
## Retry Patterns
**Never use `waitForTimeout`** - it's always wrong.
| Pattern | Use Case |
| ------------------------ | ---------------------------------------------------- |
| Auto-retrying assertions | `toBeVisible()`, `toHaveText()`, etc. (prefer these) |
| `expect.poll()` | Single value polling |
| `expect().toPass()` | Multiple assertions that must all pass |
```typescript
// Prefer auto-retrying assertions when possible
await expect(node).toBeVisible()
// Single value polling
await expect.poll(() => widget.getValue(), { timeout: 2000 }).toBe(100)
// Multiple conditions
await expect(async () => {
expect(await node1.getValue()).toBe('foo')
expect(await node2.getValue()).toBe('bar')
}).toPass({ timeout: 2000 })
```
## Screenshot Baselines
- **Screenshots are Linux-only.** Don't commit local screenshots.
- **To update baselines:** Add PR label `New Browser Test Expectations`
- **Mask dynamic content:**
```typescript
await expect(comfyPage.canvas).toHaveScreenshot('page.png', {
mask: [page.locator('.timestamp')]
})
```
## CI Debugging
1. Download artifacts from failed CI run
2. Extract and view trace: `npx playwright show-trace trace.zip`
3. CI deploys HTML report to Cloudflare Pages (link in PR comment)
4. Reproduce CI: `CI=true pnpm test:browser`
5. Local runs: `pnpm test:browser:local`
## Anti-Patterns
Avoid these common mistakes:
1. **Arbitrary waits** - Use retrying assertions instead
```typescript
// ❌ await page.waitForTimeout(500)
// ✅ await expect(element).toBeVisible()
```
2. **Implementation-tied selectors** - Use test IDs or semantic selectors
```typescript
// ❌ page.locator('div.container > button.btn-primary')
// ✅ page.getByTestId('submit-button')
```
3. **Missing nextFrame after canvas ops** - Canvas needs sync time
```typescript
await node.drag({ x: 50, y: 50 })
await comfyPage.nextFrame() // Required
```
4. **Shared state between tests** - Tests must be independent
```typescript
// ❌ let sharedData // Outside test
// ✅ Define state inside each test
```
## Quick Start Template
```typescript
// Path depends on test file location - adjust '../' segments accordingly
import {
comfyPageFixture as test,
comfyExpect as expect
} from '../fixtures/ComfyPage'
test.describe('FeatureName', { tag: ['@canvas'] }, () => {
test.afterEach(async ({ comfyPage }) => {
await comfyPage.canvasOps.resetView()
})
test('should do something', async ({ comfyPage }) => {
await comfyPage.workflow.loadWorkflow('myWorkflow')
const node = (await comfyPage.nodeOps.getNodeRefsByTitle('KSampler'))[0]
// ... test logic
await expect(comfyPage.canvas).toHaveScreenshot('expected.png')
})
})
```
## Finding Patterns
```bash
# Find similar tests
grep -r "KSampler" browser_tests/tests/
# Find usage of a fixture method
grep -r "loadWorkflow" browser_tests/tests/
# Find tests with specific tag
grep -r '@screenshot' browser_tests/tests/
```
## Key Files to Read
| Purpose | Path |
| ----------------- | ------------------------------------------ |
| Main fixture | `browser_tests/fixtures/ComfyPage.ts` |
| Helper classes | `browser_tests/fixtures/helpers/` |
| Component objects | `browser_tests/fixtures/components/` |
| Test selectors | `browser_tests/fixtures/selectors.ts` |
| Vue Node helpers | `browser_tests/fixtures/VueNodeHelpers.ts` |
| Test assets | `browser_tests/assets/` |
| Existing tests | `browser_tests/tests/` |
**Read the fixture code directly** - it's the source of truth for available methods.

View File

@@ -1,143 +0,0 @@
---
name: writing-storybook-stories
description: 'Write or update Storybook stories for Vue components in ComfyUI_frontend. Use when adding, modifying, reviewing, or debugging `.stories.ts` files, Storybook docs, component demos, or visual catalog entries in `src/` or `apps/desktop-ui/`.'
---
# Write Storybook Stories for ComfyUI_frontend
## Workflow
1. !!!!IMPORTANT Confirm the worktree is on a `feat/*` or `fix/*` branch. Base PRs on the local `main`, not a fork branch.
2. Read the component source first. Understand props, emits, slots, exposed methods, and any supporting types or composables.
3. Read nearby stories before writing anything.
- Search stories: `rg --files src apps | rg '\.stories\.ts$'`
- Inspect title patterns: `rg -n "title:\\s*'" src apps --glob '*.stories.ts'`
4. If a Figma link is provided, list the states you need to cover before writing stories.
5. Co-locate the story file with the component: `ComponentName.stories.ts`.
6. Add each variation on separate stories, except hover state. this should be automatically applied by the implementation and not require a separate story.
7. Run Storybook and validation checks before handing off.
## Match Local Conventions
- Copy the closest neighboring story instead of forcing one universal template.
- Most repo stories use `@storybook/vue3-vite`. Some stories under `apps/desktop-ui` still use `@storybook/vue3`; keep the local convention for that area.
- Add `tags: ['autodocs']` unless the surrounding stories in that area intentionally omit it.
- Use `ComponentPropsAndSlots<typeof Component>` when it helps with prop and slot typing.
- Keep `render` functions stateful when needed. Use `ref()`, `computed()`, and `toRefs(args)` instead of mutating Storybook args directly.
- Use `args.default` or other slot-shaped args when the component content is provided through slots.
- Use `ComponentExposed` only when a component's exposed API breaks the normal Storybook typing.
- Add decorators for realistic width or background context when the component needs it.
## Title Patterns
Do not invent titles from scratch when a close sibling story already exists. Match the nearest domain pattern.
| Component area | Typical title pattern |
| ------------------------------------------------------- | ------------------------------------ |
| `src/components/ui/button/Button.vue` | `Components/Button/Button` |
| `src/components/ui/input/Input.vue` | `Components/Input` |
| `src/components/ui/search-input/SearchInput.vue` | `Components/Input/SearchInput` |
| `src/components/common/SearchBox.vue` | `Components/Input/SearchBox` |
| `src/renderer/extensions/vueNodes/widgets/components/*` | `Widgets/<WidgetName>` |
| `src/platform/assets/components/*` | `Platform/Assets/<ComponentName>` |
| `apps/desktop-ui/src/components/*` | `Desktop/Components/<ComponentName>` |
| `apps/desktop-ui/src/views/*` | `Desktop/Views/<ViewName>` |
If multiple patterns seem plausible, follow the closest sibling story in the same folder tree.
## Common Story Shapes
### Stateful input or `v-model`
```typescript
export const Default: Story = {
render: (args) => ({
components: { MyComponent },
setup() {
const { disabled, size } = toRefs(args)
const value = ref('Hello world')
return { value, disabled, size }
},
template:
'<MyComponent v-model="value" :disabled="disabled" :size="size" />'
})
}
```
### Slot-driven content
```typescript
const meta: Meta<ComponentPropsAndSlots<typeof Button>> = {
argTypes: {
default: { control: 'text' }
},
args: {
default: 'Button'
}
}
export const SingleButton: Story = {
render: (args) => ({
components: { Button },
setup() {
return { args }
},
template: '<Button v-bind="args">{{ args.default }}</Button>'
})
}
```
### Variants or edge cases grid
```typescript
export const AllVariants: Story = {
render: () => ({
components: { MyComponent },
template: `
<div class="grid gap-4 sm:grid-cols-2">
<MyComponent />
<MyComponent disabled />
<MyComponent loading />
<MyComponent invalid />
</div>
`
})
}
```
## Figma Mapping
- Extract the named states from the design first.
- Prefer explicit prop-driven stories such as `Disabled`, `Loading`, `Invalid`, `WithPlaceholder`, `AllSizes`, or `EdgeCases`.
- Add an aggregate story such as `AllVariants`, `AllSizes`, or `EdgeCases` when side-by-side comparison is useful.
- Use pseudo-state parameters only if the addon is already configured in this repo.
- If a Figma state cannot be represented exactly, capture the closest prop-driven version and explain the gap in the story docs.
## Component-Specific Notes
- Widget components often need a minimal `SimplifiedWidget` object. Build it in `setup()` and use `computed()` when `args` change `widget.options`.
- Input and search components often need a width-constrained wrapper so they render at realistic sizes.
- Asset and platform cards often need background decorators such as `bg-base-background` and fixed-width containers.
- Desktop installer stories may need custom `backgrounds` parameters and may intentionally keep the older Storybook import style used by neighboring files.
- Use semantic tokens such as `bg-base-background` and `bg-node-component-surface` instead of `dark:` variants or hardcoded theme assumptions.
## Checklist
- [ ] Read the component source and any supporting types or composables
- [ ] Match the nearest local title pattern and story style
- [ ] Include a baseline story; name it `Default` only when that matches nearby conventions
- [ ] Add focused stories for meaningful states
- [ ] Add `tags: ['autodocs']`
- [ ] Keep the story co-located with the component
- [ ] Run `pnpm storybook`
- [ ] Run `pnpm typecheck`
- [ ] Run `pnpm lint`
## Avoid
- Do not guess props, emits, slots, or exposed methods.
- Do not force one generic title convention across the repo.
- Do not mutate Storybook args directly for `v-model` components.
- Do not introduce `dark:` Tailwind variants in story wrappers.
- Do not create barrel files.
- Do not assume every story needs `layout: 'centered'` or a `Default` export; follow the nearest existing pattern.

View File

@@ -1,4 +0,0 @@
interface:
display_name: 'ComfyUI Storybook Stories'
short_description: 'Write Vue Storybook stories for ComfyUI'
default_prompt: 'Use $writing-storybook-stories to add or update a Storybook story for this ComfyUI_frontend component.'

View File

@@ -1,48 +0,0 @@
issue_enrichment:
auto_enrich:
enabled: true
reviews:
high_level_summary: false
request_changes_workflow: true
auto_review:
drafts: true
ignore_title_keywords:
- '[release]'
- '[backport'
ignore_usernames:
- comfy-pr-bot
- github-actions
- github-actions[bot]
pre_merge_checks:
override_requested_reviewers_only: true
custom_checks:
- name: End-to-end regression coverage for fixes
mode: error
instructions: |
Use only PR metadata already available in the review context: the PR title, commit subjects in this PR, the files changed in this PR relative to the PR base (equivalent to `base...head`), and the PR description.
Do not rely on shell commands. Do not inspect reverse diffs, files changed only on the base branch, or files outside this PR. If the changed-file list or commit subjects are unavailable, mark the check inconclusive instead of guessing.
Pass if at least one of the following is true:
1. Neither the PR title nor any commit subject in the PR uses bug-fix language such as `fix`, `fixed`, `fixes`, `fixing`, `bugfix`, or `hotfix`.
2. The PR changes at least one file under `browser_tests/`.
3. The PR description includes a concrete, non-placeholder explanation of why an end-to-end regression test was not added.
Fail otherwise. When failing, mention which bug-fix signal you found and ask the author to either add or update a Playwright regression test under `browser_tests/` or add a concrete explanation in the PR description of why an end-to-end regression test is not practical.
- name: ADR compliance for entity/litegraph changes
mode: warning
instructions: |
Use only PR metadata already available in the review context: the changed-file list relative to the PR base, the PR description, and the diff content. Do not rely on shell commands.
This check applies ONLY when the PR modifies files under `src/lib/litegraph/`, `src/ecs/`, or files related to graph entities (nodes, links, widgets, slots, reroutes, groups, subgraphs).
If none of those paths appear in the changed files, pass immediately.
When applicable, check for:
1. **Command pattern (ADR 0003)**: Entity state mutations must be serializable, idempotent, deterministic commands — not imperative fire-and-forget side effects. Flag direct spatial mutation (`node.pos =`, `node.size =`, `group.pos =`) outside of a store or command, and any new void-returning mutation API that should produce a command object.
2. **God-object growth (ADR 0008)**: New methods/properties added to `LGraphNode`, `LGraphCanvas`, `LGraph`, or `Subgraph` that add responsibilities rather than extracting/migrating existing ones.
3. **ECS data/behavior separation (ADR 0008)**: Component-like data structures that contain methods or back-references to parent entities. ECS components must be plain data. New OOP instance patterns (`node.someProperty`, `node.someMethod()`) for data that should be a World component.
4. **Extension ecosystem (ADR 0008)**: Changes to extension-facing callbacks (`onConnectionsChange`, `onRemoved`, `onAdded`, `onConfigure`, `onConnectInput/Output`, `onWidgetChanged`), `node.widgets` access, `node.serialize` overrides, or `graph._version++` without migration guidance. These affect 40+ custom node repos.
Pass if none of these patterns are found in the diff.
When warning, reference the specific ADR by number and link to `docs/adr/` for context. Frame findings as directional guidance since ADR 0003 and 0008 are in Proposed status.

View File

@@ -0,0 +1,21 @@
---
description: Creating unit tests
globs:
alwaysApply: false
---
# Creating unit tests
- This project uses `vitest` for unit testing
- Tests are stored in the `test/` directory
- Tests should be cross-platform compatible; able to run on Windows, macOS, and linux
- e.g. the use of `path.resolve`, or `path.join` and `path.sep` to ensure that tests work the same on all platforms
- Tests should be mocked properly
- Mocks should be cleanly written and easy to understand
- Mocks should be re-usable where possible
## Unit test style
- Prefer the use of `test.extend` over loose variables
- To achieve this, import `test as baseTest` from `vitest`
- Never use `it`; `test` should be used in place of this

61
.cursorrules Normal file
View File

@@ -0,0 +1,61 @@
# Vue 3 Composition API Project Rules
## Vue 3 Composition API Best Practices
- Use setup() function for component logic
- Utilize ref and reactive for reactive state
- Implement computed properties with computed()
- Use watch and watchEffect for side effects
- Implement lifecycle hooks with onMounted, onUpdated, etc.
- Utilize provide/inject for dependency injection
- Use vue 3.5 style of default prop declaration. Example:
```typescript
const { nodes, showTotal = true } = defineProps<{
nodes: ApiNodeCost[]
showTotal?: boolean
}>()
```
- Organize vue component in <template> <script> <style> order
## Project Structure
```
src/
components/
constants/
composables/
views/
stores/
services/
App.vue
main.ts
```
## Styling Guidelines
- Use Tailwind CSS for styling
- Implement responsive design with Tailwind CSS
## PrimeVue Component Guidelines
DO NOT use deprecated PrimeVue components. Use these replacements instead:
- Dropdown → Use Select (import from 'primevue/select')
- OverlayPanel → Use Popover (import from 'primevue/popover')
- Calendar → Use DatePicker (import from 'primevue/datepicker')
- InputSwitch → Use ToggleSwitch (import from 'primevue/toggleswitch')
- Sidebar → Use Drawer (import from 'primevue/drawer')
- Chips → Use AutoComplete with multiple enabled and typeahead disabled
- TabMenu → Use Tabs without panels
- Steps → Use Stepper without panels
- InlineMessage → Use Message component
## Development Guidelines
1. Leverage VueUse functions for performance-enhancing styles
2. Use es-toolkit for utility functions
3. Use TypeScript for type safety
4. Implement proper props and emits definitions
5. Utilize Vue 3's Teleport component when needed
6. Use Suspense for async components
7. Implement proper error handling
8. Follow Vue 3 style guide and naming conventions
9. Use Vite for fast development and building
10. Use vue-i18n in composition API for any string literals. Place new translation entries in src/locales/en/main.json
11. Never use deprecated PrimeVue components listed above

View File

@@ -5,52 +5,31 @@ PLAYWRIGHT_TEST_URL=http://localhost:5173
# Proxy target of the local development server
# Note: localhost:8188 does not work.
# Cloud auto-detection: Setting this to any *.comfy.org URL automatically enables
# cloud mode (DISTRIBUTION=cloud) without needing to set DISTRIBUTION separately.
# Examples: https://testcloud.comfy.org/, https://stagingcloud.comfy.org/,
# https://pr-123.testenvs.comfy.org/, https://cloud.comfy.org/
DEV_SERVER_COMFYUI_URL=http://127.0.0.1:8188
# Allow dev server access from remote IP addresses.
# If true, the vite dev server will listen on all addresses, including LAN
# and public addresses.
# VITE_REMOTE_DEV=true
VITE_REMOTE_DEV=false
# The directory containing the ComfyUI installation used to run Playwright tests.
# If you aren't using a separate install for testing, point this to your regular install.
TEST_COMFYUI_DIR=/home/ComfyUI
# Whether to enable minification of the frontend code.
# ENABLE_MINIFY=true
ENABLE_MINIFY=true
# Whether to disable proxying the `/templates` route. If true, allows you to
# serve templates from the ComfyUI_frontend/public/templates folder (for
# locally testing changes to templates). When false or nonexistent, the
# templates are served via the normal method from the server's python site
# Whether to disable proxying the `/templates` route. If true, allows you to
# serve templates from the ComfyUI_frontend/public/templates folder (for
# locally testing changes to templates). When false or nonexistent, the
# templates are served via the normal method from the server's python site
# packages.
# DISABLE_TEMPLATES_PROXY=true
DISABLE_TEMPLATES_PROXY=false
# If playwright tests are being run via vite dev server, Vue plugins will
# invalidate screenshots. When `true`, vite plugins will not be loaded.
# DISABLE_VUE_PLUGINS=true
DISABLE_VUE_PLUGINS=false
# Algolia credentials required for developing with the new custom node manager.
ALGOLIA_APP_ID=4E0RO38HS8
ALGOLIA_API_KEY=684d998c36b67a9a9fce8fc2d8860579
# Enable PostHog debug logging in the browser console.
# VITE_POSTHOG_DEBUG=true
# Sentry ENV vars replace with real ones for debugging
# SENTRY_AUTH_TOKEN=private-token # get from sentry
# SENTRY_ORG=comfy-org
# SENTRY_PROJECT=cloud-frontend-staging
# SENTRY_PROJECT_PROD= # prod project slug for sourcemap uploads
# ── QA Skill (scripts/qa-record.ts) ──
# Required for automated bug reproduction via `pnpm exec tsx scripts/qa-record.ts`
# GEMINI_API_KEY is required — used for PR analysis, video review, and TTS narration
GEMINI_API_KEY=
# ANTHROPIC_API_KEY is optional locally — Claude Agent SDK auto-detects Claude Code session
# Required in CI (set as GitHub Actions secret)
# ANTHROPIC_API_KEY=

View File

@@ -25,6 +25,3 @@ e3bb29ceb8174b8bbca9e48ec7d42cd540f40efa
# [refactor] Improve updates/notifications domain organization (#5590)
27ab355f9c73415dc39f4d3f512b02308f847801
# Migrate Tailwind styles to design-system package
9f19d8fb4bd22518879343b49c05634dca777df0

16
.gitattributes vendored
View File

@@ -1,8 +1,16 @@
# Force all text files to use LF line endings
* text=auto eol=lf
# Default
* text=auto
# Force TS to LF to make the unixy scripts not break on Windows
*.cjs text eol=lf
*.js text eol=lf
*.json text eol=lf
*.mjs text eol=lf
*.mts text eol=lf
*.ts text eol=lf
*.vue text eol=lf
*.yaml text eol=lf
# Generated files
packages/registry-types/src/comfyRegistryTypes.ts linguist-generated=true
packages/ingest-types/src/types.gen.ts linguist-generated=true
packages/ingest-types/src/zod.gen.ts linguist-generated=true
src/workbench/extensions/manager/types/generatedManagerTypes.ts linguist-generated=true

23
.github/AGENTS.md vendored
View File

@@ -1,23 +0,0 @@
# PR Review Context
Context for automated PR review system.
## Review Scope
This automated review performs comprehensive analysis:
- Architecture and design patterns
- Security vulnerabilities
- Performance implications
- Code quality and maintainability
- Integration concerns
For implementation details, see `.claude/commands/comprehensive-pr-review.md`.
## GitHub Actions: Fork PR Permissions
Fork PRs get a **read-only `GITHUB_TOKEN`** — no PR comments, no secret access, no pushing.
Any workflow that needs write access must use the **two-workflow split**: a `pull_request`-triggered `ci-*.yaml` uploads artifacts (including PR metadata), then a `workflow_run`-triggered `pr-*.yaml` downloads them and posts comments with write permissions. See `ci-size-data``pr-size-report` or `ci-perf-report``pr-perf-report`. Use `.github/actions/post-pr-report-comment` for the comment step.
Never write PR comments directly from `pull_request` workflows or use `pull_request_target` to run untrusted code.

38
.github/CLAUDE.md vendored
View File

@@ -1,4 +1,36 @@
<!-- A rose by any other name would smell as sweet,
But Claude insists on files named for its own conceit. -->
# ComfyUI Frontend - Claude Review Context
@AGENTS.md
This file provides additional context for the automated PR review system.
## Quick Reference
### PrimeVue Component Migrations
When reviewing, flag these deprecated components:
- `Dropdown` → Use `Select` from 'primevue/select'
- `OverlayPanel` → Use `Popover` from 'primevue/popover'
- `Calendar` → Use `DatePicker` from 'primevue/datepicker'
- `InputSwitch` → Use `ToggleSwitch` from 'primevue/toggleswitch'
- `Sidebar` → Use `Drawer` from 'primevue/drawer'
- `Chips` → Use `AutoComplete` with multiple enabled and typeahead disabled
- `TabMenu` → Use `Tabs` without panels
- `Steps` → Use `Stepper` without panels
- `InlineMessage` → Use `Message` component
### API Utilities Reference
- `api.apiURL()` - Backend API calls (/prompt, /queue, /view, etc.)
- `api.fileURL()` - Static file access (templates, extensions)
- `$t()` / `i18n.global.t()` - Internationalization
- `DOMPurify.sanitize()` - HTML sanitization
## Review Scope
This automated review performs comprehensive analysis including:
- Architecture and design patterns
- Security vulnerabilities
- Performance implications
- Code quality and maintainability
- Integration concerns
For implementation details, see `.claude/commands/comprehensive-pr-review.md`.

View File

@@ -10,7 +10,10 @@ body:
options:
- label: I am running the latest version of ComfyUI
required: true
- label: I have custom nodes enabled
- label: I have searched existing issues to make sure this isn't a duplicate
required: true
- label: I have tested with all custom nodes disabled ([see how](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled))
required: true
- type: textarea
id: description
@@ -33,9 +36,9 @@ body:
3. Click Queue Prompt
4. See error
value: |
1.
2.
3.
1.
2.
3.
validations:
required: true
@@ -57,7 +60,7 @@ body:
attributes:
label: ComfyUI Frontend Version
description: Found in Settings > About (e.g., "1.3.45")
placeholder: '1.3.45'
placeholder: "1.3.45"
validations:
required: true

View File

@@ -4,6 +4,13 @@ labels: []
type: Feature
body:
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the problem you're experiencing, and that it's not addressed in a recent build/commit.
options:
- label: I have searched the existing issues and checked the recent builds/commits
required: true
- type: markdown
attributes:
value: |

View File

@@ -1,119 +0,0 @@
name: Post Release Summary Comment
description: Post or update a PR comment summarizing release links with diff, derived versions, and optional extras.
author: ComfyUI Frontend Team
inputs:
issue-number:
description: Optional PR number override (defaults to the current pull request)
default: ''
version_file:
description: Path to the JSON file containing the current version (relative to repo root)
required: true
outputs:
prev_version:
description: Previous version derived from the parent commit
value: ${{ steps.build.outputs.prev_version }}
runs:
using: composite
steps:
- name: Build comment body
id: build
shell: bash
run: |
set -euo pipefail
VERSION_FILE="${{ inputs.version_file }}"
REPO="${{ github.repository }}"
if [[ -z "$VERSION_FILE" ]]; then
echo "::error::version_file input is required" >&2
exit 1
fi
PREV_JSON=$(git show HEAD^1:"$VERSION_FILE" 2>/dev/null || true)
if [[ -z "$PREV_JSON" ]]; then
echo "::error::Unable to read $VERSION_FILE from parent commit" >&2
exit 1
fi
PREV_VERSION=$(printf '%s' "$PREV_JSON" | node -pe "const data = JSON.parse(require('fs').readFileSync(0, 'utf8')); if (!data.version) { process.exit(1); } data.version")
if [[ -z "$PREV_VERSION" ]]; then
echo "::error::Unable to determine previous version from $VERSION_FILE" >&2
exit 1
fi
NEW_VERSION=$(node -pe "const fs=require('fs');const data=JSON.parse(fs.readFileSync(process.argv[1],'utf8'));if(!data.version){process.exit(1);}data.version" "$VERSION_FILE")
if [[ -z "$NEW_VERSION" ]]; then
echo "::error::Unable to determine current version from $VERSION_FILE" >&2
exit 1
fi
MARKER='release-summary'
MESSAGE='Publish jobs finished successfully:'
LINKS_VALUE=''
case "$VERSION_FILE" in
package.json)
LINKS_VALUE=$(printf '%s\n%s' \
'PyPI|https://pypi.org/project/comfyui-frontend-package/{{version}}/' \
'npm types|https://www.npmjs.com/package/@comfyorg/comfyui-frontend-types/v/{{version}}')
;;
apps/desktop-ui/package.json)
MARKER='desktop-release-summary'
LINKS_VALUE='npm desktop UI|https://www.npmjs.com/package/@comfyorg/desktop-ui/v/{{version}}'
;;
esac
DIFF_PREFIX='v'
DIFF_LABEL='Diff'
DIFF_URL="https://github.com/${REPO}/compare/${DIFF_PREFIX}${PREV_VERSION}...${DIFF_PREFIX}${NEW_VERSION}"
COMMENT_FILE=$(mktemp)
{
echo "<!--$MARKER:$DIFF_PREFIX$NEW_VERSION-->"
echo "$MESSAGE"
echo ""
echo "- $DIFF_LABEL: [\`$DIFF_PREFIX$PREV_VERSION...$DIFF_PREFIX$NEW_VERSION\`]($DIFF_URL)"
while IFS= read -r RAW_LINE; do
LINE=$(echo "$RAW_LINE" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
[[ -z "$LINE" ]] && continue
if [[ "$LINE" != *"|"* ]]; then
echo "::warning::Skipping malformed link entry: $LINE" >&2
continue
fi
LABEL=${LINE%%|*}
URL_TEMPLATE=${LINE#*|}
URL=${URL_TEMPLATE//\{\{version\}\}/$NEW_VERSION}
URL=${URL//\{\{prev_version\}\}/$PREV_VERSION}
echo "- $LABEL: [\`$NEW_VERSION\`]($URL)"
done <<< "$LINKS_VALUE"
echo ""
} > "$COMMENT_FILE"
{
echo "body<<COMMENT_BODY_END_MARKER"
cat "$COMMENT_FILE"
echo "COMMENT_BODY_END_MARKER"
} >> "$GITHUB_OUTPUT"
echo "prev_version=$PREV_VERSION" >> "$GITHUB_OUTPUT"
echo "marker_search=<!--$MARKER:" >> "$GITHUB_OUTPUT"
echo "new_version=$NEW_VERSION" >> "$GITHUB_OUTPUT"
- name: Find existing comment
id: find
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
with:
issue-number: ${{ inputs.issue-number || github.event.pull_request.number }}
comment-author: github-actions[bot]
body-includes: ${{ steps.build.outputs.marker_search }}
- name: Post or update comment
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
issue-number: ${{ inputs.issue-number || github.event.pull_request.number }}
comment-id: ${{ steps.find.outputs.comment-id }}
body: ${{ steps.build.outputs.body }}
edit-mode: replace

View File

@@ -1,35 +0,0 @@
name: Post PR Report Comment
description: Reads a markdown report file and posts/updates a single idempotent comment on a PR.
inputs:
pr-number:
description: PR number to comment on
required: true
report-file:
description: Path to the markdown report file
required: true
comment-marker:
description: HTML comment marker for idempotent matching
required: true
token:
description: GitHub token with pull-requests write permission
required: true
runs:
using: composite
steps:
- name: Read report
id: report
uses: juliangruber/read-file-action@b549046febe0fe86f8cb4f93c24e284433f9ab58 # v1.1.7
with:
path: ${{ inputs.report-file }}
- name: Create or update PR comment
uses: actions-cool/maintain-one-comment@4b2dbf086015f892dcb5e8c1106f5fccd6c1476b # v3.2.0
with:
token: ${{ inputs.token }}
number: ${{ inputs.pr-number }}
body: |
${{ steps.report.outputs.content }}
${{ inputs.comment-marker }}
body-include: ${{ inputs.comment-marker }}

View File

@@ -1,55 +0,0 @@
name: Setup ComfyUI Server
description: 'Setup ComfyUI server for continuous integration (with ComfyUI_devtools node installed)'
inputs:
extra_server_params:
description: 'Additional parameters to pass to ComfyUI server'
required: false
default: ''
launch_server:
description: 'Whether to launch the server after setup'
required: false
default: 'false'
runs:
using: 'composite'
steps:
# Note: this workflow assume frontend repo is checked out and is built in ../dist
# Checkout ComfyUI repo, install the dev_tools node and start server
- name: Checkout ComfyUI
uses: actions/checkout@v6
with:
repository: 'comfyanonymous/ComfyUI'
path: 'ComfyUI'
- name: Install ComfyUI_devtools from frontend repo
shell: bash
run: |
mkdir -p ComfyUI/custom_nodes/ComfyUI_devtools
if ! cp -r ./tools/devtools/* ComfyUI/custom_nodes/ComfyUI_devtools/; then
echo "::error::Failed to copy ComfyUI_devtools from ./tools/devtools/"
echo "::error::This action assumes the ComfyUI_frontend repository is checked out in the current working directory."
echo "::error::Please ensure you have run 'actions/checkout@v5' before calling this action."
exit 1
fi
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install Python requirements
shell: bash
working-directory: ComfyUI
run: |
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Start ComfyUI server
if: ${{ inputs.launch_server == 'true' }}
shell: bash
working-directory: ComfyUI
run: |
python main.py --cpu --multi-user --front-end-root ../dist ${{ inputs.extra_server_params }} &
wait-for-it --service 127.0.0.1:8188 -t 600

View File

@@ -1,31 +0,0 @@
name: Setup ComfyUI Frontend
description: 'Install nodejs/pnpm/dependencies and optionally build ComfyUI_frontend'
inputs:
include_build_step:
description: 'Include the build step to build the frontend. Set to true for workflows that need a built frontend'
required: false
default: 'false'
runs:
using: 'composite'
steps:
# Note: this workflow assume frontend repo is checked out in the root of the workspace
# Install pnpm, Node.js, build frontend
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
cache-dependency-path: './pnpm-lock.yaml'
- name: Install dependencies
shell: bash
run: pnpm install --frozen-lockfile
- name: Build ComfyUI_frontend
if: ${{ inputs.include_build_step == 'true' }}
shell: bash
run: pnpm build

View File

@@ -0,0 +1,67 @@
name: Setup Frontend
description: 'Setup ComfyUI frontend development environment'
inputs:
extra_server_params:
description: 'Additional parameters to pass to ComfyUI server'
required: false
default: ''
runs:
using: 'composite'
steps:
- name: Checkout ComfyUI
uses: actions/checkout@v4
with:
repository: 'comfyanonymous/ComfyUI'
path: 'ComfyUI'
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v4
with:
repository: 'Comfy-Org/ComfyUI_frontend'
path: 'ComfyUI_frontend'
- name: Copy ComfyUI_devtools from frontend repo
shell: bash
run: |
mkdir -p ComfyUI/custom_nodes/ComfyUI_devtools
cp -r ComfyUI_frontend/tools/devtools/* ComfyUI/custom_nodes/ComfyUI_devtools/
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 'lts/*'
cache: 'pnpm'
cache-dependency-path: 'ComfyUI_frontend/pnpm-lock.yaml'
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install Python requirements
shell: bash
working-directory: ComfyUI
run: |
python -m pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
pip install wait-for-it
- name: Build & Install ComfyUI_frontend
shell: bash
working-directory: ComfyUI_frontend
run: |
pnpm install --frozen-lockfile
pnpm build
- name: Start ComfyUI server
shell: bash
working-directory: ComfyUI
run: |
python main.py --cpu --multi-user --front-end-root ../ComfyUI_frontend/dist ${{ inputs.extra_server_params }} &
wait-for-it --service 127.0.0.1:8188 -t 600

View File

@@ -1,28 +0,0 @@
name: Setup Playwright
description: Cache and install Playwright browsers with dependencies
runs:
using: composite
steps:
- name: Detect Playwright version
id: detect-version
shell: bash
run: |
PLAYWRIGHT_VERSION=$(pnpm ls @playwright/test --json | jq --raw-output '.[0].devDependencies["@playwright/test"].version')
echo "playwright-version=$PLAYWRIGHT_VERSION" >> $GITHUB_OUTPUT
- name: Cache Playwright Browsers
uses: actions/cache@v5 # v5.0.2
id: cache-playwright-browsers
with:
path: '~/.cache/ms-playwright'
key: ${{ runner.os }}-playwright-browsers-${{ steps.detect-version.outputs.playwright-version }}
- name: Install Playwright Browsers
if: steps.cache-playwright-browsers.outputs.cache-hit != 'true'
shell: bash
run: pnpm exec playwright install chromium --with-deps
- name: Install Playwright Browsers (operating system dependencies)
if: steps.cache-playwright-browsers.outputs.cache-hit == 'true'
shell: bash
run: pnpm exec playwright install-deps

View File

@@ -1,23 +0,0 @@
name: Start ComfyUI Server
description: 'Start ComfyUI server in a container environment (assumes ComfyUI is pre-installed)'
inputs:
front_end_root:
description: 'Path to frontend dist directory'
required: false
default: '$GITHUB_WORKSPACE/dist'
timeout:
description: 'Timeout in seconds for server startup'
required: false
default: '600'
runs:
using: 'composite'
steps:
- name: Copy devtools and start server
shell: bash
run: |
set -euo pipefail
cp -r ./tools/devtools/* /ComfyUI/custom_nodes/ComfyUI_devtools/
cd /ComfyUI && python3 main.py --cpu --multi-user --front-end-root "${{ inputs.front_end_root }}" &
wait-for-it --service 127.0.0.1:8188 -t ${{ inputs.timeout }}

View File

@@ -34,4 +34,4 @@ Follow Vue 3 style guide and naming conventions
Use Vite for fast development and building
Use vue-i18n in composition API for any string literals. Place new translation entries in src/locales/en/main.json.
Use vue-i18n in composition API for any string literals. Place new translation entries in src/locales/en/main.json.

View File

@@ -1,3 +0,0 @@
{
"posthog-js@*": { "licenses": "Apache-2.0" }
}

View File

@@ -1,21 +0,0 @@
# GitHub Workflows
## Naming Convention
Workflow files follow a consistent naming pattern: `<prefix>-<descriptive-name>.yaml`
### Category Prefixes
| Prefix | Purpose | Example |
| ---------- | ----------------------------------- | ------------------------------------ |
| `ci-` | Testing, linting, validation | `ci-tests-e2e.yaml` |
| `release-` | Version management, publishing | `release-version-bump.yaml` |
| `pr-` | PR automation (triggered by labels) | `pr-claude-review.yaml` |
| `api-` | External Api type generation | `api-update-registry-api-types.yaml` |
| `i18n-` | Internationalization updates | `i18n-update-core.yaml` |
## Documentation
Each workflow file contains comments explaining its purpose, triggers, and behavior. For specific details about what each workflow does, refer to the comments at the top of each `.yaml` file.
For GitHub Actions documentation, see [Events that trigger workflows](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows).

268
.github/workflows/backport.yaml vendored Normal file
View File

@@ -0,0 +1,268 @@
name: Auto Backport
on:
pull_request_target:
types: [closed, labeled]
branches: [main]
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to backport'
required: true
type: string
force_rerun:
description: 'Force rerun even if backports exist'
required: false
type: boolean
default: false
jobs:
backport:
if: >
(github.event_name == 'pull_request_target' &&
github.event.pull_request.merged == true &&
contains(github.event.pull_request.labels.*.name, 'needs-backport')) ||
github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
issues: write
steps:
- name: Validate inputs for manual triggers
if: github.event_name == 'workflow_dispatch'
run: |
# Validate PR number format
if ! [[ "${{ inputs.pr_number }}" =~ ^[0-9]+$ ]]; then
echo "::error::Invalid PR number format. Must be a positive integer."
exit 1
fi
# Validate PR exists and is merged
if ! gh pr view "${{ inputs.pr_number }}" --json merged >/dev/null 2>&1; then
echo "::error::PR #${{ inputs.pr_number }} not found or inaccessible."
exit 1
fi
MERGED=$(gh pr view "${{ inputs.pr_number }}" --json merged --jq '.merged')
if [ "$MERGED" != "true" ]; then
echo "::error::PR #${{ inputs.pr_number }} is not merged. Only merged PRs can be backported."
exit 1
fi
# Validate PR has needs-backport label
if ! gh pr view "${{ inputs.pr_number }}" --json labels --jq '.labels[].name' | grep -q "needs-backport"; then
echo "::error::PR #${{ inputs.pr_number }} does not have 'needs-backport' label."
exit 1
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Check if backports already exist
id: check-existing
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number }}
run: |
# Check for existing backport PRs for this PR number
EXISTING_BACKPORTS=$(gh pr list --state all --search "backport-${PR_NUMBER}-to" --json title,headRefName,baseRefName | jq -r '.[].headRefName')
if [ -z "$EXISTING_BACKPORTS" ]; then
echo "skip=false" >> $GITHUB_OUTPUT
exit 0
fi
# For manual triggers with force_rerun, proceed anyway
if [ "${{ github.event_name }}" = "workflow_dispatch" ] && [ "${{ inputs.force_rerun }}" = "true" ]; then
echo "skip=false" >> $GITHUB_OUTPUT
echo "::warning::Force rerun requested - existing backports will be updated"
exit 0
fi
echo "Found existing backport PRs:"
echo "$EXISTING_BACKPORTS"
echo "skip=true" >> $GITHUB_OUTPUT
echo "::warning::Backport PRs already exist for PR #${PR_NUMBER}, skipping to avoid duplicates"
- name: Extract version labels
if: steps.check-existing.outputs.skip != 'true'
id: versions
run: |
# Extract version labels (e.g., "1.24", "1.22")
VERSIONS=""
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
# For manual triggers, get labels from the PR
LABELS=$(gh pr view ${{ inputs.pr_number }} --json labels | jq -r '.labels[].name')
else
# For automatic triggers, extract from PR event
LABELS='${{ toJSON(github.event.pull_request.labels) }}'
LABELS=$(echo "$LABELS" | jq -r '.[].name')
fi
for label in $LABELS; do
# Match version labels like "1.24" (major.minor only)
if [[ "$label" =~ ^[0-9]+\.[0-9]+$ ]]; then
# Validate the branch exists before adding to list
if git ls-remote --exit-code origin "core/${label}" >/dev/null 2>&1; then
VERSIONS="${VERSIONS}${label} "
else
echo "::warning::Label '${label}' found but branch 'core/${label}' does not exist"
fi
fi
done
if [ -z "$VERSIONS" ]; then
echo "::error::No version labels found (e.g., 1.24, 1.22)"
exit 1
fi
echo "versions=${VERSIONS}" >> $GITHUB_OUTPUT
echo "Found version labels: ${VERSIONS}"
- name: Backport commits
if: steps.check-existing.outputs.skip != 'true'
id: backport
env:
PR_NUMBER: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number }}
run: |
FAILED=""
SUCCESS=""
# Get PR data for manual triggers
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
PR_DATA=$(gh pr view ${{ inputs.pr_number }} --json title,mergeCommit)
PR_TITLE=$(echo "$PR_DATA" | jq -r '.title')
MERGE_COMMIT=$(echo "$PR_DATA" | jq -r '.mergeCommit.oid')
else
PR_TITLE="${{ github.event.pull_request.title }}"
MERGE_COMMIT="${{ github.event.pull_request.merge_commit_sha }}"
fi
for version in ${{ steps.versions.outputs.versions }}; do
echo "::group::Backporting to core/${version}"
TARGET_BRANCH="core/${version}"
BACKPORT_BRANCH="backport-${PR_NUMBER}-to-${version}"
# Fetch target branch (fail if doesn't exist)
if ! git fetch origin "${TARGET_BRANCH}"; then
echo "::error::Target branch ${TARGET_BRANCH} does not exist"
FAILED="${FAILED}${version}:branch-missing "
echo "::endgroup::"
continue
fi
# Create backport branch
git checkout -b "${BACKPORT_BRANCH}" "origin/${TARGET_BRANCH}"
# Try cherry-pick
if git cherry-pick "${MERGE_COMMIT}"; then
git push origin "${BACKPORT_BRANCH}"
SUCCESS="${SUCCESS}${version}:${BACKPORT_BRANCH} "
echo "Successfully created backport branch: ${BACKPORT_BRANCH}"
# Return to main (keep the branch, we need it for PR)
git checkout main
else
# Get conflict info
CONFLICTS=$(git diff --name-only --diff-filter=U | tr '\n' ',')
git cherry-pick --abort
echo "::error::Cherry-pick failed due to conflicts"
FAILED="${FAILED}${version}:conflicts:${CONFLICTS} "
# Clean up the failed branch
git checkout main
git branch -D "${BACKPORT_BRANCH}"
fi
echo "::endgroup::"
done
echo "success=${SUCCESS}" >> $GITHUB_OUTPUT
echo "failed=${FAILED}" >> $GITHUB_OUTPUT
if [ -n "${FAILED}" ]; then
exit 1
fi
- name: Create PR for each successful backport
if: steps.check-existing.outputs.skip != 'true' && steps.backport.outputs.success
env:
GH_TOKEN: ${{ secrets.PR_GH_TOKEN }}
PR_NUMBER: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number }}
run: |
# Get PR data for manual triggers
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
PR_DATA=$(gh pr view ${{ inputs.pr_number }} --json title,author)
PR_TITLE=$(echo "$PR_DATA" | jq -r '.title')
PR_AUTHOR=$(echo "$PR_DATA" | jq -r '.author.login')
else
PR_TITLE="${{ github.event.pull_request.title }}"
PR_AUTHOR="${{ github.event.pull_request.user.login }}"
fi
for backport in ${{ steps.backport.outputs.success }}; do
IFS=':' read -r version branch <<< "${backport}"
if PR_URL=$(gh pr create \
--base "core/${version}" \
--head "${branch}" \
--title "[backport ${version}] ${PR_TITLE}" \
--body "Backport of #${PR_NUMBER} to \`core/${version}\`"$'\n\n'"Automatically created by backport workflow." \
--label "backport" 2>&1); then
# Extract PR number from URL
PR_NUM=$(echo "${PR_URL}" | grep -o '[0-9]*$')
if [ -n "${PR_NUM}" ]; then
gh pr comment "${PR_NUMBER}" --body "@${PR_AUTHOR} Successfully backported to #${PR_NUM}"
fi
else
echo "::error::Failed to create PR for ${version}: ${PR_URL}"
# Still try to comment on the original PR about the failure
gh pr comment "${PR_NUMBER}" --body "@${PR_AUTHOR} Backport branch created but PR creation failed for \`core/${version}\`. Please create the PR manually from branch \`${branch}\`"
fi
done
- name: Comment on failures
if: steps.check-existing.outputs.skip != 'true' && failure() && steps.backport.outputs.failed
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
PR_DATA=$(gh pr view ${{ inputs.pr_number }} --json author,mergeCommit)
PR_NUMBER="${{ inputs.pr_number }}"
PR_AUTHOR=$(echo "$PR_DATA" | jq -r '.author.login')
MERGE_COMMIT=$(echo "$PR_DATA" | jq -r '.mergeCommit.oid')
else
PR_NUMBER="${{ github.event.pull_request.number }}"
PR_AUTHOR="${{ github.event.pull_request.user.login }}"
MERGE_COMMIT="${{ github.event.pull_request.merge_commit_sha }}"
fi
for failure in ${{ steps.backport.outputs.failed }}; do
IFS=':' read -r version reason conflicts <<< "${failure}"
if [ "${reason}" = "branch-missing" ]; then
gh pr comment "${PR_NUMBER}" --body "@${PR_AUTHOR} Backport failed: Branch \`core/${version}\` does not exist"
elif [ "${reason}" = "conflicts" ]; then
# Convert comma-separated conflicts back to newlines for display
CONFLICTS_LIST=$(echo "${conflicts}" | tr ',' '\n' | sed 's/^/- /')
COMMENT_BODY="@${PR_AUTHOR} Backport to \`core/${version}\` failed: Merge conflicts detected."$'\n\n'"Please manually cherry-pick commit \`${MERGE_COMMIT}\` to the \`core/${version}\` branch."$'\n\n'"<details><summary>Conflicting files</summary>"$'\n\n'"${CONFLICTS_LIST}"$'\n\n'"</details>"
gh pr comment "${PR_NUMBER}" --body "${COMMENT_BODY}"
fi
done

View File

@@ -1,10 +1,10 @@
# Description: Builds Storybook and runs visual regression testing via Chromatic, deploys previews to Cloudflare Pages
name: 'CI: Tests Storybook'
name: Storybook and Chromatic CI
# - [Automate Chromatic with GitHub Actions • Chromatic docs]( https://www.chromatic.com/docs/github-actions/ )
on:
workflow_dispatch: # Allow manual triggering
pull_request:
push:
branches: [main]
jobs:
@@ -16,8 +16,8 @@ jobs:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v5
- name: Post starting comment
env:
GITHUB_TOKEN: ${{ github.token }}
@@ -26,7 +26,8 @@ jobs:
./scripts/cicd/pr-storybook-deploy-and-comment.sh \
"${{ github.event.pull_request.number }}" \
"${{ github.head_ref }}" \
"starting"
"starting" \
"$(date -u '+%m/%d/%Y, %I:%M:%S %p')"
# Build Storybook for all PRs (free Cloudflare deployment)
storybook-build:
@@ -37,10 +38,34 @@ jobs:
workflow-url: ${{ steps.workflow-url.outputs.url }}
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v5
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'pnpm'
- name: Cache tool outputs
uses: actions/cache@v4
with:
path: |
.cache
storybook-static
tsconfig.tsbuildinfo
key: storybook-cache-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('src/**/*.{ts,vue,js}', '*.config.*', '.storybook/**/*') }}
restore-keys: |
storybook-cache-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}-
storybook-cache-${{ runner.os }}-
storybook-tools-cache-${{ runner.os }}-
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build Storybook
run: pnpm build-storybook
@@ -59,7 +84,7 @@ jobs:
- name: Upload Storybook build
if: success() && github.event.pull_request.head.repo.fork == false
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v4
with:
name: storybook-static
path: storybook-static/
@@ -76,16 +101,40 @@ jobs:
chromatic-storybook-url: ${{ steps.chromatic.outputs.storybookUrl }}
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v5
with:
fetch-depth: 0 # Required for Chromatic baseline
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 10
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'pnpm'
- name: Cache tool outputs
uses: actions/cache@v4
with:
path: |
.cache
storybook-static
tsconfig.tsbuildinfo
key: storybook-cache-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('src/**/*.{ts,vue,js}', '*.config.*', '.storybook/**/*') }}
restore-keys: |
storybook-cache-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}-
storybook-cache-${{ runner.os }}-
storybook-tools-cache-${{ runner.os }}-
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build Storybook and run Chromatic
id: chromatic
uses: chromaui/action@07791f8243f4cb2698bf4d00426baf4b2d1cb7e0 # v13.3.5
uses: chromaui/action@latest
with:
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
buildScriptName: build-storybook
@@ -115,18 +164,18 @@ jobs:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v5
- name: Download Storybook build
if: needs.storybook-build.outputs.conclusion == 'success'
uses: actions/download-artifact@v7
uses: actions/download-artifact@v4
with:
name: storybook-static
path: storybook-static
- name: Make deployment script executable
run: chmod +x scripts/cicd/pr-storybook-deploy-and-comment.sh
- name: Deploy Storybook and comment on PR
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
@@ -140,29 +189,6 @@ jobs:
"${{ github.head_ref }}" \
"completed"
# Deploy Storybook to production URL on main branch push
deploy-production:
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Build Storybook
run: pnpm build-storybook
- name: Deploy to Cloudflare Pages (production)
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: |
npx wrangler@^4.0.0 pages deploy storybook-static \
--project-name=comfy-storybook \
--branch=main
# Update comment with Chromatic URLs for version-bump branches
update-comment-with-chromatic:
needs: [chromatic-deployment, deploy-and-comment]
@@ -172,30 +198,30 @@ jobs:
pull-requests: write
steps:
- name: Update comment with Chromatic URLs
uses: actions/github-script@v8
uses: actions/github-script@v7
with:
script: |
const buildUrl = '${{ needs.chromatic-deployment.outputs.chromatic-build-url }}';
const storybookUrl = '${{ needs.chromatic-deployment.outputs.chromatic-storybook-url }}';
// Find the existing Storybook comment
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ github.event.pull_request.number }}
});
const storybookComment = comments.find(comment =>
const storybookComment = comments.find(comment =>
comment.body.includes('<!-- STORYBOOK_BUILD_STATUS -->')
);
if (storybookComment && buildUrl && storybookUrl) {
// Append Chromatic info to existing comment
const updatedBody = storybookComment.body.replace(
/---\n(.*)$/s,
`---\n### 🎨 Chromatic Visual Tests\n- 📊 [View Chromatic Build](${buildUrl})\n- 📚 [View Chromatic Storybook](${storybookUrl})\n\n$1`
);
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,

View File

@@ -1,98 +0,0 @@
name: 'CI: Dist Telemetry Scan'
on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build project
run: pnpm build
env:
DISTRIBUTION: localhost
- name: Scan dist for GTM telemetry references
run: |
set -euo pipefail
echo '🔍 Scanning for Google Tag Manager references...'
if rg --no-ignore -n \
-g '*.html' \
-g '*.js' \
-e 'Google Tag Manager' \
-e '(?i)\bgtm\.js\b' \
-e '(?i)googletagmanager\.com/gtm\.js\\?id=' \
-e '(?i)googletagmanager\.com/ns\.html\\?id=' \
dist; then
echo '❌ ERROR: Google Tag Manager references found in dist assets!'
echo 'GTM must be properly tree-shaken from OSS builds.'
exit 1
fi
echo '✅ No GTM references found'
- name: Scan dist for Mixpanel telemetry references
run: |
set -euo pipefail
echo '🔍 Scanning for Mixpanel references...'
if rg --no-ignore -n \
-g '*.html' \
-g '*.js' \
-e '(?i)mixpanel\.init' \
-e '(?i)mixpanel\.identify' \
-e 'MixpanelTelemetryProvider' \
-e 'mp\.comfy\.org' \
-e 'mixpanel-browser' \
-e '(?i)mixpanel\.track\(' \
dist; then
echo '❌ ERROR: Mixpanel references found in dist assets!'
echo 'Mixpanel must be properly tree-shaken from OSS builds.'
echo ''
echo 'To fix this:'
echo '1. Use the TelemetryProvider pattern (see src/platform/telemetry/)'
echo '2. Call telemetry via useTelemetry() hook'
echo '3. Use conditional dynamic imports behind isCloud checks'
exit 1
fi
echo '✅ No Mixpanel references found'
- name: Scan dist for PostHog telemetry references
run: |
set -euo pipefail
echo '🔍 Scanning for PostHog references...'
if rg --no-ignore -n \
-g '*.html' \
-g '*.js' \
-e '(?i)posthog\.init' \
-e '(?i)posthog\.capture' \
-e 'PostHogTelemetryProvider' \
-e 'ph\.comfy\.org' \
-e 'posthog-js' \
dist; then
echo '❌ ERROR: PostHog references found in dist assets!'
echo 'PostHog must be properly tree-shaken from OSS builds.'
exit 1
fi
echo '✅ No PostHog references found'

View File

@@ -1,18 +0,0 @@
# Description: Validates JSON syntax in all tracked .json files (excluding tsconfig*.json) using jq
name: 'CI: JSON Validation'
on:
push:
branches:
- main
pull_request:
paths:
- '**/*.json'
jobs:
json-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Validate JSON syntax
run: ./scripts/cicd/check-json.sh

View File

@@ -1,102 +0,0 @@
# Description: Linting and code formatting validation for pull requests
name: 'CI: Lint Format'
on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: write
pull-requests: write
jobs:
lint-and-format:
runs-on: ubuntu-latest
steps:
- name: Checkout PR
uses: actions/checkout@v6
with:
ref: ${{ !github.event.pull_request.head.repo.fork && github.head_ref || github.ref }}
token: ${{ !github.event.pull_request.head.repo.fork && secrets.PR_GH_TOKEN || github.token }}
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Detect browser_tests changes
id: changed-paths
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
with:
filters: |
browser_tests:
- 'browser_tests/**'
- name: Run ESLint with auto-fix
run: pnpm lint:fix
- name: Run Stylelint with auto-fix
run: pnpm stylelint:fix
- name: Run oxfmt with auto-format
run: pnpm format
- name: Check for changes
id: verify-changed-files
run: |
if [ -n "$(git status --porcelain)" ]; then
echo "changed=true" >> $GITHUB_OUTPUT
else
echo "changed=false" >> $GITHUB_OUTPUT
fi
- name: Commit changes
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name == github.repository
run: |
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add .
git commit -m "[automated] Apply ESLint and Oxfmt fixes"
git push
- name: Fail for fork PRs with unfixed lint/format issues
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name != github.repository
run: |
echo "::error::Linting/formatting issues found. Since this PR is from a fork, auto-fix cannot be applied automatically."
echo ""
echo "Please run these commands locally and push the changes:"
echo " pnpm lint:fix"
echo " pnpm stylelint:fix"
echo " pnpm format"
echo ""
echo "Or set up pre-commit hooks to automatically format on every commit:"
echo " pnpm prepare"
echo ""
echo "See CONTRIBUTING.md for more details."
exit 1
- name: Final validation
run: |
pnpm lint
pnpm stylelint
pnpm format:check
pnpm knip
- name: Typecheck browser tests
if: steps.changed-paths.outputs.browser_tests == 'true'
run: pnpm typecheck:browser
- name: Comment on PR about auto-fix
if: steps.verify-changed-files.outputs.changed == 'true' && github.event.pull_request.head.repo.full_name == github.repository
continue-on-error: true
uses: actions/github-script@v8
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '## 🔧 Auto-fixes Applied\n\nThis PR has been automatically updated to fix linting and formatting issues.\n\n**⚠️ Important**: Your local branch is now behind. Run `git pull` before making additional changes to avoid conflicts.\n\n### Changes made:\n- ESLint auto-fixes\n- Oxfmt formatting'
})

View File

@@ -1,115 +0,0 @@
name: 'CI: OSS Assets Validation'
on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
push:
branches: [main, dev*]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
validate-fonts:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build project
run: pnpm build
env:
DISTRIBUTION: localhost
- name: Check for proprietary fonts in dist
run: |
set -euo pipefail
echo '🔍 Checking dist for proprietary ABCROM fonts...'
if [ ! -d "dist" ] || [ -z "$(ls -A dist)" ]; then
echo '❌ ERROR: dist/ directory missing or empty!'
exit 1
fi
# Check for ABCROM font files
if find dist/ -type f -iname '*abcrom*' \
\( -name '*.woff' -o -name '*.woff2' -o -name '*.ttf' -o -name '*.otf' \) \
-print -quit | grep -q .; then
echo ''
echo '❌ ERROR: Found proprietary ABCROM font files in dist!'
echo ''
find dist/ -type f -iname '*abcrom*' \
\( -name '*.woff' -o -name '*.woff2' -o -name '*.ttf' -o -name '*.otf' \)
echo ''
echo 'ABCROM fonts are proprietary and should not ship to OSS builds.'
echo ''
echo 'To fix this:'
echo '1. Use conditional font loading based on isCloud'
echo '2. Ensure fonts are dynamically imported, not bundled'
echo '3. Check vite config for font handling'
exit 1
fi
echo '✅ No proprietary fonts found in dist'
validate-licenses:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
- name: Use Node.js
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with:
node-version-file: '.nvmrc'
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Validate production dependency licenses
run: |
set -euo pipefail
echo '🔍 Checking production dependency licenses...'
# Use license-checker-rseidelsohn (actively maintained fork, handles monorepos)
# Exclude internal @comfyorg packages from license check
# Run in if condition to capture exit code
if npx license-checker-rseidelsohn@4 \
--production \
--summary \
--excludePackages '@comfyorg/comfyui-frontend;@comfyorg/design-system;@comfyorg/ingest-types;@comfyorg/registry-types;@comfyorg/shared-frontend-utils;@comfyorg/tailwind-utils;@comfyorg/comfyui-electron-types' \
--clarificationsFile .github/license-clarifications.json \
--onlyAllow 'MIT;MIT*;Apache-2.0;BSD-2-Clause;BSD-3-Clause;ISC;0BSD;BlueOak-1.0.0;Python-2.0;CC0-1.0;Unlicense;(MIT OR Apache-2.0);(MIT OR GPL-3.0);(Apache-2.0 OR MIT);(MPL-2.0 OR Apache-2.0);CC-BY-4.0;CC-BY-3.0;GPL-3.0-only'; then
echo ''
echo '✅ All production dependency licenses are approved!'
else
echo ''
echo '❌ ERROR: Found dependencies with non-approved licenses!'
echo ''
echo 'To fix this:'
echo '1. Check the license of the problematic package'
echo '2. Find an alternative package with an approved license'
echo '3. If the license is safe and OSI-approved, add it to the --onlyAllow list'
echo ''
echo 'For more info on OSI-approved licenses:'
echo 'https://opensource.org/licenses'
exit 1
fi

View File

@@ -1,115 +0,0 @@
name: 'CI: Performance Report'
on:
push:
branches: [main, core/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
concurrency:
group: perf-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
jobs:
perf-tests:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
timeout-minutes: 30
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.12
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
permissions:
contents: write
packages: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Setup frontend
uses: ./.github/actions/setup-frontend
with:
include_build_step: true
- name: Start ComfyUI server
uses: ./.github/actions/start-comfyui-server
- name: Run performance tests
id: perf
continue-on-error: true
run: pnpm exec playwright test --project=performance --workers=1 --repeat-each=3
- name: Upload perf metrics
if: always()
uses: actions/upload-artifact@v6
with:
name: perf-metrics
path: test-results/perf-metrics.json
retention-days: 30
if-no-files-found: warn
- name: Save PR metadata
if: github.event_name == 'pull_request'
run: |
mkdir -p temp/perf-meta
echo "${{ github.event.number }}" > temp/perf-meta/number.txt
echo "${{ github.event.pull_request.base.ref }}" > temp/perf-meta/base.txt
echo "${{ github.event.pull_request.head.sha }}" > temp/perf-meta/head-sha.txt
- name: Upload PR metadata
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v6
with:
name: perf-meta
path: temp/perf-meta/
- name: Save perf baseline to perf-data branch
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.perf.outcome == 'success'
continue-on-error: true
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git config url."https://x-access-token:${GH_TOKEN}@github.com/".insteadOf "https://github.com/"
cp test-results/perf-metrics.json /tmp/perf-metrics.json
git fetch origin perf-data || {
echo "Creating perf-data branch"
git checkout --orphan perf-data
git rm -rf . 2>/dev/null || true
echo "# Performance Baselines" > README.md
mkdir -p baselines
git add README.md baselines
git commit -m "Initialize perf-data branch"
git push origin perf-data
git fetch origin perf-data
}
git worktree add /tmp/perf-data origin/perf-data
TIMESTAMP=$(date -u +%Y%m%dT%H%M%SZ)
SHA=$(echo "${{ github.sha }}" | cut -c1-8)
mkdir -p /tmp/perf-data/baselines
cp /tmp/perf-metrics.json "/tmp/perf-data/baselines/perf-${TIMESTAMP}-${SHA}.json"
# Keep only last 20 baselines
cd /tmp/perf-data
ls -t baselines/perf-*.json 2>/dev/null | tail -n +21 | xargs -r rm
git -C /tmp/perf-data add baselines/
git -C /tmp/perf-data commit -m "perf: add baseline for ${SHA}" || echo "No changes to commit"
git -C /tmp/perf-data push origin HEAD:perf-data
git worktree remove /tmp/perf-data --force 2>/dev/null || true
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,26 +0,0 @@
# Description: Runs shellcheck on tracked shell scripts when they change
name: 'CI: Shell Validation'
on:
push:
branches:
- main
paths:
- '**/*.sh'
pull_request:
paths:
- '**/*.sh'
jobs:
shell-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install shellcheck
run: |
sudo apt-get update
sudo apt-get install -y shellcheck
- name: Run shellcheck
run: bash ./scripts/cicd/check-shell.sh

View File

@@ -1,46 +0,0 @@
name: 'CI: Size Data'
on:
push:
branches:
- main
pull_request:
branches:
- main
concurrency:
group: size-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
collect:
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Build project
run: pnpm build
- name: Collect size data
run: node scripts/size-collect.js
- name: Save PR metadata
if: ${{ github.event_name == 'pull_request' }}
run: |
echo ${{ github.event.number }} > ./temp/size/number.txt
echo ${{ github.base_ref }} > ./temp/size/base.txt
echo ${{ github.event.pull_request.head.sha }} > ./temp/size/head-sha.txt
- name: Upload size data
uses: actions/upload-artifact@v6
with:
name: size-data
path: temp/size

View File

@@ -1,240 +0,0 @@
# Description: End-to-end testing with Playwright across multiple browsers, deploys test reports to Cloudflare Pages
name: 'CI: Tests E2E'
on:
push:
branches: [main, master, core/*, desktop/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
setup:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
with:
include_build_step: true
# Upload only built dist/ (containerized test jobs will pnpm install without cache)
- name: Upload built frontend
uses: actions/upload-artifact@v6
with:
name: frontend-dist
path: dist/
retention-days: 1
# Build cloud distribution for @cloud tagged tests
# NX_SKIP_NX_CACHE=true is required because `nx build` was already run
# for the OSS distribution above. Without skipping cache, Nx returns
# the cached OSS build since env vars aren't part of the cache key.
- name: Build cloud frontend
run: NX_SKIP_NX_CACHE=true pnpm build:cloud
- name: Upload cloud frontend
uses: actions/upload-artifact@v6
with:
name: frontend-dist-cloud
path: dist/
retention-days: 1
# Sharded chromium tests
playwright-tests-chromium-sharded:
needs: setup
runs-on: ubuntu-latest
timeout-minutes: 60
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.16
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
permissions:
contents: read
packages: read
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4, 5, 6, 7, 8]
shardTotal: [8]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Download built frontend
uses: actions/download-artifact@v7
with:
name: frontend-dist
path: dist/
- name: Start ComfyUI server
uses: ./.github/actions/start-comfyui-server
- name: Install frontend deps
run: pnpm install --frozen-lockfile
# Run sharded tests (browsers pre-installed in container)
- name: Run Playwright tests (Shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
id: playwright
run: pnpm exec playwright test --project=chromium --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --reporter=blob
env:
PLAYWRIGHT_BLOB_OUTPUT_DIR: ./blob-report
- name: Upload blob report
uses: actions/upload-artifact@v6
if: ${{ !cancelled() }}
with:
name: blob-report-chromium-${{ matrix.shardIndex }}
path: blob-report/
retention-days: 1
playwright-tests:
# Ideally, each shard runs test in 6 minutes, but allow up to 15 minutes
timeout-minutes: 15
needs: setup
runs-on: ubuntu-latest
container:
image: ghcr.io/comfy-org/comfyui-ci-container:0.0.16
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
permissions:
contents: read
packages: read
strategy:
fail-fast: false
matrix:
browser: [chromium-2x, chromium-0.5x, mobile-chrome, cloud]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Download built frontend
uses: actions/download-artifact@v7
with:
name: ${{ matrix.browser == 'cloud' && 'frontend-dist-cloud' || 'frontend-dist' }}
path: dist/
- name: Start ComfyUI server
uses: ./.github/actions/start-comfyui-server
- name: Install frontend deps
run: pnpm install --frozen-lockfile
# Run tests (browsers pre-installed in container)
- name: Run Playwright tests (${{ matrix.browser }})
id: playwright
run: pnpm exec playwright test --project=${{ matrix.browser }} --reporter=blob
env:
PLAYWRIGHT_BLOB_OUTPUT_DIR: ./blob-report
- name: Generate HTML and JSON reports
if: always()
run: |
# Generate HTML report from blob
pnpm exec playwright merge-reports --reporter=html ./blob-report
# Generate JSON report separately with explicit output path
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report/report.json \
pnpm exec playwright merge-reports --reporter=json ./blob-report
- name: Upload Playwright report
uses: actions/upload-artifact@v6
if: always()
with:
name: playwright-report-${{ matrix.browser }}
path: ./playwright-report/
retention-days: 30
# Merge sharded test reports (no container needed - only runs CLI)
merge-reports:
needs: [playwright-tests-chromium-sharded]
runs-on: ubuntu-latest
if: ${{ !cancelled() }}
steps:
- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0
with:
version: 10
- name: Download blob reports
uses: actions/download-artifact@v7
with:
path: ./all-blob-reports
pattern: blob-report-chromium-*
merge-multiple: true
- name: Merge into HTML Report
run: |
# Generate HTML report
pnpm dlx @playwright/test merge-reports --reporter=html ./all-blob-reports
# Generate JSON report separately with explicit output path
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report/report.json \
pnpm dlx @playwright/test merge-reports --reporter=json ./all-blob-reports
- name: Upload HTML report
uses: actions/upload-artifact@v6
with:
name: playwright-report-chromium
path: ./playwright-report/
retention-days: 30
#### BEGIN Deployment and commenting (non-forked PRs only)
# when using pull_request event, we have permission to comment directly
# if its a forked repo, we need to use workflow_run event in a separate workflow (pr-playwright-deploy.yaml)
# Post starting comment for non-forked PRs
comment-on-pr-start:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Post starting comment
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
chmod +x scripts/cicd/pr-playwright-deploy-and-comment.sh
./scripts/cicd/pr-playwright-deploy-and-comment.sh \
"${{ github.event.pull_request.number }}" \
"${{ github.head_ref }}" \
"starting"
# Deploy and comment for non-forked PRs only
deploy-and-comment:
needs: [playwright-tests, merge-reports]
runs-on: ubuntu-latest
if: always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
permissions:
pull-requests: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Download all playwright reports
uses: actions/download-artifact@v7
with:
pattern: playwright-report-*
path: reports
- name: Deploy reports and comment on PR
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
GITHUB_TOKEN: ${{ github.token }}
GITHUB_SHA: ${{ github.event.pull_request.head.sha }}
run: |
bash ./scripts/cicd/pr-playwright-deploy-and-comment.sh \
"${{ github.event.pull_request.number }}" \
"${{ github.head_ref }}" \
"completed"
#### END Deployment and commenting (non-forked PRs only)

View File

@@ -1,35 +0,0 @@
# Description: Unit and component testing with Vitest + coverage reporting
name: 'CI: Tests Unit'
on:
push:
branches: [main, master, dev*, core/*, desktop/*]
paths-ignore: ['**/*.md']
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths-ignore: ['**/*.md']
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Run Vitest tests with coverage
run: pnpm test:coverage
- name: Upload coverage to Codecov
if: always()
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
with:
files: coverage/lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false

View File

@@ -1,21 +0,0 @@
name: Validate Action SHA Pins
on:
pull_request:
paths:
- '.github/workflows/**'
- '.github/actions/**'
- '.pinact.yaml'
permissions:
contents: read
jobs:
validate-pins:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: suzuki-shunsuke/pinact-action@3d49c6412901042473ffa78becddab1aea46bbea # v1.3.1
with:
skip_push: 'true'

View File

@@ -1,33 +0,0 @@
# Description: Build and validate the marketing website (apps/website)
name: 'CI: Website Build'
on:
push:
branches: [main, master, website/*]
paths:
- 'apps/website/**'
- 'packages/design-system/**'
- 'pnpm-lock.yaml'
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
paths:
- 'apps/website/**'
- 'packages/design-system/**'
- 'pnpm-lock.yaml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Build website
run: pnpm --filter @comfyorg/website build

View File

@@ -1,33 +0,0 @@
# Description: Validates YAML syntax and style using yamllint with relaxed rules
name: 'CI: YAML Validation'
on:
push:
branches:
- main
paths:
- '**/*.yml'
- '**/*.yaml'
pull_request:
paths:
- '**/*.yml'
- '**/*.yaml'
jobs:
yaml-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.x'
- name: Install yamllint
run: |
python -m pip install --upgrade pip
python -m pip install yamllint
- name: Validate YAML syntax and style
run: ./scripts/cicd/check-yaml.sh

Some files were not shown because too many files have changed in this diff Show More