Commit Graph

63 Commits

Author SHA1 Message Date
snomiao
48d01745cd feat: add test count display to Playwright PR comments (#5458)
* feat: add test count display to Playwright PR comments

- Add extract-playwright-counts.mjs script to parse test results from Playwright reports
- Update pr-playwright-deploy-and-comment.sh to extract and display test counts
- Show overall summary with passed/failed/flaky/skipped counts
- Display per-browser test counts inline with report links
- Use dynamic status icons based on test results (//⚠️)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: include skipped test count in per-browser display

- Add skipped test extraction for individual browser reports
- Update per-browser display format to show all four counts:
  ( passed /  failed / ⚠️ flaky / ⏭️ skipped)
- Provides complete test result visibility at a glance

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: improve test count extraction reliability in CI

- Use absolute paths for script and report directories
- Add debug logging to help diagnose extraction issues
- Move counts display after View Report link as requested
- Format: [View Report](url) •  passed /  failed / ⚠️ flaky / ⏭️ skipped

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: generate JSON reports alongside HTML for test count extraction

- Add JSON reporter to Playwright test runs
- Generate report.json alongside HTML reports
- Store JSON report in playwright-report directory
- This enables accurate test count extraction from CI artifacts

The HTML reports alone don't contain easily extractable test statistics
as they use a React app with dynamically loaded data. JSON reports
provide direct access to test counts.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: correct JSON reporter syntax for Playwright tests

- Use proper syntax for JSON reporter with outputFile option
- Run separate commands for HTML and JSON report merging
- Specify output path directly in reporter configuration
- Ensures report.json is created in playwright-report directory

This fixes the "No such file or directory" error when trying to move
report.json file, as it wasn't being created in the first place.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Revert "fix: correct JSON reporter syntax for Playwright tests"

This reverts commit 605d7cc1e2.

* fix: use correct Playwright reporter syntax with comma-separated list

- Use --reporter=html,json syntax (comma-separated, not space)
- Move test-results.json to playwright-report/report.json after generation
- Remove incorrect PLAYWRIGHT_JSON_OUTPUT_NAME env variable
- Add || true to prevent failure if JSON file doesn't exist

The JSON reporter outputs to test-results.json by default when using
the comma-separated reporter list syntax.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: improve test count extraction reliability in CI

- Use separate --reporter flags for list, html, and json
- Set PLAYWRIGHT_JSON_OUTPUT_NAME env var to specify JSON output path
- Run HTML and JSON report generation separately for merged reports
- Ensures report.json is created in playwright-report directory

The combined reporter syntax wasn't creating the JSON file properly.
Using separate reporter flags with env var ensures JSON is generated.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update scripts/cicd/pr-playwright-deploy-and-comment.sh

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* refactor: convert extraction script to TypeScript and use tsx

- Convert extract-playwright-counts.mjs to TypeScript (.ts)
- Add proper TypeScript types for better type safety
- Use tsx for execution instead of node
- Auto-install tsx in CI if not available
- Better alignment with the TypeScript codebase

This provides better type safety and consistency with the rest of
the codebase while maintaining the same functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(pr-playwright-deploy-and-comment.sh): move tsx installation check to the beginning of the script for better organization and efficiency

* [auto-fix] Apply ESLint and Prettier fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: GitHub Action <action@github.com>
2025-09-13 01:18:18 -07:00
snomiao
6fbd692370 fix: Split PR deployment workflow for forked vs non-forked repos (#5425)
* [fix] Consolidate Playwright workflow jobs to fix missing deployment links

The issue in PR #5298 was caused by missing deployment-info artifact
creation. The deploy-reports job was deploying to Cloudflare but wasn't
creating the deployment-info-* artifacts that comment-tests-completed
job expected to download.

This change consolidates the deployment and commenting into a single job,
eliminating the artifact dependency and ensuring links are always available.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Split PR deployment workflow for forked vs non-forked repos

- Extract deployment logic to reusable script (scripts/cicd/pr-playwright-deploy-and-comment.sh)
- Non-forked PRs: Use direct pull_request event in test-ui.yaml (faster)
- Forked PRs: Use workflow_run in pr-playwright-deploy.yaml (handles permissions)
- Add starting comment for both forked and non-forked PRs
- Make Cloudflare tokens optional for starting status comments

* refactor: Simplify PR deployment workflow and script

- Consolidate workflow into single job with clearer structure
- Reduce script from 200+ to ~140 lines
- Simplify deployment retry logic and comment generation
- Remove redundant checks and unnecessary complexity

* fix: Add debugging and wrangler installation to deployment script

- Add debug output to identify missing reports
- Install wrangler if not available
- Show deployment attempts and failures
- Log available reports before deployment

* chore: Trigger CI to test deployment workflow

* fix: Fix browser artifact name mismatch in deployment script

- Use dot notation (0.5x) for artifact names as Playwright creates them
- Convert to dash notation (0-5x) for Cloudflare project names
- Properly handle browser name display in comments

* refactor: Convert deployment script to POSIX sh for better compatibility

- Replace bash arrays with space-separated strings
- Use while loops instead of bash-specific for syntax
- Remove bash-specific string manipulation features
- Replace local variables (not required in functions)
- Ensure compatibility with standard /bin/sh

* fix: Fix deployment script output to properly capture URLs

- Redirect debug messages to stderr
- Only output URL to stdout for proper capture
- This fixes the missing deployment links in PR comments

* fix: Add input validation to prevent command injection

- Validate PR number is numeric only
- Sanitize branch name at script start
- Validate status parameter values
- Use pre-sanitized branch throughout script
- Addresses high-severity security issue from PR review

* fix: Add null checks and logging to workflow condition

- Add explicit null checks for head_repository and repository
- Add debug logging to help diagnose workflow trigger issues
- Prevents potential failures from undefined repository objects
- Addresses medium-severity issue from PR review

* fix: Pin wrangler to major version 4 with error handling

- Pin wrangler to major version 4 (^4.0.0) for stability
- Add error handling if wrangler installation fails
- Return 'failed' status if installation fails
- Addresses dependency management issue from PR review

* perf: Implement parallel deployments to reduce CI time

- Deploy all browser reports in parallel using background processes
- Use temporary directory to collect deployment results
- Wait for all deployments to complete before generating comment
- Maintains result order for consistent output
- Significantly reduces deployment time from sequential to parallel execution

* fix: Use specific comment ID for updates instead of edit-last

- Use GitHub API to find exact comment ID
- Update specific comment by ID to avoid editing wrong comment
- Prevents race conditions if user posts between finding and editing
- More reliable comment updates

* fix(workflows/test-ui.yaml): change condition to always run deploy job for pull requests to ensure deployment consistency

* fix(workflows/test-ui.yaml): change condition to always run deploy job for pull requests to ensure deployment consistency

* fix(pr-playwright-deploy-and-comment.sh): remove npx prefix from wrangler command for consistency and simplicity

* fix(pr-playwright-deploy-and-comment.sh): remove npx prefix from wrangler command for consistency and simplicity

* Update scripts/cicd/pr-playwright-deploy-and-comment.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(pr-playwright-deploy-and-comment.sh): improve regex for URL extraction to include valid characters and ensure correct URL format

* chore(pr-playwright-deploy-and-comment.sh): move wrangler installation to the beginning of the script to avoid redundancy and improve efficiency

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-08 23:11:26 -07:00
Alexander Brown
58313ea05b devex: re-add caching of playwright browser installs (#5350) 2025-09-04 12:18:11 -07:00
Johnpaul Chiwetelu
3c247e57f3 Remove caching of Playwright browsers from CI workflow (#5325)
* [refactor] Remove caching of Playwright browsers from CI workflow

* restore upload artifact path

* [fix] Adjust timeout for playwright tests to allow up to 15 minutes

* [fix] Update artifact name for Playwright report to match standard naming convention
2025-09-03 23:22:36 +01:00
Johnpaul Chiwetelu
caee3832a5 Playwright Test Sharding (#5311)
* [feat] Enhance Playwright testing workflow with sharding and report merging

* feat: shard playwright tests into 8 processes
2025-09-03 12:16:37 -07:00
snomiao
c3c2681819 [feat] Implement pull_request_target deployment for forked PRs - solves secret access issue (#5209)
* [feat] Fix CI workflow issues for forked PRs and improve test diagnostics

This commit addresses two critical blockers in the CI workflow:

1. **Cloudflare Token Access Issue**:
   - Added conditional deployment that skips Cloudflare Pages for forked PRs
   - Forked PRs now get artifact-based report access instead of live URLs
   - Maintains security by preventing secret access from external repos

2. **Test Startup Issues**:
   - Enhanced ComfyUI server startup with better diagnostics
   - Added server PID tracking and process status verification
   - Improved error messages and timeout handling

Additional improvements:
- Updated PR comment logic to handle both deployment scenarios
- Added FORK_TESTING.md documentation for contributors
- Enhanced deployment info handling for summary generation

Fixes #5207

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat] Implement pull_request_target deployment for forked PRs

Add secure deployment solution for Playwright reports from forked PRs using pull_request_target event.

Key Changes:
- Add deploy-playwright-reports.yaml workflow using pull_request_target
- Update test-ui.yaml to work with new deployment approach
- Add comprehensive security documentation

Security Features:
- No untrusted code execution (artifacts only)
- Follows GitHub security best practices
- Maintains full secret access for deployment
- Clear audit trail and logging

Fixes #5207

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat] Implement cost-optimized deployment with webhook triggers

Replace expensive polling mechanism with repository_dispatch webhooks to reduce GitHub Actions costs by 85%.

Key improvements:
- Remove 30-minute polling wait in deploy-playwright-reports.yaml
- Add repository_dispatch trigger for instant deployment activation
- Implement concurrency controls to prevent redundant deployments
- Add webhook trigger from test completion in test-ui.yaml
- Maintain security and forked PR support

Cost benefits:
- Eliminates 4 Ubuntu runners waiting up to 30min each
- Reduces API calls from 240+ to 1 per PR
- Event-driven architecture for better reliability
- No timeout risks or polling overhead

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [cleanup] Simplify PR testing approach per review feedback

- Revert enhanced ComfyUI server startup logging
- Remove complex fork handling and webhook triggers
- Simplify Cloudflare deployment to original approach
- Remove FORK_TESTING.md and PULL_REQUEST_TARGET_DEPLOYMENT.md files
- Remove deploy-playwright-reports.yaml workflow
- Documentation moved to PR comments for better visibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat] Implement workflow_run architecture for CI comment/deploy separation

Restructures CI workflows to use workflow_run triggers, improving forked PR support and simplifying core testing workflows.

- pr-playwright-comment.yaml - Comments Playwright test results after Tests CI completion
- pr-storybook-comment.yaml - Comments Storybook build status after Chromatic completion
- pr-playwright-deploy.yaml - Deploys Playwright reports with secret access after Tests CI completion

- chromatic.yaml - Removed all commenting logic, focused on Chromatic testing only
- test-ui.yaml - Removed deployment, commenting, and comment-summary job; focused on Playwright testing only

-  Better forked PR support - workflow_run has access to secrets for deployment
-  Cleaner separation of concerns - testing vs commenting/deployment
-  Reduced complexity in core testing workflows
-  Improved reliability for external contributors

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat] Implement workflow_run for both start and completion events

- Updated pr-playwright-comment.yaml to trigger on both 'requested' and 'completed' events
- Updated pr-storybook-comment.yaml to trigger on both 'requested' and 'completed' events
- Added conditional logic to post different messages for workflow start vs completion
- Added "Tests are starting..." message when workflows begin
- Added "Build is starting..." message for Storybook builds
- Maintained existing completion logic with full test results and reports

This allows users to see immediate feedback when their workflows start running,
improving the user experience by providing real-time status updates.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [cleanup] Remove continue-on-error from comment workflows

Comment workflow failures should be visible rather than silently ignored.
This allows better debugging when PR comments fail to post.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [feat] Add logging when no PR found in comment workflows

- Add explicit logging step when steps.pr.outputs.result == 'null'
- Shows branch name, workflow run ID, repository, and event details
- Improves debugging when workflow_run triggers but finds no open PR
- Helps identify issues with branch name matching or PR state

Previously these workflows would silently skip all steps when no PR was found,
making it difficult to debug why comments weren't being posted.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update workflow formatting

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [security] Implement security hardening for workflow_run workflows

- Add URL sanitization for deployment report links to prevent malicious URL injection
- Pin third-party GitHub Actions to commit hashes for supply chain security
- Add repository validation checks to prevent workflow misconfiguration
- Validate deployment URLs against pages.dev pattern before using in comments

Following security recommendations from code review to implement defense-in-depth.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [security] Pin only third-party actions to commit hashes

Keep official GitHub actions (actions/github-script, actions/download-artifact) pinned to version tags as they are trusted first-party actions, while only pinning third-party edumserrano/find-create-or-update-comment to commit hash for supply chain security.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-30 11:04:01 -07:00
Christian Byrne
1bdc190154 [feat] Replace Claude loading GIF with ComfyUI logo in test workflow (#5229)
Update the GitHub workflow to use ComfyUI's animated favicon progress
frames instead of the Claude logo for PR test status comments.
2025-08-27 16:38:19 +00:00
Alexander Brown
1b83d6b5a6 Migration: pnpm (#5215)
* migration: npm to pnpm
Step 1, package and lockfile

* migration: npm to pnpm
Step 2: docs / LLM instructions

* migration: npm to pnpm
Step 3: More documentation updates

* migration: npm to pnpm
Step 4: Even more documentation

* migration: npm to pnpm
Step 5: GitHub Actions

* migration: npm to pnpm
Step 6: PNPM installation in actions. This merge is going to be painful.

* migration: npm to pnpm
Unignore and add pnpm lockfile.

* migration: npm to pnpm
package-lock.json -> pnpm-lock.yaml

* migration: explicit @primeuix/styled, move glob to prod deps

* migration: more explicit deps required by the importmap plugin and vite

* fix: missed merge artifact

* fix: Make sure pnpm is available to install wrangler

* migration: pnpm for dev-release.yaml

* migration: new setup action version
Won't work until that is updated and a new release is cut.

* migration: Playwright needs uuid

* migration: Add explicit deps for lobehub

* chore(version-bump.yaml): change cache from npm to pnpm to optimize package management and improve build performance

* migration: install pnpm in version-bump action

---------

Co-authored-by: snomiao <snomiao@gmail.com>
2025-08-27 06:10:15 -07:00
snomiao
62e06f4358 [ci] Enhance CI/CD caching across all workflows (#5117)
* [ci] Enhance CI/CD caching across all workflows

- Add tool cache steps for node_modules/.cache, .cache, and .eslintcache
- Enable npm caching for Node.js setup actions where missing
- Add cache configurations for ESLint, Prettier, Knip, and other build tools
- Improve build performance by caching tool outputs between runs
- Use unique cache keys per workflow to avoid conflicts

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [ci] Enhance CI/CD caching with comprehensive improvements

Extends caching strategy beyond PR #5107 with additional optimizations:

- **i18n.yaml**: Add tool cache for locale generation workflow
- **update-electron-types.yaml**: Add tool cache for type updates
- **test-ui.yaml**: Add Playwright browser cache and Python pip cache
- **vitest.yaml**: Add coverage directory caching
- **chromatic.yaml**: Add Storybook static build cache
- **All build workflows**: Add TypeScript incremental build cache
- **Type generation workflows**: Add repository caching for external repos
- **lint-and-format.yaml**: Improve cache key granularity with source hash

Performance improvements:
- Playwright browser downloads cached across runs
- Python pip dependencies cached
- TSC incremental compilation cache preserved
- Storybook static builds cached
- External repository clones cached
- More granular cache invalidation based on source changes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [ci] Optimize CI/CD caching for better performance

- Enhance cache keys with more specific file patterns for better cache hit rates
- Add missing cache directories (.prettierCache, .knip-cache, .vitest-cache, tsconfig.tsbuildinfo)
- Improve cache key naming for clarity (lint-format, vitest, storybook, playwright)
- Add better fallback cache restore keys
- Include browser-specific caching for Playwright tests
- Add TypeScript build info caching across workflows

Expected improvements:
- 20-40% faster builds on cache hits
- More granular cache invalidation
- Better cache utilization across tools

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Apply suggestion from @DrJKL

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* [ci] Remove redundant node_modules/.cache from workflow caches

setup-node already handles node_modules caching when cache: 'npm' is enabled.
Removed conflicting node_modules/.cache entries from:
- .github/workflows/lint-and-format.yaml
- .github/workflows/vitest.yaml
- .github/workflows/test-ui.yaml

This prevents cache conflicts and follows best practices for npm caching.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update .github/workflows/chromatic.yaml

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* Update .github/workflows/update-electron-types.yaml

Co-authored-by: Alexander Brown <drjkl@comfy.org>

* fix: address reviewer feedback on cache optimization PR (#5200)

* fix: remove duplicate cache entries and fix YAML formatting

- Remove duplicate path entries in dev-release.yaml
- Remove redundant node_modules/.cache entries since setup-node handles npm caching
- Fix YAML indentation issues across workflow files

Addresses reviewer feedback on PR #5117 about duplication and maintenance

* fix: add cache-dependency-path for setup-node in test-ui workflow

The test-ui workflow runs at repo root but package-lock.json is in ComfyUI_frontend/
subdirectory, causing setup-node cache to fail. Added cache-dependency-path to fix this.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alexander Brown <drjkl@comfy.org>
Co-authored-by: Christian Byrne <cbyrne@comfy.org>
2025-08-26 12:26:17 -07:00
snomiao
8646ca4162 [ci] Complete implementation of safe-to-fail CI steps (#5210)
* [ci] Make Playwright deploy step safe to fail

Add continue-on-error: true to Deploy to Cloudflare Pages step to prevent
Cloudflare API issues from blocking essential testing processes.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [ci] Make lint-and-format comment steps safe to fail

Add continue-on-error: true to PR comment steps in lint workflow:
- Comment on PR about auto-fix (line 63)
- Comment on PR about manual fix needed (line 76)

This prevents GitHub API permission errors from blocking
essential linting processes while maintaining comment functionality.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-26 09:25:35 -07:00
snomiao
c75255327a [ci] Make Chromatic and Playwright comment steps safe to fail (#5156)
Add continue-on-error: true to all PR comment steps in both chromatic.yaml and test-ui.yaml workflows to prevent GitHub API permission errors (403) from blocking essential CI processes.

Changes:
- chromatic.yaml: Added continue-on-error to 2 comment steps
- test-ui.yaml: Added continue-on-error to 4 comment steps

This ensures that visual testing (Chromatic) and browser testing (Playwright) continue to run even when PR commenting fails due to token permissions.

Fixes #5149

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-23 13:28:55 +08:00
snomiao
337fb2100a [ci] Add retry logic to wrangler page deploy step (#5118)
- Implement 3-attempt retry mechanism for Cloudflare Pages deployment
- Add 10-second delay between retry attempts
- Install wrangler globally to ensure CLI availability
- Maintain existing functionality while improving stability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 09:50:38 -07:00
snomiao
2240645a7d [feat] Add Cloudflare Pages deployment for Playwright test reports (#5045)
* [feat] Add Cloudflare Pages deployment for Playwright test reports

- Deploy test reports to separate Cloudflare projects per browser
- Add real-time PR comments with progressive test status updates
- Use wrangler-action for unified Cloudflare tooling
- Support cross-browser testing with individual report links
- Document CI/CD integration in browser_tests/README.md

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Cloudflare project name for chromium-0.5x browser

* Extract project name transformation to variable for consistent URL formatting

* chore(ci): update branch filters for push and pull_request events in test-ui workflow to refine CI triggers

* [feat] Improve test-ui deployment with branch isolation and building page

- Use Cloudflare Pages --branch flag for proper branch isolation instead of modifying project names
- Add auto-refresh building page that shows test progress in real-time
- Deploy building page immediately when tests start for instant feedback
- Update URL generation to use branch-based Cloudflare Pages URLs format
- Maintain clean project names while isolating branches properly

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): increase sleep duration from 5 to 10 seconds for cache propagation and restore cached setup steps for improved workflow efficiency

* [refactor] Remove building-page to reduce complexity

- Remove auto-refresh building page and related deployment steps
- Simplify PR comments to show basic test status without progress page
- Keep branch-based deployment for proper isolation while reducing complexity
- Maintain clean workflow focused on core functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): add a separator in the workflow file for better readability and organization of the test status section

* [feat] Add Cloudflare Pages deployment for Playwright test reports

- Deploy test reports to separate Cloudflare projects per browser
- Add real-time PR comments with progressive test status updates
- Use wrangler-action for unified Cloudflare tooling
- Support cross-browser testing with individual report links
- Document CI/CD integration in browser_tests/README.md

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Cloudflare project name for chromium-0.5x browser

* Extract project name transformation to variable for consistent URL formatting

* chore(ci): update branch filters for push and pull_request events in test-ui workflow to refine CI triggers

* [feat] Improve test-ui deployment with branch isolation and building page

- Use Cloudflare Pages --branch flag for proper branch isolation instead of modifying project names
- Add auto-refresh building page that shows test progress in real-time
- Deploy building page immediately when tests start for instant feedback
- Update URL generation to use branch-based Cloudflare Pages URLs format
- Maintain clean project names while isolating branches properly

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): increase sleep duration from 5 to 10 seconds for cache propagation and restore cached setup steps for improved workflow efficiency

* [refactor] Remove building-page to reduce complexity

- Remove auto-refresh building page and related deployment steps
- Simplify PR comments to show basic test status without progress page
- Keep branch-based deployment for proper isolation while reducing complexity
- Maintain clean workflow focused on core functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): add a separator in the workflow file for better readability and organization of the test status section

* [fix] Address PR review feedback - improve workflow architecture and security

- [HIGH] Fix continue-on-error masking by adding final test result check that fails CI on test failures
- [MEDIUM] Move branch sanitization to setup job to reduce performance overhead
- [MEDIUM] Add compatibility-date to Cloudflare deployment for stability
- [LOW] Extract date format to environment variable to follow DRY principle
- [LOW] Quote shell variables properly to prevent word splitting
- [LOW] Update documentation to use dynamic branch-specific URLs

Addresses all security, performance, and code quality issues raised in automated PR review.
Maintains test report deployment while ensuring CI integrity.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): replace loading emoji with an image for better visual consistency in test logs
style(test-ui.yaml): clean up whitespace in the workflow file for improved readability

* style(test-ui.yaml): format message to combine two lines into one for better readability

* chore(test-ui.yaml): add a blank line for better readability in the workflow file

* style(test-ui.yaml): update loading image alt text and format messages for better readability in GitHub Actions workflow

* [architecture] Separate test execution from deployment - clean CI design

BREAKING: Remove continue-on-error from test execution for proper CI integrity

**Clean Architecture Changes:**
- Remove `continue-on-error: true` from Playwright test execution
- Create separate `deploy-reports` job that always runs for debugging
- Test jobs now properly fail when tests fail (maintains CI integrity)
- Reports still deploy for debugging via dedicated deployment job
- Capture and pass actual exit codes between jobs via artifacts

**Benefits:**
-  CI fails when tests fail (no longer masked)
-  Reports still deploy for debugging regardless of test outcome
-  Clean separation of concerns (test vs deploy responsibilities)
-  Proper job dependencies and error handling
-  Individual browser test results preserved

**Job Flow:**
1. `setup` - Cache and prepare environment
2. `playwright-tests` - Run tests, fail if tests fail, upload artifacts
3. `deploy-reports` - Always deploy reports using artifacts (parallel)
4. `comment-summary` - Generate summary and fail workflow if needed

This addresses the high-priority architecture concern about continue-on-error
masking test failures while maintaining report deployment functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [refactor] Simplify deployment architecture - remove over-engineering

**Reverted to clean, simple approach based on feedback:**

1.  **Faster deployment** - Deploy immediately after each test (no waiting for matrix completion)
2.  **Remove unnecessary GITHUB_OUTPUT** - Don't save exit codes, use step.conclusion instead
3.  **Single job approach** - Use `if: always()` instead of separate deploy-reports job

**Key Changes:**
- Removed separate `deploy-reports` job (86 lines deleted!)
- Deploy in same job with `if: always()` - much faster
- Use `steps.playwright.conclusion` instead of captured exit codes
- Cleaner, simpler architecture with same functionality

**Benefits:**
- 🚀 **Much faster** - Reports deploy immediately per browser, not waiting for all tests
- 🧹 **Simpler** - One job handles test + deploy, easier to understand
-  **Still maintains CI integrity** - Tests fail properly when they should
- 📊 **Reports always deploy** - Available for debugging regardless of test outcome

The previous approach was over-engineered. This is much cleaner and faster.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(workflow): reorder condition in PR comment step for clarity and consistency

* chore(test-ui.yaml): update deployment command to remove compatibility date for better maintainability
docs(test-ui.yaml): add note to always() condition for clarity on artifact upload behavior

* [performance] Remove redundant branch sanitization - 75% processing reduction

**Issue**: Complex bash string operations running 4 times per build in matrix jobs
**Solution**: Remove duplicate branch sanitization, use pre-computed value from setup job

**Before**: Branch sanitization ran in both setup job AND each matrix job (5 total times)
**After**: Branch sanitization runs only once in setup job, reused via outputs

**Performance Impact**:
- 4 redundant tr/sed operations eliminated (matrix chromium, chromium-2x, chromium-0.5x, mobile-chrome)
- 75% reduction in branch name processing overhead
- Cleaner, more maintainable code

**Implementation**:
- Setup job: Computes `sanitized-branch` output once
- Matrix jobs: Use `${{ needs.setup.outputs.sanitized-branch }}` directly
- No duplicate string processing logic

Addresses PR review comment: [performance] medium Priority - Complex bash string operations in GitHub Actions matrix

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-19 10:22:45 -07:00
Christian Byrne
b1117b9838 [ci] Add chromium-0.5x to test matrix (#4880)
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-09 14:15:08 -07:00
Christian Byrne
8bf60777e7 [CI] Exclude vue-nodes-migration branch from playwright tests (#4844) 2025-08-08 16:52:25 -07:00
Christian Byrne
5c71854a96 [ci] Enable CI tests for all feature branch PRs (#4591) 2025-07-30 01:27:02 -07:00
filtered
ebedf1074d [CI] Fix intermittent actions/cache errors (#4220) 2025-06-18 03:55:05 -07:00
filtered
0832347f47 [CI] Fix intermittent failure when using actions/cache (#4219) 2025-06-18 01:24:42 -07:00
Christian Byrne
4cc6a15fde Fix refresh combos command fails on nodes using V2 spec (#3848) 2025-05-10 16:06:25 -04:00
Christian Byrne
2daa51421c Remove fetch-templates script (#3500) 2025-04-18 20:34:03 -04:00
Christian Byrne
87bf2310b6 Support previewing animated image uploads (#3479)
Co-authored-by: github-actions <github-actions@github.com>
2025-04-17 10:20:09 -04:00
Chenlei Hu
6e72e1924e Update ComfyUI_devtools version in CI (#3368) 2025-04-09 14:59:07 -04:00
Chenlei Hu
5b2b3cdacf [CI] Use gh action to update litegraph (#2808) 2025-03-02 16:57:30 -05:00
Chenlei Hu
9431c955a6 [CI] Include workflow templates at build time (#2775) 2025-03-02 15:22:18 -05:00
Chenlei Hu
ba4bb5774e [CI] Update frontend install method (#2783) 2025-03-01 10:10:04 -05:00
Chenlei Hu
a814f9f902 [Test] Run unittest with vitest (#2779) 2025-02-28 22:09:17 -05:00
Chenlei Hu
a244f295a6 Remove server elements from unit tests (#2777) 2025-02-28 20:01:40 -05:00
Chenlei Hu
3e54146afd [CI] Refactor test-ui gh action (#2776) 2025-02-28 19:46:57 -05:00
bymyself
74e8852958 Fix combo values from optional inputs not changed when refreshing (#2733) 2025-02-26 10:35:22 -05:00
filtered
bc43cf0290 [CI] Always test LTS branches (#2641)
- Adds core/* and desktop/* to CI testing worfklows
2025-02-20 13:51:17 +11:00
Margen67
7bc48c5074 Formatting/cleanup (#2594) 2025-02-17 10:10:00 -05:00
bymyself
cd8c0d2865 Add refresh button to remote (lazy) widgets (#2494) 2025-02-11 10:31:32 -05:00
bymyself
eeb1c34ada Update public API syntax of remote (lazy) widgets (#2477)
Co-authored-by: huchenlei <huchenlei@proton.me>
2025-02-09 12:41:14 -05:00
Chenlei Hu
91a3d1228e Reference specific ComfyUI_devtools in CI (#2479) 2025-02-09 12:00:26 -05:00
Chenlei Hu
642094ead1 Remove legacy slow jest tests (#2092) 2024-12-29 16:25:06 -05:00
Chenlei Hu
fc9e347055 Disable slow jest tests (#1591)
* Disable slow jest tests

* nit
2024-11-18 20:10:03 -05:00
filtered
caa3ac2068 Add playwright concurrency - multi-user mode (#1400)
* Add playwright concurrency - multi-user mode

* Add extra server params

* nit

* Update to v2.1 action

---------

Co-authored-by: huchenlei <huchenlei@proton.me>
2024-11-02 09:51:24 -04:00
Chenlei Hu
fdb01c06f2 Split playwright tests to multiple runners by project (#789) 2024-09-12 11:35:36 +09:00
Chenlei Hu
ca6bf7d054 Split jest unit test and playwright test into different actions (#787)
* Split jest unit test and playwright test into different actions

* Use composite action

* Add tag v1
2024-09-12 10:54:50 +09:00
Chenlei Hu
70d5e98c73 Update github action run conditions (#682) 2024-08-30 15:32:26 -04:00
Chenlei Hu
3e457f812d Execution Error Dialog Revamp (One click issue searching and filing) (#595)
* Add basic error dialog

* 2 level error report

* Add find issue button

* nit

* Add file issue button

* Single dialog

* nit

* Fix long text wrapping

* Merge component

* Test execution error dialog
2024-08-22 15:55:38 -04:00
pythongosssss
3e7b0a4907 Test using latest examples (#361) 2024-08-10 16:46:33 -04:00
Chenlei Hu
2aa9166079 Disable flux example workflow test (#278) 2024-08-01 21:17:28 -04:00
Chenlei Hu
89b195dc13 Only install chromium in github action (#210) 2024-07-23 14:21:26 -04:00
Chenlei Hu
631a060fff Enable CI for dev* branches (#165) 2024-07-19 12:23:48 -04:00
pythongosssss
32d81c13ad Add testing for ComfyUI examples (#95)
* Add testing for ComfyUI examples

* Remove examples, add test to github action

* Create dir

* Update readme
2024-07-07 10:38:28 -04:00
Chenlei Hu
a26802fea6 Change repo reference in CI workflow (#76)
* Swap checkout order

* Change owner of repo
2024-07-01 16:32:16 -04:00
Chenlei Hu
41df39d86c Fix CI ref (#74) 2024-07-01 13:10:42 -04:00
Chenlei Hu
fdee295265 Enable CI on push (#56) 2024-06-26 14:42:59 -04:00
Chenlei Hu
e1424a48ca Split github actions (#51) 2024-06-24 11:24:53 -04:00