Comfy-Org/ComfyUI_frontend
1
0
Fork 0
mirror of https://github.com/Comfy-Org/ComfyUI_frontend.git synced 2026-01-26 19:09:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

[feat] Add Cloudflare Pages deployment for Playwright test reports (#5045)

Browse Source 
* [feat] Add Cloudflare Pages deployment for Playwright test reports

- Deploy test reports to separate Cloudflare projects per browser
- Add real-time PR comments with progressive test status updates
- Use wrangler-action for unified Cloudflare tooling
- Support cross-browser testing with individual report links
- Document CI/CD integration in browser_tests/README.md

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Cloudflare project name for chromium-0.5x browser

* Extract project name transformation to variable for consistent URL formatting

* chore(ci): update branch filters for push and pull_request events in test-ui workflow to refine CI triggers

* [feat] Improve test-ui deployment with branch isolation and building page

- Use Cloudflare Pages --branch flag for proper branch isolation instead of modifying project names
- Add auto-refresh building page that shows test progress in real-time
- Deploy building page immediately when tests start for instant feedback
- Update URL generation to use branch-based Cloudflare Pages URLs format
- Maintain clean project names while isolating branches properly

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): increase sleep duration from 5 to 10 seconds for cache propagation and restore cached setup steps for improved workflow efficiency

* [refactor] Remove building-page to reduce complexity

- Remove auto-refresh building page and related deployment steps
- Simplify PR comments to show basic test status without progress page
- Keep branch-based deployment for proper isolation while reducing complexity
- Maintain clean workflow focused on core functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): add a separator in the workflow file for better readability and organization of the test status section

* [feat] Add Cloudflare Pages deployment for Playwright test reports

- Deploy test reports to separate Cloudflare projects per browser
- Add real-time PR comments with progressive test status updates
- Use wrangler-action for unified Cloudflare tooling
- Support cross-browser testing with individual report links
- Document CI/CD integration in browser_tests/README.md

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix Cloudflare project name for chromium-0.5x browser

* Extract project name transformation to variable for consistent URL formatting

* chore(ci): update branch filters for push and pull_request events in test-ui workflow to refine CI triggers

* [feat] Improve test-ui deployment with branch isolation and building page

- Use Cloudflare Pages --branch flag for proper branch isolation instead of modifying project names
- Add auto-refresh building page that shows test progress in real-time
- Deploy building page immediately when tests start for instant feedback
- Update URL generation to use branch-based Cloudflare Pages URLs format
- Maintain clean project names while isolating branches properly

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): increase sleep duration from 5 to 10 seconds for cache propagation and restore cached setup steps for improved workflow efficiency

* [refactor] Remove building-page to reduce complexity

- Remove auto-refresh building page and related deployment steps
- Simplify PR comments to show basic test status without progress page
- Keep branch-based deployment for proper isolation while reducing complexity
- Maintain clean workflow focused on core functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): add a separator in the workflow file for better readability and organization of the test status section

* [fix] Address PR review feedback - improve workflow architecture and security

- [HIGH] Fix continue-on-error masking by adding final test result check that fails CI on test failures
- [MEDIUM] Move branch sanitization to setup job to reduce performance overhead
- [MEDIUM] Add compatibility-date to Cloudflare deployment for stability
- [LOW] Extract date format to environment variable to follow DRY principle
- [LOW] Quote shell variables properly to prevent word splitting
- [LOW] Update documentation to use dynamic branch-specific URLs

Addresses all security, performance, and code quality issues raised in automated PR review.
Maintains test report deployment while ensuring CI integrity.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore(test-ui.yaml): replace loading emoji with an image for better visual consistency in test logs
style(test-ui.yaml): clean up whitespace in the workflow file for improved readability

* style(test-ui.yaml): format message to combine two lines into one for better readability

* chore(test-ui.yaml): add a blank line for better readability in the workflow file

* style(test-ui.yaml): update loading image alt text and format messages for better readability in GitHub Actions workflow

* [architecture] Separate test execution from deployment - clean CI design

BREAKING: Remove continue-on-error from test execution for proper CI integrity

**Clean Architecture Changes:**
- Remove `continue-on-error: true` from Playwright test execution
- Create separate `deploy-reports` job that always runs for debugging
- Test jobs now properly fail when tests fail (maintains CI integrity)
- Reports still deploy for debugging via dedicated deployment job
- Capture and pass actual exit codes between jobs via artifacts

**Benefits:**
- ✅ CI fails when tests fail (no longer masked)
- ✅ Reports still deploy for debugging regardless of test outcome
- ✅ Clean separation of concerns (test vs deploy responsibilities)
- ✅ Proper job dependencies and error handling
- ✅ Individual browser test results preserved

**Job Flow:**
1. `setup` - Cache and prepare environment
2. `playwright-tests` - Run tests, fail if tests fail, upload artifacts
3. `deploy-reports` - Always deploy reports using artifacts (parallel)
4. `comment-summary` - Generate summary and fail workflow if needed

This addresses the high-priority architecture concern about continue-on-error
masking test failures while maintaining report deployment functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [refactor] Simplify deployment architecture - remove over-engineering

**Reverted to clean, simple approach based on feedback:**

1. ✅ **Faster deployment** - Deploy immediately after each test (no waiting for matrix completion)
2. ✅ **Remove unnecessary GITHUB_OUTPUT** - Don't save exit codes, use step.conclusion instead
3. ✅ **Single job approach** - Use `if: always()` instead of separate deploy-reports job

**Key Changes:**
- Removed separate `deploy-reports` job (86 lines deleted!)
- Deploy in same job with `if: always()` - much faster
- Use `steps.playwright.conclusion` instead of captured exit codes
- Cleaner, simpler architecture with same functionality

**Benefits:**
- 🚀 **Much faster** - Reports deploy immediately per browser, not waiting for all tests
- 🧹 **Simpler** - One job handles test + deploy, easier to understand
- ✅ **Still maintains CI integrity** - Tests fail properly when they should
- 📊 **Reports always deploy** - Available for debugging regardless of test outcome

The previous approach was over-engineered. This is much cleaner and faster.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(workflow): reorder condition in PR comment step for clarity and consistency

* chore(test-ui.yaml): update deployment command to remove compatibility date for better maintainability
docs(test-ui.yaml): add note to always() condition for clarity on artifact upload behavior

* [performance] Remove redundant branch sanitization - 75% processing reduction

**Issue**: Complex bash string operations running 4 times per build in matrix jobs
**Solution**: Remove duplicate branch sanitization, use pre-computed value from setup job

**Before**: Branch sanitization ran in both setup job AND each matrix job (5 total times)
**After**: Branch sanitization runs only once in setup job, reused via outputs

**Performance Impact**:
- 4 redundant tr/sed operations eliminated (matrix chromium, chromium-2x, chromium-0.5x, mobile-chrome)
- 75% reduction in branch name processing overhead
- Cleaner, more maintainable code

**Implementation**:
- Setup job: Computes `sanitized-branch` output once
- Matrix jobs: Use `${{ needs.setup.outputs.sanitized-branch }}` directly
- No duplicate string processing logic

Addresses PR review comment: [performance] medium Priority - Complex bash string operations in GitHub Actions matrix

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
snomiao
2025-08-20 01:22:45 +08:00
committed by GitHub
parent cf9847a11e
commit 2240645a7d
2 changed files with 262 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

225
.github/workflows/test-ui.yaml vendored
View File

@@ -4,13 +4,18 @@ on:
push:
branches: [main, master, core/*, desktop/*]
pull_request:
branches-ignore: [wip/*, draft/*, temp/*, vue-nodes-migration]
branches-ignore:
[wip/*, draft/*, temp/*, vue-nodes-migration, sno-playwright-*]
env:
DATE_FORMAT: '+%m/%d/%Y, %I:%M:%S %p'
jobs:
setup:
runs-on: ubuntu-latest
outputs:
cache-key: ${{ steps.cache-key.outputs.key }}
sanitized-branch: ${{ steps.branch-info.outputs.sanitized }}
steps:
- name: Checkout ComfyUI
uses: actions/checkout@v4
@@ -36,6 +41,29 @@ jobs:
with:
node-version: lts/*
- name: Get current time
id: current-time
run: echo "time=$(date -u '${{ env.DATE_FORMAT }}')" >> $GITHUB_OUTPUT
- name: Comment PR - Tests Started
if: github.event_name == 'pull_request'
uses: edumserrano/find-create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
body-includes: '<!-- PLAYWRIGHT_TEST_STATUS -->'
comment-author: 'github-actions[bot]'
edit-mode: append
body: |
<!-- PLAYWRIGHT_TEST_STATUS -->
---
<img alt='claude-loading-gif' src="https://github.com/user-attachments/assets/5ac382c7-e004-429b-8e35-7feb3e8f9c6f" width="14px" height="14px" style="vertical-align: middle; margin-left: 4px;" />
<bold>[${{ steps.current-time.outputs.time }} UTC] Preparing browser tests across multiple browsers...</bold>
---
*This comment will be updated when tests complete*
- name: Build ComfyUI_frontend
run: |
npm ci
@@ -46,6 +74,14 @@ jobs:
id: cache-key
run: echo "key=$(date +%s)" >> $GITHUB_OUTPUT
- name: Generate sanitized branch name
id: branch-info
run: |
# Get branch name and sanitize it for Cloudflare branch names
BRANCH_NAME="${{ github.head_ref || github.ref_name }}"
SANITIZED_BRANCH=$(echo "$BRANCH_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9-]/-/g' | sed 's/--*/-/g' | sed 's/^-\|-$//g')
echo "sanitized=${SANITIZED_BRANCH}" >> $GITHUB_OUTPUT
- name: Save cache
uses: actions/cache/save@5a3ec84eff668545956fd18022155c47e93e2684
with:
@@ -57,6 +93,10 @@ jobs:
playwright-tests:
needs: setup
runs-on: ubuntu-latest
permissions:
pull-requests: write
issues: write
contents: read
strategy:
fail-fast: false
matrix:
@@ -78,6 +118,32 @@ jobs:
with:
python-version: '3.10'
- name: Get current time
id: current-time
run: echo "time=$(date -u '${{ env.DATE_FORMAT }}')" >> $GITHUB_OUTPUT
- name: Set project name
id: project-name
run: |
if [ "${{ matrix.browser }}" = "chromium-0.5x" ]; then
echo "name=comfyui-playwright-chromium-0-5x" >> $GITHUB_OUTPUT
else
echo "name=comfyui-playwright-${{ matrix.browser }}" >> $GITHUB_OUTPUT
fi
echo "branch=${{ needs.setup.outputs.sanitized-branch }}" >> $GITHUB_OUTPUT
- name: Comment PR - Browser Test Started
if: github.event_name == 'pull_request'
uses: edumserrano/find-create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
body-includes: '<!-- PLAYWRIGHT_TEST_STATUS -->'
comment-author: 'github-actions[bot]'
edit-mode: append
body: |
<img alt='claude-loading-gif' src="https://github.com/user-attachments/assets/5ac382c7-e004-429b-8e35-7feb3e8f9c6f" width="14px" height="14px" style="vertical-align: middle; margin-left: 4px;" />
<bold>${{ matrix.browser }}</bold>: Running tests...
- name: Install requirements
run: |
python -m pip install --upgrade pip
@@ -97,12 +163,165 @@ jobs:
working-directory: ComfyUI_frontend
- name: Run Playwright tests (${{ matrix.browser }})
run: npx playwright test --project=${{ matrix.browser }}
id: playwright
run: npx playwright test --project=${{ matrix.browser }} --reporter=html
working-directory: ComfyUI_frontend
- uses: actions/upload-artifact@v4
if: always()
if: always() # note: use always() to allow results to be upload/report even tests failed.
with:
name: playwright-report-${{ matrix.browser }}
path: ComfyUI_frontend/playwright-report/
retention-days: 30
- name: Deploy to Cloudflare Pages (${{ matrix.browser }})
id: cloudflare-deploy
if: always()
uses: cloudflare/wrangler-action@v3
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
command: pages deploy ComfyUI_frontend/playwright-report --project-name=${{ steps.project-name.outputs.name }} --branch=${{ steps.project-name.outputs.branch }}
- name: Save deployment info for summary
if: always()
run: |
mkdir -p deployment-info
# Use step conclusion to determine test result
if [ "${{ steps.playwright.conclusion }}" = "success" ]; then
EXIT_CODE="0"
else
EXIT_CODE="1"
fi
DEPLOYMENT_URL="${{ steps.cloudflare-deploy.outputs.deployment-url || steps.cloudflare-deploy.outputs.url || format('https://{0}.{1}.pages.dev', steps.project-name.outputs.branch, steps.project-name.outputs.name) }}"
echo "${{ matrix.browser }}|${EXIT_CODE}|${DEPLOYMENT_URL}" > deployment-info/${{ matrix.browser }}.txt
- name: Upload deployment info
if: always()
uses: actions/upload-artifact@v4
with:
name: deployment-info-${{ matrix.browser }}
path: deployment-info/
retention-days: 1
- name: Get completion time
id: completion-time
if: always()
run: echo "time=$(date -u '${{ env.DATE_FORMAT }}')" >> $GITHUB_OUTPUT
- name: Comment PR - Browser Test Complete
if: always() && github.event_name == 'pull_request'
uses: edumserrano/find-create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
body-includes: '<!-- PLAYWRIGHT_TEST_STATUS -->'
comment-author: 'github-actions[bot]'
edit-mode: append
body: |
${{ steps.playwright.conclusion == 'success' && '✅' || '❌' }} **${{ matrix.browser }}**: ${{ steps.playwright.conclusion == 'success' && 'Tests passed!' || 'Tests failed!' }} [View Report](${{ steps.cloudflare-deploy.outputs.deployment-url || format('https://{0}.{1}.pages.dev', steps.project-name.outputs.branch, steps.project-name.outputs.name) }})
comment-summary:
needs: playwright-tests
runs-on: ubuntu-latest
if: always() && github.event_name == 'pull_request'
permissions:
pull-requests: write
steps:
- name: Download all deployment info
uses: actions/download-artifact@v4
with:
pattern: deployment-info-*
merge-multiple: true
path: deployment-info
- name: Get completion time
id: completion-time
run: echo "time=$(date -u '${{ env.DATE_FORMAT }}')" >> $GITHUB_OUTPUT
- name: Generate comment body
id: comment-body
run: |
echo "<!-- PLAYWRIGHT_TEST_STATUS -->" > comment.md
echo "## 🎭 Playwright Test Results" >> comment.md
echo "" >> comment.md
# Check if all tests passed
ALL_PASSED=true
for file in deployment-info/*.txt; do
if [ -f "$file" ]; then
browser=$(basename "$file" .txt)
info=$(cat "$file")
exit_code=$(echo "$info" | cut -d'|' -f2)
if [ "$exit_code" != "0" ]; then
ALL_PASSED=false
break
fi
fi
done
if [ "$ALL_PASSED" = "true" ]; then
echo "✅ **All tests passed across all browsers!**" >> comment.md
else
echo "❌ **Some tests failed!**" >> comment.md
fi
echo "" >> comment.md
echo "⏰ Completed at: ${{ steps.completion-time.outputs.time }} UTC" >> comment.md
echo "" >> comment.md
echo "### 📊 Test Reports by Browser" >> comment.md
for file in deployment-info/*.txt; do
if [ -f "$file" ]; then
browser=$(basename "$file" .txt)
info=$(cat "$file")
exit_code=$(echo "$info" | cut -d'|' -f2)
url=$(echo "$info" | cut -d'|' -f3)
if [ "$exit_code" = "0" ]; then
status="✅"
else
status="❌"
fi
echo "- $status **$browser**: [View Report]($url)" >> comment.md
fi
done
echo "" >> comment.md
echo "---" >> comment.md
if [ "$ALL_PASSED" = "true" ]; then
echo "🎉 Your tests are passing across all browsers!" >> comment.md
else
echo "⚠️ Please check the test reports for details on failures." >> comment.md
fi
- name: Comment PR - Tests Complete
uses: edumserrano/find-create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
body-includes: '<!-- PLAYWRIGHT_TEST_STATUS -->'
comment-author: 'github-actions[bot]'
edit-mode: replace
body-path: comment.md
- name: Check test results and fail if needed
run: |
# Check if all tests passed and fail the job if not
ALL_PASSED=true
for file in deployment-info/*.txt; do
if [ -f "$file" ]; then
info=$(cat "$file")
exit_code=$(echo "$info" | cut -d'|' -f2)
if [ "$exit_code" != "0" ]; then
ALL_PASSED=false
break
fi
fi
done
if [ "$ALL_PASSED" = "false" ]; then
echo "❌ Tests failed in one or more browsers. Failing the CI job."
exit 1
else
echo "✅ All tests passed across all browsers!"
fi