mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-02-03 14:54:37 +00:00
## Overview Removes **all `actions/cache` steps** from GitHub Actions workflows after empirical testing showed that they actually **slow down CI/CD by 11%** rather than speeding it up. ## Context As discussed in #5988, the codebase has evolved with components moving into the `/packages` directory structure. The review comment suggested removing the entire `actions/cache` step rather than just the `.cache` path to properly evaluate performance impact. ## Performance Benchmark Results Empirical testing on this PR (commits38695ae0bvsab16635c5) revealed that **removing cache steps improves CI performance across all workflows**: | Workflow | WITHOUT Cache | WITH Cache | Improvement | |----------|---------------|------------|-------------| | **CI: Lint Format** | 208s (3m 28s) | 226s (3m 46s) | **-18s (-8.7%)** ⚡ | | **CI: Tests Unit** | 160s (2m 40s) | 177s (2m 57s) | **-17s (-10.6%)** ⚡ | | **CI: Tests Storybook** | 65s (1m 5s) | 78s (1m 18s) | **-13s (-20.0%)** ⚡ | | **Total Pipeline** | **433s (7m 13s)** | **481s (8m 1s)** | **-48s (-11.1%)** ⚡ | ### Why is caching slower? 1. **Cache overhead exceeds benefits**: Time spent saving/restoring cache > time saved from cached content 2. **Complex cache key computation**: Hash calculations for file patterns add processing time 3. **Network I/O cost**: Each cache step adds network round-trips 4. **Tools already optimize incrementally**: ESLint, Vitest, Prettier handle their own incremental checks efficiently ## Changes Removed the entire `actions/cache` step from 8 workflow files: - `ci-lint-format.yaml` - Removed tool outputs cache (.eslintcache, .prettierCache, .knip-cache, tsconfig.tsbuildinfo) - `ci-tests-storybook.yaml` - Removed storybook-static and tsconfig.tsbuildinfo cache (both jobs) - `ci-tests-unit.yaml` - Removed coverage and .vitest-cache - `api-update-electron-api-types.yaml` - Removed tsconfig.tsbuildinfo cache - `api-update-manager-api-types.yaml` - Removed tool cache and ComfyUI-Manager repo cache - `api-update-registry-api-types.yaml` - Removed tool cache and comfy-api repo cache - `release-draft-create.yaml` - Removed tsconfig.tsbuildinfo cache - `release-pypi-dev.yaml` - Removed dist and tsconfig.tsbuildinfo cache **What remains cached:** - ✅ pnpm packages via `cache: 'pnpm'` in setup-node actions (the most valuable cache) - ✅ Tool-specific incremental caches generated fresh each run - ✅ Docker layer caching (where applicable) ## Testing - ✅ Empirical performance testing completed (see benchmark results above) - ✅ All cache steps removed successfully - ✅ No structural changes to workflow logic - ✅ pnpm package caching remains active ## Conclusion The benchmark data clearly shows that removing `actions/cache` steps results in **faster, simpler CI workflows**. The overhead of cache management exceeds any benefit, especially with pnpm package caching already handling the most time-consuming dependency installations. **Recommendation: ✅ Proceed with this change** ## Test Methodology 1. **WITHOUT cache** (commit [38695ae0b](https://github.com/Comfy-Org/ComfyUI_frontend/commit/38695ae0b)): Removed all `actions/cache` steps → [Workflow run](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18654024806) 2. **WITH cache** (commit [ab16635c5](https://github.com/Comfy-Org/ComfyUI_frontend/commit/ab16635c5)): Temporarily restored all `actions/cache` steps → [Workflow run](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18654143363) 3. **Final state** (commit [3ce876f87](https://github.com/Comfy-Org/ComfyUI_frontend/commit/3ce876f87)): Restored no-cache version (current) Fixes #5988 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: snomiao <7323030+snomiao@users.noreply.github.com> Co-authored-by: snomiao <snomiao@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
205 lines
7.2 KiB
YAML
205 lines
7.2 KiB
YAML
name: "CI: Tests Storybook"
|
|
description: "Builds Storybook and runs visual regression testing via Chromatic, deploys previews to Cloudflare Pages"
|
|
|
|
on:
|
|
workflow_dispatch: # Allow manual triggering
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
jobs:
|
|
# Post starting comment for non-forked PRs
|
|
comment-on-pr-start:
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
|
|
permissions:
|
|
pull-requests: write
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Post starting comment
|
|
env:
|
|
GITHUB_TOKEN: ${{ github.token }}
|
|
run: |
|
|
chmod +x scripts/cicd/pr-storybook-deploy-and-comment.sh
|
|
./scripts/cicd/pr-storybook-deploy-and-comment.sh \
|
|
"${{ github.event.pull_request.number }}" \
|
|
"${{ github.head_ref }}" \
|
|
"starting" \
|
|
"$(date -u '+%m/%d/%Y, %I:%M:%S %p')"
|
|
|
|
# Build Storybook for all PRs (free Cloudflare deployment)
|
|
storybook-build:
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'pull_request'
|
|
outputs:
|
|
conclusion: ${{ steps.job-status.outputs.conclusion }}
|
|
workflow-url: ${{ steps.workflow-url.outputs.url }}
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Install pnpm
|
|
uses: pnpm/action-setup@v4
|
|
with:
|
|
version: 10
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
cache: 'pnpm'
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Build Storybook
|
|
run: pnpm build-storybook
|
|
|
|
- name: Set job status
|
|
id: job-status
|
|
if: always()
|
|
run: |
|
|
echo "conclusion=${{ job.status }}" >> $GITHUB_OUTPUT
|
|
|
|
- name: Get workflow URL
|
|
id: workflow-url
|
|
if: always()
|
|
run: |
|
|
echo "url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" >> $GITHUB_OUTPUT
|
|
|
|
- name: Upload Storybook build
|
|
if: success() && github.event.pull_request.head.repo.fork == false
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: storybook-static
|
|
path: storybook-static/
|
|
retention-days: 7
|
|
|
|
# Chromatic deployment only for version-bump-* branches or manual triggers
|
|
chromatic-deployment:
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && startsWith(github.head_ref, 'version-bump-'))
|
|
outputs:
|
|
conclusion: ${{ steps.job-status.outputs.conclusion }}
|
|
workflow-url: ${{ steps.workflow-url.outputs.url }}
|
|
chromatic-build-url: ${{ steps.chromatic.outputs.buildUrl }}
|
|
chromatic-storybook-url: ${{ steps.chromatic.outputs.storybookUrl }}
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@v5
|
|
with:
|
|
fetch-depth: 0 # Required for Chromatic baseline
|
|
|
|
- name: Install pnpm
|
|
uses: pnpm/action-setup@v4
|
|
with:
|
|
version: 10
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
cache: 'pnpm'
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Build Storybook and run Chromatic
|
|
id: chromatic
|
|
uses: chromaui/action@latest
|
|
with:
|
|
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
|
|
buildScriptName: build-storybook
|
|
autoAcceptChanges: 'main' # Auto-accept changes on main branch
|
|
exitOnceUploaded: true # Don't wait for UI tests to complete
|
|
onlyChanged: true # Only capture changed stories
|
|
|
|
- name: Set job status
|
|
id: job-status
|
|
if: always()
|
|
run: |
|
|
echo "conclusion=${{ job.status }}" >> $GITHUB_OUTPUT
|
|
|
|
- name: Get workflow URL
|
|
id: workflow-url
|
|
if: always()
|
|
run: |
|
|
echo "url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" >> $GITHUB_OUTPUT
|
|
|
|
# Deploy and comment for non-forked PRs only
|
|
deploy-and-comment:
|
|
needs: [storybook-build]
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false && always()
|
|
permissions:
|
|
pull-requests: write
|
|
contents: read
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Download Storybook build
|
|
if: needs.storybook-build.outputs.conclusion == 'success'
|
|
uses: actions/download-artifact@v4
|
|
with:
|
|
name: storybook-static
|
|
path: storybook-static
|
|
|
|
- name: Make deployment script executable
|
|
run: chmod +x scripts/cicd/pr-storybook-deploy-and-comment.sh
|
|
|
|
- name: Deploy Storybook and comment on PR
|
|
env:
|
|
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
|
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
|
GITHUB_TOKEN: ${{ github.token }}
|
|
WORKFLOW_CONCLUSION: ${{ needs.storybook-build.outputs.conclusion }}
|
|
WORKFLOW_URL: ${{ needs.storybook-build.outputs.workflow-url }}
|
|
run: |
|
|
./scripts/cicd/pr-storybook-deploy-and-comment.sh \
|
|
"${{ github.event.pull_request.number }}" \
|
|
"${{ github.head_ref }}" \
|
|
"completed"
|
|
|
|
# Update comment with Chromatic URLs for version-bump branches
|
|
update-comment-with-chromatic:
|
|
needs: [chromatic-deployment, deploy-and-comment]
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false && startsWith(github.head_ref, 'version-bump-') && needs.chromatic-deployment.outputs.chromatic-build-url != ''
|
|
permissions:
|
|
pull-requests: write
|
|
steps:
|
|
- name: Update comment with Chromatic URLs
|
|
uses: actions/github-script@v7
|
|
with:
|
|
script: |
|
|
const buildUrl = '${{ needs.chromatic-deployment.outputs.chromatic-build-url }}';
|
|
const storybookUrl = '${{ needs.chromatic-deployment.outputs.chromatic-storybook-url }}';
|
|
|
|
// Find the existing Storybook comment
|
|
const { data: comments } = await github.rest.issues.listComments({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
issue_number: ${{ github.event.pull_request.number }}
|
|
});
|
|
|
|
const storybookComment = comments.find(comment =>
|
|
comment.body.includes('<!-- STORYBOOK_BUILD_STATUS -->')
|
|
);
|
|
|
|
if (storybookComment && buildUrl && storybookUrl) {
|
|
// Append Chromatic info to existing comment
|
|
const updatedBody = storybookComment.body.replace(
|
|
/---\n(.*)$/s,
|
|
`---\n### 🎨 Chromatic Visual Tests\n- 📊 [View Chromatic Build](${buildUrl})\n- 📚 [View Chromatic Storybook](${storybookUrl})\n\n$1`
|
|
);
|
|
|
|
await github.rest.issues.updateComment({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
comment_id: storybookComment.id,
|
|
body: updatedBody
|
|
});
|
|
}
|