Files
ComfyUI_frontend/.github/workflows/ci-tests-e2e-coverage.yaml
Christian Byrne 8abfa678a3 fix: harden e2e coverage workflow and fix GH Pages deploy (#11381)
## Problem

The GH Pages coverage deploy has been failing since #11291 merged —
every `CI: E2E Coverage` workflow run errors out and
https://comfy-org.github.io/ComfyUI_frontend/ returns 404.

Additionally, two correctness/security issues were identified in the
workflow (filed as #11374 and #11375).

## Changes

1. **`--ignore-errors source` on genhtml** — merged LCOV data includes
paths like `localhost-8188/assets/main-BRkC1B8m.js` from Playwright V8
coverage instrumented runtime bundles that don't exist as source files
in CI, causing genhtml to error out
2. **Pin checkout to `workflow_run.head_sha`** — in `workflow_run`
context, the default checkout ref points to the default branch, not the
commit that triggered the upstream run; genhtml could annotate against
wrong source files (#11375)
3. **Gate deploy on `event == 'push'`** — a fork branch named `main`
could satisfy the branch check and overwrite production coverage; adding
the event guard prevents this (#11375)
4. **Include workflow run link in placeholder HTML** — when no coverage
data is available, the placeholder page now links back to the workflow
run for debugging (#11374)

## Fixes

- Fixes the GH Pages 404 caused by #11291
- Fixes #11374
- Fixes #11375
2026-05-13 23:06:36 +00:00

159 lines
5.9 KiB
YAML

name: 'CI: E2E Coverage'
on:
workflow_run:
workflows: ['CI: Tests E2E']
types:
- completed
concurrency:
group: e2e-coverage-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: true
permissions:
contents: read
jobs:
merge:
if: >
github.repository == 'Comfy-Org/ComfyUI_frontend' &&
github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
has-coverage: ${{ steps.coverage-shards.outputs.has-coverage }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup frontend
uses: ./.github/actions/setup-frontend
- name: Download all shard coverage data
uses: dawidd6/action-download-artifact@0bd50d53a6d7fb5cb921e607957e9cc12b4ce392 # v12
with:
run_id: ${{ github.event.workflow_run.id }}
name: e2e-coverage-shard-.*
name_is_regexp: true
path: temp/coverage-shards
if_no_artifact_found: warn
- name: Detect shard coverage data
id: coverage-shards
run: |
if [ -d temp/coverage-shards ] && find temp/coverage-shards -name 'coverage.lcov' -type f | grep -q .; then
echo "has-coverage=true" >> "$GITHUB_OUTPUT"
else
echo "has-coverage=false" >> "$GITHUB_OUTPUT"
echo "No E2E coverage shard artifacts found; treating this run as skipped." >> "$GITHUB_STEP_SUMMARY"
fi
- name: Install lcov
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: sudo apt-get install -y -qq lcov
- name: Merge shard coverage into single LCOV
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
mkdir -p coverage/playwright
LCOV_FILES=$(find temp/coverage-shards -name 'coverage.lcov' -type f)
ADD_ARGS=""
for f in $LCOV_FILES; do ADD_ARGS="$ADD_ARGS -a $f"; done
lcov $ADD_ARGS -o coverage/playwright/coverage.lcov
wc -l coverage/playwright/coverage.lcov
- name: Validate merged coverage
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
MERGED_SF=$(grep -c '^SF:' coverage/playwright/coverage.lcov || echo 0)
MERGED_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
MERGED_LF=$(awk -F: '/^LF:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
echo "### Merged coverage" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_SF** source files" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_LH / $MERGED_LF** lines hit" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Shard | Files | Lines Hit |" >> "$GITHUB_STEP_SUMMARY"
echo "|-------|-------|-----------|" >> "$GITHUB_STEP_SUMMARY"
for f in $(find temp/coverage-shards -name 'coverage.lcov' -type f | sort); do
SHARD=$(basename "$(dirname "$f")")
SHARD_SF=$(grep -c '^SF:' "$f" || echo 0)
SHARD_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' "$f")
echo "| $SHARD | $SHARD_SF | $SHARD_LH |" >> "$GITHUB_STEP_SUMMARY"
if [ "$MERGED_LH" -lt "$SHARD_LH" ]; then
echo "::error::Merged LH ($MERGED_LH) < shard LH ($SHARD_LH) in $SHARD — possible data loss"
fi
done
- name: Upload merged coverage data
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: actions/upload-artifact@v6
with:
name: e2e-coverage
path: coverage/playwright/
retention-days: 30
if-no-files-found: warn
- name: Upload E2E coverage to Codecov
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5.5.3
with:
files: coverage/playwright/coverage.lcov
flags: e2e
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
- name: Generate HTML coverage report
if: steps.coverage-shards.outputs.has-coverage == 'true'
run: |
if [ ! -s coverage/playwright/coverage.lcov ]; then
echo "No coverage data; generating placeholder report."
mkdir -p coverage/html
WORKFLOW_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}"
echo "<html><body><h1>No E2E coverage data available for this run.</h1><p><a href=\"${WORKFLOW_URL}\">View workflow run</a></p></body></html>" > coverage/html/index.html
exit 0
fi
genhtml coverage/playwright/coverage.lcov \
-o coverage/html \
--title "ComfyUI E2E Coverage" \
--no-function-coverage \
--precision 1 \
--ignore-errors source
- name: Upload HTML report artifact
if: steps.coverage-shards.outputs.has-coverage == 'true'
uses: actions/upload-artifact@v6
with:
name: e2e-coverage-html
path: coverage/html/
retention-days: 30
deploy:
needs: merge
if: >
github.event.workflow_run.head_branch == 'main' &&
needs.merge.outputs.has-coverage == 'true' &&
github.event.workflow_run.event == 'push'
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Download HTML report
uses: actions/download-artifact@v7
with:
name: e2e-coverage-html
path: coverage/html
- name: Upload to GitHub Pages
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3.0.1
with:
path: coverage/html
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5