mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-02-20 06:44:32 +00:00
87f5480462
## Problem
Our CI tests were experiencing non-reproducible results where:
- Tests would pass on a PR initially
- The same PR would fail later when main HEAD changed
- Screenshot comparisons showed excessive differences between expected
vs actual
- Blake identified: *"tests are not reproducible inside a branch - they
change every time main HEAD changes"*
## Root Cause
The issue was caused by **explicit `repository` parameters** in our
`actions/checkout` steps:
```yaml
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v5
with:
repository: 'Comfy-Org/ComfyUI_frontend' # ← This was the problem!
path: 'ComfyUI_frontend'
```
According to GitHub Actions documentation:
> **When checking out the repository that triggered a workflow, `ref`
defaults to the reference or SHA for that event. Otherwise, uses the
default branch.**
When you specify an explicit `repository` parameter (even if it's the
same repo), GitHub Actions treats it as "otherwise" and defaults to the
**main branch** instead of using the **PR context**.
## The Fix
Remove the explicit `repository` parameter when checking out the same
repository:
```yaml
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v5
with:
path: 'ComfyUI_frontend' # No repository parameter = uses PR context
```
## Changes Made
- ✅ Removed `repository: 'Comfy-Org/ComfyUI_frontend'` from setup job
checkout
- ✅ Removed `repository: 'Comfy-Org/ComfyUI_frontend'` from
merge-reports job checkout
- ✅ Updated setup-frontend action to use `actions/checkout@v5` for
consistency
- ✅ Simplified workflow by removing unnecessary `ref` and `fetch-depth`
parameters
## How This Fixes the Problem
**Before:**
- Setup job checked out main branch (due to explicit repository)
- Tests ran PR code against main branch snapshots
- Results varied based on what was in main at the time
**After:**
- Setup job checks out PR merge commit (natural PR context)
- Tests run PR code against PR snapshots
- Results are consistent and reproducible
## Why It Worked Before (Sometimes)
The explicit `repository` parameter has been there for a long time, but
the issue became more apparent recently due to:
1. GitHub Actions behavior changes over time
2. Increased frequency of main branch updates
3. More sensitive screenshot comparison tests
4. Complex cache/restore workflow where timing mattered
The fix ensures deterministic behavior regardless of GitHub's internal
changes.
## Testing
This change makes the CI behavior explicit and predictable:
- ✅ PR tests will always use PR context
- ✅ Push tests will always use pushed commit
- ✅ No dependency on GitHub's default behavior interpretation
- ✅ Simplified workflow with fewer moving parts
Resolves the issues described in `.github/workflows/problem.log`.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-5950-Fix-CI-Remove-explicit-repository-parameter-causing-non-reproducible-test-results-2846d73d36508159a848c4a2e14a0fb1)
by [Unito](https://www.unito.io)
Co-authored-by: Alexander Brown <drjkl@comfy.org>
66 lines
1.8 KiB
YAML
66 lines
1.8 KiB
YAML
name: Setup Frontend
|
|
description: 'Setup ComfyUI frontend development environment'
|
|
inputs:
|
|
extra_server_params:
|
|
description: 'Additional parameters to pass to ComfyUI server'
|
|
required: false
|
|
default: ''
|
|
runs:
|
|
using: 'composite'
|
|
steps:
|
|
- name: Checkout ComfyUI
|
|
uses: actions/checkout@v5
|
|
with:
|
|
repository: 'comfyanonymous/ComfyUI'
|
|
path: 'ComfyUI'
|
|
|
|
- name: Checkout ComfyUI_frontend
|
|
uses: actions/checkout@v5
|
|
with:
|
|
path: 'ComfyUI_frontend'
|
|
|
|
- name: Copy ComfyUI_devtools from frontend repo
|
|
shell: bash
|
|
run: |
|
|
mkdir -p ComfyUI/custom_nodes/ComfyUI_devtools
|
|
cp -r ComfyUI_frontend/tools/devtools/* ComfyUI/custom_nodes/ComfyUI_devtools/
|
|
|
|
- name: Install pnpm
|
|
uses: pnpm/action-setup@v4
|
|
with:
|
|
version: 10
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 'lts/*'
|
|
cache: 'pnpm'
|
|
cache-dependency-path: 'ComfyUI_frontend/pnpm-lock.yaml'
|
|
|
|
- name: Setup Python
|
|
uses: actions/setup-python@v4
|
|
with:
|
|
python-version: '3.10'
|
|
|
|
- name: Install Python requirements
|
|
shell: bash
|
|
working-directory: ComfyUI
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
|
|
pip install -r requirements.txt
|
|
pip install wait-for-it
|
|
|
|
- name: Build & Install ComfyUI_frontend
|
|
shell: bash
|
|
working-directory: ComfyUI_frontend
|
|
run: |
|
|
pnpm install --frozen-lockfile
|
|
pnpm build
|
|
|
|
- name: Start ComfyUI server
|
|
shell: bash
|
|
working-directory: ComfyUI
|
|
run: |
|
|
python main.py --cpu --multi-user --front-end-root ../ComfyUI_frontend/dist ${{ inputs.extra_server_params }} &
|
|
wait-for-it --service 127.0.0.1:8188 -t 600 |