- Renamed parameter from skip_build to include_build_step
- Changed default from 'false' to 'false' (building now opt-in)
- Updated all workflow usages to explicitly set include_build_step: 'true' where building is needed
- Removed explicit parameter for workflows that don't need building
As suggested by @DrJKL in PR review
Improves clarity by using positive naming convention for the parameter.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Move cache restore to setup-frontend action BEFORE build operations
- Remove duplicate cache step from setup-playwright action
- Use cache/restore instead of cache to avoid auto-save behavior
- Rename cache key from 'playwright-setup-cache' to 'tool-cache' for clarity
- Include source file hashes in cache key for proper invalidation
Benefits:
- Cache is now restored before tools run, allowing them to use cached data
- Eliminates duplicate caching of ./.cache directory
- Cache properly invalidates when source files or configs change
- Follows GitHub Actions best practice of restore before, save after pattern
- The workspace cache in tests-ci.yaml handles saving the complete state
Note: The .cache directory contains outputs from ESLint, Prettier, Stylelint,
Knip, and TypeScript incremental builds. These should be restored before any
build/lint operations run.
This refactoring improves the CI/CD workflow structure by:
- Split the monolithic setup-frontend action into two focused actions:
- setup-frontend: Now only handles frontend dependency installation and building
- setup-comfyui-server: New action for ComfyUI server setup and launch
- Simplified workflow structure with better separation of concerns:
- Frontend and server setup are now independent and reusable
- Each action has clearer responsibilities and inputs
- Removed duplicate setup code across workflows
- Improved tests-ci.yaml workflow:
- Uses cache/save and cache/restore for the entire workspace
- Test jobs now restore cached build instead of rebuilding
- Reduced redundant setup steps in each test shard
- Better parallelization with faster test execution
- Updated all locale update workflows to use new action structure
- Made setup-playwright a standalone reusable action
Benefits:
- Faster CI runs by reducing redundant builds
- More maintainable with DRY principle
- Easier to debug individual components
- Better action reusability across workflows
## Problem
Our CI tests were experiencing non-reproducible results where:
- Tests would pass on a PR initially
- The same PR would fail later when main HEAD changed
- Screenshot comparisons showed excessive differences between expected
vs actual
- Blake identified: *"tests are not reproducible inside a branch - they
change every time main HEAD changes"*
## Root Cause
The issue was caused by **explicit `repository` parameters** in our
`actions/checkout` steps:
```yaml
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v5
with:
repository: 'Comfy-Org/ComfyUI_frontend' # ← This was the problem!
path: 'ComfyUI_frontend'
```
According to GitHub Actions documentation:
> **When checking out the repository that triggered a workflow, `ref`
defaults to the reference or SHA for that event. Otherwise, uses the
default branch.**
When you specify an explicit `repository` parameter (even if it's the
same repo), GitHub Actions treats it as "otherwise" and defaults to the
**main branch** instead of using the **PR context**.
## The Fix
Remove the explicit `repository` parameter when checking out the same
repository:
```yaml
- name: Checkout ComfyUI_frontend
uses: actions/checkout@v5
with:
path: 'ComfyUI_frontend' # No repository parameter = uses PR context
```
## Changes Made
- ✅ Removed `repository: 'Comfy-Org/ComfyUI_frontend'` from setup job
checkout
- ✅ Removed `repository: 'Comfy-Org/ComfyUI_frontend'` from
merge-reports job checkout
- ✅ Updated setup-frontend action to use `actions/checkout@v5` for
consistency
- ✅ Simplified workflow by removing unnecessary `ref` and `fetch-depth`
parameters
## How This Fixes the Problem
**Before:**
- Setup job checked out main branch (due to explicit repository)
- Tests ran PR code against main branch snapshots
- Results varied based on what was in main at the time
**After:**
- Setup job checks out PR merge commit (natural PR context)
- Tests run PR code against PR snapshots
- Results are consistent and reproducible
## Why It Worked Before (Sometimes)
The explicit `repository` parameter has been there for a long time, but
the issue became more apparent recently due to:
1. GitHub Actions behavior changes over time
2. Increased frequency of main branch updates
3. More sensitive screenshot comparison tests
4. Complex cache/restore workflow where timing mattered
The fix ensures deterministic behavior regardless of GitHub's internal
changes.
## Testing
This change makes the CI behavior explicit and predictable:
- ✅ PR tests will always use PR context
- ✅ Push tests will always use pushed commit
- ✅ No dependency on GitHub's default behavior interpretation
- ✅ Simplified workflow with fewer moving parts
Resolves the issues described in `.github/workflows/problem.log`.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-5950-Fix-CI-Remove-explicit-repository-parameter-causing-non-reproducible-test-results-2846d73d36508159a848c4a2e14a0fb1)
by [Unito](https://www.unito.io)
Co-authored-by: Alexander Brown <drjkl@comfy.org>
## Summary
- Removed the optional `ref: master` parameter from the ComfyUI checkout
step in the setup-frontend action
- The ref parameter defaults to the repository's default branch when
omitted
## Details
The `ref: master` specification in
`.github/actions/setup-frontend/action.yml` is unnecessary since GitHub
Actions will automatically use the repository's default branch when the
ref parameter is not provided.
This simplifies the configuration and makes it more maintainable, as the
action will automatically follow any future changes to the default
branch name.
## Test plan
- [ ] Verify that GitHub Actions workflows using this composite action
still work correctly
- [ ] Confirm ComfyUI is checked out properly in CI/CD pipelines
Generated with [Claude Code](https://claude.ai/code)
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-5858-Remove-optional-ref-master-from-setup-frontend-action-27e6d73d365081aeb632f2d0e76f267d)
by [Unito](https://www.unito.io)
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
- Adds a local composite action at `.github/actions/setup-frontend` to
replace the external `Comfy-Org/ComfyUI_frontend_setup_action`
- Follows the same pattern as PR #5754 (sno-playwright-composite-action)
for consistency
- Updates workflows to use the new local composite action
## Motivation
Similar to the Playwright composite action, this change:
- Reduces external dependencies on separate action repositories
- Provides better control over versioning and updates
- Maintains consistency with other composite actions in the repository
- Simplifies maintenance by keeping all CI/CD logic in one place
## Changes
### New composite action: `.github/actions/setup-frontend/action.yml`
Direct mirror of the external action with the same 2 inputs:
- `extra_server_params`: Additional parameters to pass to ComfyUI server
- `devtools_ref`: Reference to use for ComfyUI_devtools
The action:
1. Checks out ComfyUI, ComfyUI_frontend, and ComfyUI_devtools
2. Sets up pnpm, Node.js (LTS), and Python (3.10)
3. Installs all dependencies (Python packages, npm packages)
4. Builds the frontend
5. Starts the ComfyUI server with the built frontend
### Updated workflows:
- `.github/workflows/i18n.yaml`
- `.github/workflows/i18n-node-defs.yaml`
- `.github/workflows/test-browser-exp.yaml`
All workflows now use the local composite action instead of
`Comfy-Org/ComfyUI_frontend_setup_action@v3`
## Test plan
- [ ] Verify all updated workflows pass CI tests
- [ ] Confirm the composite action works in all scenarios
- [ ] Check that build and server startup work as expected
## Related PRs
- #5754 - Similar approach for Playwright composite action
🤖 Generated with [Claude Code](https://claude.ai/code)
