## Summary
Use a pre-built container image with all dependencies for Playwright E2E
tests, eliminating ~130s setup time per shard.
## Changes
- Use `ghcr.io/comfy-org/comfyui-ci-container:0.0.8` container for test
jobs
- Container includes: Playwright browsers, Node.js, pnpm, Python,
ComfyUI backend (v0.5.1), all Python deps
- Simplified setup: just copy devtools and start server (no cloning, no
pip install, no browser setup)
- Add `version-bump*` to `branches-ignore` to skip E2E tests for version
bump PRs
- Replace cache with artifacts (cache doesn't work inside containers)
## Benefits
- ~130s faster per shard (no ComfyUI clone, no pip install, no browser
download)
- Consistent environment across all test jobs
- Simpler workflow configuration
## Container Image
Repository: https://github.com/comfy-org/comfyui-ci-container
Image: `ghcr.io/comfy-org/comfyui-ci-container:0.0.8`
## Test plan
- [x] Verify CI workflow runs with container
- [x] Verify Playwright tests pass
- [x] Verify snapshot updates work correctly
---------
Co-authored-by: github-actions <github-actions@github.com>
This pull request updates the
`.github/workflows/pr-update-playwright-expectations.yaml` workflow to
improve how changed Playwright snapshot files are detected and handled,
ensuring that both tracked and untracked (new) files are included
throughout the process. The changes also add robustness to file
operations and improve the accuracy of change summaries.
**Improvements to snapshot detection and staging:**
* The workflow now detects both tracked and untracked (new) snapshot
files in `browser_tests/` when preparing changed files for staging,
ensuring that new snapshots are not missed.
* When copying changed files to the staging directory, the script now
skips files that no longer exist (e.g., deleted files), preventing
errors and unnecessary operations.
**Enhancements to change summary and commit logic:**
* The summary of changes now includes both tracked and untracked files
in `browser_tests/`, and the output is expanded to show up to 50 files
for better visibility.
* The logic for determining whether there are changes to commit now
checks for both tracked and untracked changes, ensuring commits are only
made when necessary.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-7423-Fix-snapshot-updates-commit-stage-2c76d73d36508195914ec92f37937e67)
by [Unito](https://www.unito.io)
This pull request improves the snapshot staging process in the
Playwright expectations update workflow. The main focus is to ensure
both modified and newly added snapshot files are correctly detected and
handled, and to avoid errors when files have been deleted.
**Snapshot file detection and handling improvements:**
* The workflow now detects both modified and untracked (new) snapshot
files by combining output from `git diff` and `git ls-files --others`,
ensuring all relevant snapshot changes are staged.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-7422-fix-enhance-snapshot-update-process-to-include-untracked-files-2c76d73d365081cc8023d9ed29b8781f)
by [Unito](https://www.unito.io)
This pull request updates the workflow for managing Playwright
expectation snapshots in
`.github/workflows/pr-update-playwright-expectations.yaml`. The main
focus is to improve how changed snapshot files are handled, ensuring
directory structures are correct and only necessary files are processed
and committed. The most important changes are grouped below:
**Snapshot file handling improvements:**
* When copying changed snapshot files, the workflow now strips the
`browser_tests/` prefix to avoid double nesting in the staging
directory. This ensures the directory structure remains correct when
files are merged back.
* During the merging step, the script clarifies that files are already
in the correct structure (without the `browser_tests/` prefix), so they
can be copied directly into `browser_tests/`.
**Workflow and commit logic enhancements:**
* The snapshot artifact download step is renamed for clarity, indicating
that only changed snapshot files are downloaded from shards.
* The commit step is improved to check for actual changes in
`browser_tests/` before attempting to commit and push. It sets an output
variable (`has-changes`) to control subsequent steps, adds more
informative logging, and only pushes when there are changes.
* The "Add Done Reaction" step is now conditional on both the event type
and whether there were changes to commit, preventing unnecessary
reactions.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6252-Hotfix-CI-update-test-expectations-2966d73d36508131968ee5a7f04ff787)
by [Unito](https://www.unito.io)
---------
Co-authored-by: github-actions <github-actions@github.com>
This pull request updates the Playwright snapshot update workflow to
improve efficiency and clarity. The workflow now only uploads and merges
changed snapshot files from each shard, reducing unnecessary artifact
size and processing. It also adds better logging and summaries for
easier debugging and review.
**Snapshot handling improvements:**
* Only changed snapshot files are identified, staged, and uploaded as
artifacts per shard, instead of uploading all snapshot files. This
reduces artifact size and upload time.
* During the merge step, only the changed files from each shard are
merged back into the `browser_tests` directory, preserving directory
structure and avoiding redundant operations.
**Logging and debugging enhancements:**
* Added steps to list downloaded snapshot files and summarize the
changes after merging, making it easier to see what was updated and
debug any issues.
Sample run is here
https://github.com/Myestery/ComfyUI_frontend/actions/runs/18768857441
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6250-Patch-Update-expectations-CI-2966d73d365081b790a0fad66649a10b)
by [Unito](https://www.unito.io)
This pull request makes a small change to the
`.github/workflows/pr-update-playwright-expectations.yaml` file,
removing the unconditional `if: always()` condition from the
`merge-and-commit` job. This means the job will now only run if its
dependencies succeed, rather than always running regardless of previous
job outcomes.
This pull request significantly refactors the Playwright expectations
update workflow to improve reliability, efficiency, and maintainability.
The workflow is now split into three coordinated jobs—setup, sharded
snapshot updates, and merge/commit—enabling parallel test execution and
artifact management. Key improvements include sharding Playwright
snapshot updates, robust caching and artifact handling, and more
reliable PR context handling.
**Workflow Restructuring and Sharding:**
* The workflow is split into three jobs: `setup` (prepares environment
and caches it), `update-snapshots-sharded` (runs Playwright snapshot
updates in four parallel shards), and `merge-and-commit` (merges results
and commits updates). This enables faster, more reliable snapshot
updates.
[[1]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917L15-R15)
[[2]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917R27-R175)
**Caching and Artifact Management:**
* The setup job builds and caches the entire workspace, which is then
restored by each shard for consistent environments. Each shard uploads
its updated snapshots and test reports as artifacts, which are later
downloaded and merged in the final job.
**Improved PR Context Handling:**
* PR number, branch, and comment IDs are now reliably extracted and
passed between jobs using outputs, ensuring correct association with the
PR throughout the workflow (e.g., for commenting, reactions, and pushing
updates).
[[1]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917R27-R175)
[[2]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917L92-R199)
**Job and Step Renaming/Cleanup:**
* The main job is renamed from `test` to `setup`, and redundant or
unnecessary steps (such as the old branch SHA extraction) are removed
for clarity and maintainability.
[[1]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917L15-R15)
[[2]](diffhunk://#diff-0289f4b5962314fa2d58937651c3d2a0f2c6f76e26c95d6a04d43c18b3449917R27-R175)
**Comment and Label Automation Improvements:**
* Automated GitHub comment reactions and label removals now use the
correct PR context, ensuring that feedback and status updates are
reliably posted to the right place.
┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6100-Shard-Update-Test-Expectations-PR-28f6d73d36508109bcd8d382c942d44d)
by [Unito](https://www.unito.io)
---------
Co-authored-by: sno <snomiao@gmail.com>
