[bugfix] Fix E2E test report generation for non-chromium browsers (#7193)

## Summary

Fixes an issue where Playwright HTML reports were not being generated
for `chromium-2x`, `chromium-0.5x`, and `mobile-chrome` test runs,
causing 404 errors when accessing report links in PR comments.

## Problem

Only the first report link (chromium) had contents - the other three
browsers returned 404 errors. Investigation revealed that artifacts for
non-chromium browsers were only uploading 1 file (report.json) instead
of the full HTML report with ~32 files.

The root cause was that these browsers run very few tests (1-6 tests
each), and when using `--reporter=html --reporter=json` together,
Playwright would only generate the JSON report without the HTML assets.

## Solution

Changed the workflow for non-chromium browsers to use the same approach
as the sharded chromium tests:
1. Run tests with `--reporter=blob`
2. Generate HTML and JSON reports separately using `merge-reports`

This ensures both HTML and JSON reports are always generated with
complete assets, regardless of test count.

## Testing

The fix can be verified by:
1. Checking that this PR's workflow run uploads similar file counts for
all browsers
2. Confirming all 4 report links are accessible and show proper HTML
content

Related to #7186

🤖 Generated with [Claude Code](https://claude.com/claude-code)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-7193-bugfix-Fix-E2E-test-report-generation-for-non-chromium-browsers-2c06d73d365081ba8ea3ed0d3f5d8d38)
by [Unito](https://www.unito.io)

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
sno
2025-12-06 06:30:24 +09:00
committed by GitHub
parent 3443c8fb65
commit 5e606f274f

View File

@@ -124,12 +124,19 @@ jobs:
- name: Run Playwright tests (${{ matrix.browser }})
id: playwright
run: |
# Run tests with both HTML and JSON reporters
# Run tests with blob reporter first
pnpm exec playwright test --project=${{ matrix.browser }} --reporter=blob
env:
PLAYWRIGHT_BLOB_OUTPUT_DIR: ./blob-report
- name: Generate HTML and JSON reports
if: always()
run: |
# Generate HTML report from blob
pnpm exec playwright merge-reports --reporter=html ./blob-report
# Generate JSON report separately with explicit output path
PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report/report.json \
pnpm exec playwright test --project=${{ matrix.browser }} \
--reporter=list \
--reporter=html \
--reporter=json
pnpm exec playwright merge-reports --reporter=json ./blob-report
- name: Upload Playwright report
uses: actions/upload-artifact@v4