From 5e606f274fdfb085015668baa8d3c30cef23a2df Mon Sep 17 00:00:00 2001
From: sno <snomiao@gmail.com>
Date: Sat, 6 Dec 2025 06:30:24 +0900
Subject: [PATCH] [bugfix] Fix E2E test report generation for non-chromium
 browsers (#7193)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## Summary

Fixes an issue where Playwright HTML reports were not being generated
for `chromium-2x`, `chromium-0.5x`, and `mobile-chrome` test runs,
causing 404 errors when accessing report links in PR comments.

## Problem

Only the first report link (chromium) had contents - the other three
browsers returned 404 errors. Investigation revealed that artifacts for
non-chromium browsers were only uploading 1 file (report.json) instead
of the full HTML report with ~32 files.

The root cause was that these browsers run very few tests (1-6 tests
each), and when using `--reporter=html --reporter=json` together,
Playwright would only generate the JSON report without the HTML assets.

## Solution

Changed the workflow for non-chromium browsers to use the same approach
as the sharded chromium tests:
1. Run tests with `--reporter=blob`
2. Generate HTML and JSON reports separately using `merge-reports`

This ensures both HTML and JSON reports are always generated with
complete assets, regardless of test count.

## Testing

The fix can be verified by:
1. Checking that this PR's workflow run uploads similar file counts for
all browsers
2. Confirming all 4 report links are accessible and show proper HTML
content

Related to #7186

🤖 Generated with [Claude Code](https://claude.com/claude-code)

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-7193-bugfix-Fix-E2E-test-report-generation-for-non-chromium-browsers-2c06d73d365081ba8ea3ed0d3f5d8d38)
by [Unito](https://www.unito.io)

Co-authored-by: Claude <noreply@anthropic.com>
---
 .github/workflows/ci-tests-e2e.yaml | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/.github/workflows/ci-tests-e2e.yaml b/.github/workflows/ci-tests-e2e.yaml
index f4a770574..0363ec47a 100644
--- a/.github/workflows/ci-tests-e2e.yaml
+++ b/.github/workflows/ci-tests-e2e.yaml
@@ -124,12 +124,19 @@ jobs:
       - name: Run Playwright tests (${{ matrix.browser }})
         id: playwright
         run: |
-          # Run tests with both HTML and JSON reporters
+          # Run tests with blob reporter first
+          pnpm exec playwright test --project=${{ matrix.browser }} --reporter=blob
+        env:
+          PLAYWRIGHT_BLOB_OUTPUT_DIR: ./blob-report
+
+      - name: Generate HTML and JSON reports
+        if: always()
+        run: |
+          # Generate HTML report from blob
+          pnpm exec playwright merge-reports --reporter=html ./blob-report
+          # Generate JSON report separately with explicit output path
           PLAYWRIGHT_JSON_OUTPUT_NAME=playwright-report/report.json \
-          pnpm exec playwright test --project=${{ matrix.browser }} \
-            --reporter=list \
-            --reporter=html \
-            --reporter=json
+          pnpm exec playwright merge-reports --reporter=json ./blob-report
 
       - name: Upload Playwright report
         uses: actions/upload-artifact@v4