fix: add validation to E2E coverage shard merge (#11290)

## Summary

Add a validation step after merging E2E coverage shards to detect data
loss and improve observability.

## Changes

- **What**: After `lcov -a` merges shard LCOVs, a new step parses merged
+ per-shard stats (source files, lines hit) and writes them to the
**GitHub Actions job summary** as a markdown table. If merged `LH`
(lines hit) is less than any single shard's `LH`, an error annotation is
emitted — this invariant should never be violated since merging should
only add coverage.
- Helps diagnose the 68% → 42% E2E coverage drop after sharding was
introduced.

## Review Focus

The step is informational — it emits `::error::` annotations but does
not `exit 1`, so it won't block the workflow. We can make it a hard
failure once we're confident the merge is stable.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11290-fix-add-validation-to-E2E-coverage-shard-merge-3446d73d365081c8a942e92deba92006)
by [Unito](https://www.unito.io)
This commit is contained in:
Christian Byrne
2026-04-15 21:39:51 -07:00
committed by GitHub
parent 6fb90b224d
commit 5c07198acb

View File

@@ -54,6 +54,33 @@ jobs:
lcov $ADD_ARGS -o coverage/playwright/coverage.lcov
wc -l coverage/playwright/coverage.lcov
- name: Validate merged coverage
run: |
SHARD_COUNT=$(find temp/coverage-shards -name 'coverage.lcov' -type f | wc -l | tr -d ' ')
if [ "$SHARD_COUNT" -eq 0 ]; then
echo "::error::No shard coverage.lcov files found under temp/coverage-shards"
exit 1
fi
MERGED_SF=$(grep -c '^SF:' coverage/playwright/coverage.lcov || echo 0)
MERGED_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
MERGED_LF=$(awk -F: '/^LF:/{s+=$2}END{print s+0}' coverage/playwright/coverage.lcov)
echo "### Merged coverage" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_SF** source files" >> "$GITHUB_STEP_SUMMARY"
echo "- **$MERGED_LH / $MERGED_LF** lines hit" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Shard | Files | Lines Hit |" >> "$GITHUB_STEP_SUMMARY"
echo "|-------|-------|-----------|" >> "$GITHUB_STEP_SUMMARY"
for f in $(find temp/coverage-shards -name 'coverage.lcov' -type f | sort); do
SHARD=$(basename "$(dirname "$f")")
SHARD_SF=$(grep -c '^SF:' "$f" || echo 0)
SHARD_LH=$(awk -F: '/^LH:/{s+=$2}END{print s+0}' "$f")
echo "| $SHARD | $SHARD_SF | $SHARD_LH |" >> "$GITHUB_STEP_SUMMARY"
if [ "$MERGED_LH" -lt "$SHARD_LH" ]; then
echo "::error::Merged LH ($MERGED_LH) < shard LH ($SHARD_LH) in $SHARD — possible data loss"
fi
done
- name: Upload merged coverage data
if: always()
uses: actions/upload-artifact@v6