Commit Graph

3 Commits

Author SHA1 Message Date
Christian Byrne
3b4811b00d feat: deploy E2E coverage HTML report to GitHub Pages (#11291)
## Summary

Browsable E2E coverage report deployed to GitHub Pages on every main
merge, replacing the current workflow of downloading LCOV artifacts and
using an external viewer.

## Changes

- **What**: After merging shard LCOVs, run `genhtml` to produce an HTML
report with per-file line coverage. On `main`, deploy to GitHub Pages
via `actions/deploy-pages`. For PR runs, the HTML report is still
available as the `e2e-coverage-html` artifact.
- **Dependencies**: None new — `genhtml` is part of the `lcov` package
already installed in the workflow.

## Review Focus

- **GitHub Pages must be enabled**: Settings → Pages → Source → "GitHub
Actions". Without this the deploy job will fail silently.
- The deploy job only runs for `main` branch (`if:
github.event.workflow_run.head_branch == 'main'`) so PR coverage doesn't
clobber the deployed report.
- Added `pages: write` and `id-token: write` permissions to the workflow
for the Pages deployment.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11291-feat-deploy-E2E-coverage-HTML-report-to-GitHub-Pages-3446d73d36508136ba6fd806690c9cfc)
by [Unito](https://www.unito.io)

---------

Co-authored-by: Alexander Brown <drjkl@comfy.org>
2026-04-18 15:40:59 -07:00
Christian Byrne
5c07198acb fix: add validation to E2E coverage shard merge (#11290)
## Summary

Add a validation step after merging E2E coverage shards to detect data
loss and improve observability.

## Changes

- **What**: After `lcov -a` merges shard LCOVs, a new step parses merged
+ per-shard stats (source files, lines hit) and writes them to the
**GitHub Actions job summary** as a markdown table. If merged `LH`
(lines hit) is less than any single shard's `LH`, an error annotation is
emitted — this invariant should never be violated since merging should
only add coverage.
- Helps diagnose the 68% → 42% E2E coverage drop after sharding was
introduced.

## Review Focus

The step is informational — it emits `::error::` annotations but does
not `exit 1`, so it won't block the workflow. We can make it a hard
failure once we're confident the merge is stable.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-11290-fix-add-validation-to-E2E-coverage-shard-merge-3446d73d365081c8a942e92deba92006)
by [Unito](https://www.unito.io)
2026-04-15 21:39:51 -07:00
Christian Byrne
033b3dad3a feat: add Slack notification workflow for coverage improvements (#10977)
## Summary

Adds a GitHub Actions workflow + TypeScript script that posts to Slack
when a merged PR improves unit or E2E test coverage.

## Changes

- **What**: New `coverage-slack-notify.yaml` workflow triggered on push
to main. Compares current coverage against previous baselines, generates
Slack Block Kit payload with progress bars and milestone celebrations,
posts to `#p-frontend-automated-testing`.
- **Script**: `scripts/coverage-slack-notify.ts` — parses lcov files,
computes deltas, detects milestone crossings (every 5%), builds Slack
payload. Pure functions exported for testability.
- **Tests**: 26 unit tests in `scripts/coverage-slack-notify.test.ts`
covering all pure functions including edge cases (malformed lcov, exact
boundaries, zero coverage).

### Security hardening
- All `${{ }}` expressions moved from `run:` blocks to `env:` variables
- `SLACK_BOT_TOKEN` passed via env var, not inline
- Unique heredoc delimiter (timestamp-based) prevents payload injection
- `parseInt` fallback (`|| 0`) guards against malformed lcov
- PR regex anchored to first line of commit message

### Robustness
- `continue-on-error: true` on Slack post step (outage does not fail the
job)
- Baseline save guarded by `steps.unit-tests.outcome == success`
(prevents corrupt baselines on test failure)
- Channel ID commented for maintainability
- Top-level `text` field added for Slack mobile push notifications
- Author linked to GitHub profile instead of bare `@username`

## Review Focus

- Workflow step ordering and conditional logic
- Security of expression handling and secret management
- Slack payload structure and Block Kit formatting

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-10977-feat-add-Slack-notification-workflow-for-coverage-improvements-33d6d73d3650819c8950f483c83f297c)
by [Unito](https://www.unito.io)

---------

Co-authored-by: GitHub Action <action@github.com>
2026-04-14 20:58:47 -07:00