From 9447a1f5d6d52580ec88c50faf7986e6e4b84597 Mon Sep 17 00:00:00 2001 From: Benjamin Lu Date: Fri, 13 Mar 2026 09:57:56 -0700 Subject: [PATCH] test: warn on fix PRs without e2e regression coverage (#9880) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Add a CodeRabbit pre-merge warning for fix-like PRs that do not update `browser_tests/` and do not explain why no end-to-end regression test was added. Requested by Christian ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-9880-test-warn-on-fix-PRs-without-e2e-regression-coverage-3226d73d3650816eb3e1c1c7d0824edd) by [Unito](https://www.unito.io) --- .coderabbit.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.coderabbit.yaml b/.coderabbit.yaml index 5b3ec34b3f..7a380aaf1e 100644 --- a/.coderabbit.yaml +++ b/.coderabbit.yaml @@ -12,3 +12,14 @@ reviews: - comfy-pr-bot - github-actions - github-actions[bot] + pre_merge_checks: + custom_checks: + - name: End-to-end regression coverage for fixes + mode: warning + instructions: | + Pass if at least one of the following is true: + 1. Neither the PR title nor any commit subject in the PR uses bug-fix language such as `fix`, `fixed`, `fixes`, `fixing`, `bugfix`, or `hotfix`. + 2. The PR changes at least one file under `browser_tests/`. + 3. The PR description includes a concrete, non-placeholder explanation of why an end-to-end regression test was not added. + + Fail otherwise. When failing, mention which bug-fix signal you found and ask the author to either add or update a Playwright regression test under `browser_tests/` or add a concrete explanation in the PR description of why an end-to-end regression test is not practical.