[PR #373] [MERGED] feat(cursor-review): move the judge and the five cursor-approve axes to Opus 5.5 #374

New issue

Closed

opened 2026-10-26 22:39:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:39:09 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/github-workflows/pull/373
Author: @mattmillerai
Created: 10/7/2026
Status: ✅ Merged
Merged: 10/7/2026
Merged by: @mattmillerai

Base: main ← Head: matt/judge-opus-5-5

📝 Commits (2)

af4b70d feat(cursor-review): move the judge and the five cursor-approve axes to Opus 5.5
0464f87 fix(cursor-review): restore the pin staleness clock that #371 reset, and cite the 5.5 catalog read

📊 Changes

10 files changed (+54 additions, -31 deletions)

View changed files

📝 .github/cursor-review/README.md (+4 -5)
📝 .github/workflows/axis-business.yml (+1 -1)
📝 .github/workflows/axis-completeness.yml (+1 -1)
📝 .github/workflows/axis-conformance.yml (+1 -1)
📝 .github/workflows/axis-correctness.yml (+1 -1)
📝 .github/workflows/axis-design.yml (+1 -1)
📝 .github/workflows/cursor-axis-base.yml (+1 -1)
📝 .github/workflows/cursor-review.yml (+36 -17)
📝 docs/callers/cursor-approve.md (+2 -2)
📝 docs/callers/cursor-review.md (+6 -1)

📄 Description

Why

#371 moved the panel's Anthropic cell to Opus 5.5 and left the judge on Opus 5. That was me scoping, not a reason — and a half-migrated state is exactly the drift this repo keeps warning about. There's no argument for the judge staying behind.

All seven defaults move claude-opus-5-thinking-xhigh → claude-opus-5-5-xhigh. Same tier, newer family.

File	Input
`cursor-review.yml`	`judge_model` — consolidates the panel
`cursor-axis-base.yml`	`model` — shared axis base
`axis-correctness.yml` / `axis-conformance.yml` / `axis-business.yml` / `axis-design.yml` / `axis-completeness.yml`	`model`

The 33% is measured, not assumed

From our own Cursor invoices, normalised to input-equivalent tokens (output 5x, cache-write 1.25x, cache-read 0.1x input):

Model	Rate
Opus 5 `-thinking-max`	$4.97/Mtok-ie
Opus 5 `-thinking-xhigh`	$4.87/Mtok-ie
Opus 5.5	$3.32/Mtok-ie

The normalisation is validated rather than asserted: opus-5-thinking-xhigh appears in two separate billing cycles at very different volumes (469.5 and 252.1 Mtok-ie) and derives $4.86 and $4.89 — 0.6% apart.

Note what this also shows: the tier is not a rate lever. max and xhigh on Opus 5 differ by 2%, i.e. noise. A tier drop saves by generating fewer reasoning tokens, not by costing less per token.

Why this is likely bigger than #371

The judge runs on every round of every PR across the whole fleet, and the axes run per-axis on every cursor-approve decision. Those seven defaults are plausibly a larger share of panel spend than any single review cell.

On thinking, since Opus 5.5 ships no `-thinking-` variants

The plain line is the thinking model. Cursor labels non-thinking explicitly where it applies — every plain Haiku 5.5 entry reads No Thinking — and no Opus 5.5 or Sonnet 5.5 entry carries that marker. Read from the live catalog in the preflight log of run 37688671671. So this is tier-for-tier, not a silent loss of reasoning. (Same reasoning validates #371.)

Reviewer, please note the blast radius

This touches cursor-approve, not just the review panel. The five axes feed approval decisions on enrolled callers, so this changes the model making those calls — not merely an advisory review. Severity thresholds and axis prompts are untouched; only the model moves.

Gap found while doing this

catalog-drift.py covers the judge_model default via extract_judge_model, but not the six axis defaults. A delisted or newly-NO ZDR id sitting in an axis workflow would never be flagged by the weekly check. Not fixed here — worth its own PR.

Testing

.github/cursor-review/tests — 1147 passed
.github/cursor-approve/tests — 162 passed (1 skipped)
check_workflow_pins.py, check_agents_md.py --root . — pass
YAML parsed on all seven workflows
catalog-drift.py re-extracts: panel ['gpt-5.6-sol-max', 'claude-opus-5-5-xhigh', 'kimi-k3-high'], judge claude-opus-5-5-xhigh

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/github-workflows/pull/373 **Author:** [@mattmillerai](https://github.com/mattmillerai) **Created:** 10/7/2026 **Status:** ✅ Merged **Merged:** 10/7/2026 **Merged by:** [@mattmillerai](https://github.com/mattmillerai) **Base:** `main` ← **Head:** `matt/judge-opus-5-5` --- ### 📝 Commits (2) - [`af4b70d`](https://github.com/Comfy-Org/github-workflows/commit/af4b70d4df829e44755971b93bd3f1601b435b6f) feat(cursor-review): move the judge and the five cursor-approve axes to Opus 5.5 - [`0464f87`](https://github.com/Comfy-Org/github-workflows/commit/0464f8724f6f1add600c11ffe44d45c493a0f603) fix(cursor-review): restore the pin staleness clock that #371 reset, and cite the 5.5 catalog read ### 📊 Changes **10 files changed** (+54 additions, -31 deletions) <details> <summary>View changed files</summary> 📝 `.github/cursor-review/README.md` (+4 -5) 📝 `.github/workflows/axis-business.yml` (+1 -1) 📝 `.github/workflows/axis-completeness.yml` (+1 -1) 📝 `.github/workflows/axis-conformance.yml` (+1 -1) 📝 `.github/workflows/axis-correctness.yml` (+1 -1) 📝 `.github/workflows/axis-design.yml` (+1 -1) 📝 `.github/workflows/cursor-axis-base.yml` (+1 -1) 📝 `.github/workflows/cursor-review.yml` (+36 -17) 📝 `docs/callers/cursor-approve.md` (+2 -2) 📝 `docs/callers/cursor-review.md` (+6 -1) </details> ### 📄 Description ## Why #371 moved the panel's Anthropic cell to Opus 5.5 and left the judge on Opus 5. That was me scoping, not a reason — and a half-migrated state is exactly the drift this repo keeps warning about. There's no argument for the judge staying behind. All seven defaults move `claude-opus-5-thinking-xhigh` → `claude-opus-5-5-xhigh`. Same tier, newer family. | File | Input | |---|---| | `cursor-review.yml` | `judge_model` — consolidates the panel | | `cursor-axis-base.yml` | `model` — shared axis base | | `axis-correctness.yml` / `axis-conformance.yml` / `axis-business.yml` / `axis-design.yml` / `axis-completeness.yml` | `model` | ## The 33% is measured, not assumed From our own Cursor invoices, normalised to **input-equivalent tokens** (output 5x, cache-write 1.25x, cache-read 0.1x input): | Model | Rate | |---|---| | Opus 5 `-thinking-max` | $4.97/Mtok-ie | | Opus 5 `-thinking-xhigh` | $4.87/Mtok-ie | | **Opus 5.5** | **$3.32/Mtok-ie** | The normalisation is **validated rather than asserted**: `opus-5-thinking-xhigh` appears in two separate billing cycles at very different volumes (469.5 and 252.1 Mtok-ie) and derives $4.86 and $4.89 — 0.6% apart. Note what this also shows: **the tier is not a rate lever.** `max` and `xhigh` on Opus 5 differ by 2%, i.e. noise. A tier drop saves by generating fewer reasoning tokens, not by costing less per token. ## Why this is likely bigger than #371 The judge runs on **every round of every PR across the whole fleet**, and the axes run per-axis on every cursor-approve decision. Those seven defaults are plausibly a larger share of panel spend than any single review cell. ## On thinking, since Opus 5.5 ships no `-thinking-` variants The plain line **is** the thinking model. Cursor labels non-thinking explicitly where it applies — every plain Haiku 5.5 entry reads `No Thinking` — and no Opus 5.5 or Sonnet 5.5 entry carries that marker. Read from the live catalog in the preflight log of run `37688671671`. So this is tier-for-tier, not a silent loss of reasoning. (Same reasoning validates #371.) ## Reviewer, please note the blast radius This touches **cursor-approve**, not just the review panel. The five axes feed approval decisions on enrolled callers, so this changes the model making those calls — not merely an advisory review. Severity thresholds and axis prompts are untouched; only the model moves. ## Gap found while doing this `catalog-drift.py` covers the `judge_model` default via `extract_judge_model`, but **not** the six axis defaults. A delisted or newly-`NO ZDR` id sitting in an axis workflow would never be flagged by the weekly check. Not fixed here — worth its own PR. ## Testing - `.github/cursor-review/tests` — 1147 passed - `.github/cursor-approve/tests` — 162 passed (1 skipped) - `check_workflow_pins.py`, `check_agents_md.py --root .` — pass - YAML parsed on all seven workflows - `catalog-drift.py` re-extracts: panel `['gpt-5.6-sol-max', 'claude-opus-5-5-xhigh', 'kimi-k3-high']`, judge `claude-opus-5-5-xhigh` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:39:09 +00:00