[PR #86] [MERGED] chore(cursor-review): bump panel pins to claude-opus-5-thinking-max and kimi-k3-max (BE-4818) #96

New issue

Closed

opened 2026-10-26 22:36:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:36:11 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/github-workflows/pull/86
Author: @mattmillerai
Created: 7/28/2026
Status: ✅ Merged
Merged: 8/4/2026
Merged by: @mattmillerai

Base: main ← Head: matt/be-4818-bump-panel-pins

📝 Commits (2)

6fd9efa chore(cursor-review): bump panel pins to claude-opus-5-thinking-max and kimi-k3-max (BE-4818)
227ffe2 Merge origin/main, resolve cursor-review.yml catalog-comment conflict (BE-4818)

📊 Changes

2 files changed (+14 additions, -12 deletions)

View changed files

📝 .github/cursor-review/README.md (+1 -1)
📝 .github/workflows/cursor-review.yml (+13 -11)

📄 Description

ELI-5

The PR reviewer here is a panel of four AI models, one per lab. Two of them just got newer, smarter siblings, so this swaps in the new names: Anthropic's Opus 4.8 becomes Opus 5, and Moonshot's Kimi K2.7 Code becomes Kimi K3. The OpenAI and Google picks stay put. Nothing about how the review works changes — only which four models do it.

What

Refreshes the cursor-review model panel per the BE-4817 spike audit against the live cursor-agent models catalog (2026-07-27). Two slots superseded, two unchanged.

.github/workflows/cursor-review.yml — judge_model input default claude-opus-4-8-thinking-max → claude-opus-5-thinking-max (the judge is documented as matching the panel's highest reasoning tier, so it moves with the Anthropic slot).
.github/workflows/cursor-review.yml — "Define panel models" heredoc (the single source of truth the review matrix consumes via fromJSON): claude-opus-4-8-thinking-max → claude-opus-5-thinking-max, kimi-k2.7-code → kimi-k3-max. gpt-5.6-sol-max and gemini-3.1-pro unchanged. The catalog comment block above it is rewritten to record the new audit date and rationale; its structure (single-source-of-truth note, README-sync note) is preserved.
.github/cursor-review/README.md — lab/model table and the judge_model row of the inputs table.
.github/cursor-review/prompt-judge.md — the attribution example that names a panel model verbatim (found by the repo-wide grep the ticket asked for).

Panel shape (8 jobs / 4 labs / adversarial + edge-case) is unchanged.

Why these ids

Every claim in the new comment block was checked against the actual catalog printed by the preflight job of Comfy-Org/comfy-cloud-mcp-server run 30314000838 (job 90135615477, 2026-07-27T23:24Z), not taken from the ticket on faith:

claude-opus-5-thinking-max - Opus 5 1M Max Thinking — present verbatim, no (NO ZDR) marker.
kimi-k3-max - Kimi K3 — present verbatim, no (NO ZDR) marker. (kimi-k3-high / kimi-k3-low also exist; max is the top tier.)
Every claude-fable-5-* entry carries (NO ZDR), which is why Fable stays excluded from a panel that reads private-repo diffs.
Codex line stops at 5.3 — the only codex entries are gpt-5.3-codex-*; there is no 5.6 codex variant.
gemini-3.1-pro is still the only Pro-line Gemini entry; 3.5/3.6 are Flash tier.
GPT-5.6 Luna and Terra both exist as full tiers alongside Sol, matching the rewritten comment.

Cost: Opus 5 is priced identically to Opus 4.8 ($5/$25 per M), so the Anthropic swap and the judge bump are cost-neutral. Kimi K3 ($3/$15) vs K2.7 Code ($0.95/$4) is ~3.2x/3.75x per token on this lab's two panel cells — roughly +10-15% total panel cost per PR. That increase was accepted by the spike audit and is now recorded in the comment block.

Testing

python3 -m unittest discover -s .github/cursor-review/tests -p 'test_*.py' — 40 passed.
Also ran the untouched suites to be safe: agents-md-integrity (18 passed), groom (87 passed), and check_agents_md.py --root . (passed; the one warning is the pre-existing missing-CODEOWNERS DRI warning, unrelated to this change).
Parsed cursor-review.yml with PyYAML and re-parsed the heredoc as JSON to confirm the pin list is exactly ["gpt-5.6-sol-max", "claude-opus-5-thinking-max", "gemini-3.1-pro", "kimi-k3-max"] — a YAML/JSON typo in the heredoc would otherwise only surface at run time.
The real gate is CI: review has needs: [gate, diff-size, preflight] and its matrix is fromJSON(needs.preflight.outputs.models), so the preflight catalog check fails the run loudly on an unknown id rather than silently dropping a lab. Caveat: that gate does NOT cover this PR. ci-cursor-review.yml deliberately pins the reusable workflow to a merged main SHA (so a PR cannot weaken the panel reviewing it), so this PR's own cursor-review run validated the previous pins. The new ids are verified by hand against the live catalog printed by that run's preflight job (30321904319, 2026-07-28T02:01Z), which lists claude-opus-5-thinking-max and kimi-k3-max verbatim. Preflight will gate them for real on the first PR after this merges and the caller pin is bumped.

Judgment calls / notes

judge_model is not covered by the preflight check. The catalog validation only walks the pins in /tmp/models.json; JUDGE_MODEL is not among them, so a bad judge id would fail at judge time instead of preflight. I verified claude-opus-5-thinking-max against the live catalog by hand (above). Extending preflight to validate the judge id too would be a real improvement but is outside this ticket's scope — happy to file it separately.
groom.yml line 241 still mentions claude-opus-4-8 in a doc comment ("can pin an older id (e.g. claude-opus-4-8) to hold a run steady"). It is neither of the two ids the ticket targets, it belongs to a different workflow with an independent model input, and it is illustrating an older id — so the sentence is still accurate. Deliberately left alone.
Consumers do not get this until their caller SHA is bumped (out of scope per the ticket; bump-cursor-review-callers.yml with workflow_dispatch exists for that and is tracked separately).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/github-workflows/pull/86 **Author:** [@mattmillerai](https://github.com/mattmillerai) **Created:** 7/28/2026 **Status:** ✅ Merged **Merged:** 8/4/2026 **Merged by:** [@mattmillerai](https://github.com/mattmillerai) **Base:** `main` ← **Head:** `matt/be-4818-bump-panel-pins` --- ### 📝 Commits (2) - [`6fd9efa`](https://github.com/Comfy-Org/github-workflows/commit/6fd9efacee276479273fc331690dc2132641f74f) chore(cursor-review): bump panel pins to claude-opus-5-thinking-max and kimi-k3-max (BE-4818) - [`227ffe2`](https://github.com/Comfy-Org/github-workflows/commit/227ffe21742774d9ecf3b8fc7a8145c5919b300c) Merge origin/main, resolve cursor-review.yml catalog-comment conflict (BE-4818) ### 📊 Changes **2 files changed** (+14 additions, -12 deletions) <details> <summary>View changed files</summary> 📝 `.github/cursor-review/README.md` (+1 -1) 📝 `.github/workflows/cursor-review.yml` (+13 -11) </details> ### 📄 Description ## ELI-5 The PR reviewer here is a panel of four AI models, one per lab. Two of them just got newer, smarter siblings, so this swaps in the new names: Anthropic's Opus 4.8 becomes Opus 5, and Moonshot's Kimi K2.7 Code becomes Kimi K3. The OpenAI and Google picks stay put. Nothing about how the review works changes — only which four models do it. ## What Refreshes the `cursor-review` model panel per the BE-4817 spike audit against the live `cursor-agent models` catalog (2026-07-27). Two slots superseded, two unchanged. - `.github/workflows/cursor-review.yml` — `judge_model` input default `claude-opus-4-8-thinking-max` → `claude-opus-5-thinking-max` (the judge is documented as matching the panel's highest reasoning tier, so it moves with the Anthropic slot). - `.github/workflows/cursor-review.yml` — "Define panel models" heredoc (the single source of truth the `review` matrix consumes via `fromJSON`): `claude-opus-4-8-thinking-max` → `claude-opus-5-thinking-max`, `kimi-k2.7-code` → `kimi-k3-max`. `gpt-5.6-sol-max` and `gemini-3.1-pro` unchanged. The catalog comment block above it is rewritten to record the new audit date and rationale; its structure (single-source-of-truth note, README-sync note) is preserved. - `.github/cursor-review/README.md` — lab/model table and the `judge_model` row of the inputs table. - `.github/cursor-review/prompt-judge.md` — the attribution example that names a panel model verbatim (found by the repo-wide grep the ticket asked for). Panel shape (8 jobs / 4 labs / adversarial + edge-case) is unchanged. ## Why these ids Every claim in the new comment block was checked against the actual catalog printed by the preflight job of `Comfy-Org/comfy-cloud-mcp-server` run `30314000838` (job `90135615477`, 2026-07-27T23:24Z), not taken from the ticket on faith: - `claude-opus-5-thinking-max - Opus 5 1M Max Thinking` — present verbatim, **no** `(NO ZDR)` marker. - `kimi-k3-max - Kimi K3` — present verbatim, **no** `(NO ZDR)` marker. (`kimi-k3-high` / `kimi-k3-low` also exist; `max` is the top tier.) - Every `claude-fable-5-*` entry carries `(NO ZDR)`, which is why Fable stays excluded from a panel that reads private-repo diffs. - Codex line stops at 5.3 — the only codex entries are `gpt-5.3-codex-*`; there is no 5.6 codex variant. - `gemini-3.1-pro` is still the only Pro-line Gemini entry; 3.5/3.6 are Flash tier. - GPT-5.6 Luna and Terra both exist as full tiers alongside Sol, matching the rewritten comment. Cost: Opus 5 is priced identically to Opus 4.8 ($5/$25 per M), so the Anthropic swap and the judge bump are cost-neutral. Kimi K3 ($3/$15) vs K2.7 Code ($0.95/$4) is ~3.2x/3.75x per token on this lab's two panel cells — roughly +10-15% total panel cost per PR. That increase was accepted by the spike audit and is now recorded in the comment block. ## Testing - `python3 -m unittest discover -s .github/cursor-review/tests -p 'test_*.py'` — 40 passed. - Also ran the untouched suites to be safe: agents-md-integrity (18 passed), groom (87 passed), and `check_agents_md.py --root .` (passed; the one warning is the pre-existing missing-CODEOWNERS DRI warning, unrelated to this change). - Parsed `cursor-review.yml` with PyYAML and re-parsed the heredoc as JSON to confirm the pin list is exactly `["gpt-5.6-sol-max", "claude-opus-5-thinking-max", "gemini-3.1-pro", "kimi-k3-max"]` — a YAML/JSON typo in the heredoc would otherwise only surface at run time. - The real gate is CI: `review` has `needs: [gate, diff-size, preflight]` and its matrix is `fromJSON(needs.preflight.outputs.models)`, so the preflight catalog check fails the run loudly on an unknown id rather than silently dropping a lab. **Caveat:** that gate does NOT cover this PR. `ci-cursor-review.yml` deliberately pins the reusable workflow to a merged `main` SHA (so a PR cannot weaken the panel reviewing it), so this PR's own cursor-review run validated the *previous* pins. The new ids are verified by hand against the live catalog printed by that run's preflight job ([30321904319](https://github.com/Comfy-Org/github-workflows/actions/runs/30321904319/job/90159390361), 2026-07-28T02:01Z), which lists `claude-opus-5-thinking-max` and `kimi-k3-max` verbatim. Preflight will gate them for real on the first PR after this merges and the caller pin is bumped. ## Judgment calls / notes - **`judge_model` is not covered by the preflight check.** The catalog validation only walks the pins in `/tmp/models.json`; `JUDGE_MODEL` is not among them, so a bad judge id would fail at judge time instead of preflight. I verified `claude-opus-5-thinking-max` against the live catalog by hand (above). Extending preflight to validate the judge id too would be a real improvement but is outside this ticket's scope — happy to file it separately. - **`groom.yml` line 241 still mentions `claude-opus-4-8`** in a doc comment ("can pin an older id (e.g. `claude-opus-4-8`) to hold a run steady"). It is neither of the two ids the ticket targets, it belongs to a different workflow with an independent model input, and it is *illustrating* an older id — so the sentence is still accurate. Deliberately left alone. - **Consumers do not get this until their caller SHA is bumped** (out of scope per the ticket; `bump-cursor-review-callers.yml` with `workflow_dispatch` exists for that and is tracked separately). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:36:11 +00:00