[PR #230] [MERGED] ci(rosters): add a leak-safe roster shape probe (TEMPORARY) #238

New issue

Closed

opened 2026-10-26 22:37:40 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:37:40 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/github-workflows/pull/230
Author: @mattmillerai
Created: 8/26/2026
Status: ✅ Merged
Merged: 8/26/2026
Merged by: @mattmillerai

Base: main ← Head: matt/roster-shape-probe

📝 Commits (6)

d39c282 ci(rosters): add a leak-safe roster shape probe (TEMPORARY)
d5c901d fix(rosters): harden the roster shape probe per the review panel
d7f71c2 fix(rosters): close the round-2 review findings on the roster shape probe
fd5ab65 fix(rosters): close the round-3 review findings on the roster shape probe
1865e75 docs(rosters): drop the stale 'digest already public' premise from the probe header
5931d8c fix(rosters): the probe now CONFIRMS a digest instead of publishing one

📊 Changes

1 file changed (+511 additions, -0 deletions)

View changed files

➕ .github/workflows/roster-shape-probe.yml (+511 -0)

📄 Description

ELI5

We keep a list of which repos get automatic code reviews. That list lives in a
safe. You can put a new list in, but you can never open the safe to read the
current one.

Someone wants one repo added. To add one, you have to put in a whole new list —
and since we can't read the old one, we'd be writing it from memory. Guess wrong
and repos silently drop off. Nobody finds out until their automation has quietly
been dead for weeks.

The safe does shout one thing every time it's used: "I'm holding 19 things, and
here's a fingerprint of them." That's enough to check a guess, but not
enough to make one.

This PR adds a button that asks the safe a few harmless questions about the
shape of what's inside — how many items, which optional fields each item has,
what type of value sits in each field. Never the names. Never the label text. Not
even how long they are.

Shape + fingerprint together let us rebuild the list exactly and prove it
matches before writing anything. Then we add the new repo, and the safe's next
fingerprint confirms it landed.

Then we delete the button. It also expires on its own on 2026-09-30, in case we
forget.

Why

A request came in to enrol another caller in CURSOR_REVIEW_CALLERS. That turns
out to be blocked, and this PR is the unblock.

The rosters live only inside write-only Actions secrets: gh secret set
overwrites wholesale and there is no gh secret get. So adding one caller means
rewriting all of them, and we cannot currently prove what "all of them" is.

Live CURSOR_REVIEW_CALLERS is 19 callers, ad58945c… (confirmed on run
32929348118, 2026-08-26). The private ops repo has recovered membership,
order and filenames from the fleet's own bump PRs — but not the entry shape;
~116 candidate serializations have failed to match that digest. Writing an
unverified roster is how you silently unenroll 19 repos.

What this does

A workflow_dispatch-only, self-expiring probe that prints the missing
structural facts and nothing else. One run should close the digest.

count:         19
roster_sha256: ad58945c…                          <- the bumper's own audit digest
keys:          [["file","label","repo","wire_bot"],…]   <- per position; unknown names -> "<unlisted-key>"
wire_bot:      [true,false,null,"absent",…]      <- absent / null / bool kept distinct
label:         ["<string>",null,"absent",…]       <- presence/type only; never text, never length
files:         [".github/workflows/ci-cursor-review.yml","absent",null,"<unlisted>",…]

For a roster with fewer than 8 entries only count + roster_sha256 print
(both already public via the bumper's audit line) — see "The disclosure
argument".

The disclosure argument — this is the bit to review

This repo is public and so are its run logs. Actions masks a secret's exact
value but not anything derived from it, so "it came from a secret" buys nothing
here. Each line is therefore non-disclosing by construction:

printed	why it discloses nothing
`keys`	only the four documented field names; any other key prints `<unlisted-key>`, re-sorted after substitution so its position doesn't hint at the name
`wire_bot`, `label`	booleans, `null`, `"absent"` and type markers (`"<string>"`); no string content and no lengths — over a `""`/`ci`/`Core` vocabulary a length is the label
`files`	printed verbatim only when drawn from the public vocabulary of reusables this repo ships (`(ci-)?<stem>.ya?ml`, anchored); an off-vocabulary string prints `<unlisted>`; absent / null / non-string use the same four-state encoding as the other fields
`roster_sha256`	the whole-roster digest `bump-callers.sh` already logs on every run; verifies a reconstruction we hold. Its residual (a one-entry roster collapses to "is it this repo?") is pre-existing and stated in the header
(nothing below 8 entries)	the shape lines pin every preimage component but the ordered repo list; for a handful of entries that list is small enough to search offline against the already-public digest, so below `MIN_SHAPE_COUNT` the operator does the inverse search locally over the tiny per-entry shape space instead

Deliberately not printed: any repo name, any label text or length, any
repo name length, any key name outside the documented four, any filename
outside the public vocabulary, and — after review — no per-field digests:
sha256(map(.repo)) with count and filenames beside it is an unsalted
membership oracle over an enumerable Comfy-Org/* namespace, and
sha256(map(.label)) over a three-word vocabulary is just the labels in a wrapper.

If you add a line here, re-answer that question for it. If you can't state why it
discloses nothing, it doesn't go in.

One subtlety worth flagging

The per-position vectors use has(), not // null. In jq only null and
false are falsy, so .wire_bot // null reports a present-and-false
wire_bot identically to an absent one — and those two serialize differently
under jq -cS, which would leave the reconstruction unfalsifiable against the
very digest we're trying to match. Absent ("absent"), explicit null, the
boolean, and "<type>" for anything else are four distinct outputs on every
field, file included.

Blast radius

workflow_dispatch only; permissions: {}; 5-minute timeout. A first,
secret-free step fails (rather than silently skipping) on a non-main ref,
re-checks the roster name against a literal allowlist before it is used to
index secrets, and refuses to run after EXPIRES: 2026-09-30 so a forgotten
copy on main goes dead on its own — the date is shape-validated so a typo
fails closed, not open. The header says plainly that the ref check is not a
security boundary (write access is secrets access, as for every
bump-*-callers.yml). The roster gate is slurped and pinned to exactly one JSON
array of objects, so a concatenated paste is a red run rather than a green one
with a digest nothing can match. It touches no roster, opens no PR, and writes
nothing. The live bumper is not modified — this is a separate file
specifically so it cannot affect the fleet, and so it can't trip
test-bump-callers.yml's static bump-*-callers.yml guard.

Verification

actionlint .github/workflows/roster-shape-probe.yml — passes
shellcheck on both extracted step bodies — passes
python3 .github/workflow-pins/check_workflow_pins.py — passes
bash .github/bump-callers/tests/test_bump_callers.sh — 437 passed, 0 failed
Both step bodies run locally against synthetic rosters: a 9-entry roster
mixing every field state (absent / null / false / non-string per field,
listed and unlisted filenames, trailing newline, unanchored prefix, unknown
keys) whose printed digest matches jq -cS . | sha256sum; a 3-entry roster
(shape withheld) and the 7→8 threshold boundary; an array of strings, a bare
object, two concatenated documents, empty, non-JSON; no sha256sum on PATH
(exits non-zero, nothing printed); non-main ref, unknown roster name,
malformed EXPIRES, expired, expires-today

After this merges

Dispatch the probe for CURSOR_REVIEW_CALLERS
Reconstruct exactly; confirm jq -cS . | sha256sum == ad58945c… locally
Commit the verified roster to the private ops repo, then write the secret
with the new caller added (20)
Confirm the next bump run's audit line reports 20 and the new digest
Delete this workflow — or keep it briefly to capture the other uncaptured
rosters (all 12 fleets are selectable) — before it expires on 2026-09-30
(tracked: BE-9651)

Provenance

Verified: actionlint, shellcheck, check_workflow_pins.py and test_bump_callers.sh (437/0) green locally at d7f71c2; both probe step bodies exercised against synthetic rosters and every guard path (see Verification).
Deviations: round-1 panel fixes (d5c901d) dropped the two per-field digests, allowlisted the printed filenames, and replaced the job-level ref if: with a failing, self-expiring guard step; round-2 fixes (d7f71c2) allowlist key names, drop label lengths for presence/type, give file the four-state encoding, withhold the shape lines below 8 entries, validate EXPIRES, slurp the JSON gate, and set permissions: {}. A protected environment: gate was not adopted (repo-settings change, not warranted for a self-expiring throwaway); FILE_ALLOW stays a static snapshot rather than a run-time derivation (that would need a checkout or token in a job meant to hold only the secret) — the output flags possible staleness instead.

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/github-workflows/pull/230 **Author:** [@mattmillerai](https://github.com/mattmillerai) **Created:** 8/26/2026 **Status:** ✅ Merged **Merged:** 8/26/2026 **Merged by:** [@mattmillerai](https://github.com/mattmillerai) **Base:** `main` ← **Head:** `matt/roster-shape-probe` --- ### 📝 Commits (6) - [`d39c282`](https://github.com/Comfy-Org/github-workflows/commit/d39c28230e08fb68ed2ceb07ed445d3729169f6b) ci(rosters): add a leak-safe roster shape probe (TEMPORARY) - [`d5c901d`](https://github.com/Comfy-Org/github-workflows/commit/d5c901dfe13ba508bce1b6b8e49854c57e47a994) fix(rosters): harden the roster shape probe per the review panel - [`d7f71c2`](https://github.com/Comfy-Org/github-workflows/commit/d7f71c26133b4a871819b4f789bfcd260dd047ab) fix(rosters): close the round-2 review findings on the roster shape probe - [`fd5ab65`](https://github.com/Comfy-Org/github-workflows/commit/fd5ab654b12f2809d7d558772b4638d15f297055) fix(rosters): close the round-3 review findings on the roster shape probe - [`1865e75`](https://github.com/Comfy-Org/github-workflows/commit/1865e7536210fab024338a443160ccee16c715a4) docs(rosters): drop the stale 'digest already public' premise from the probe header - [`5931d8c`](https://github.com/Comfy-Org/github-workflows/commit/5931d8c2f6d9105c18c74dec95fa9878a35ba7e6) fix(rosters): the probe now CONFIRMS a digest instead of publishing one ### 📊 Changes **1 file changed** (+511 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/roster-shape-probe.yml` (+511 -0) </details> ### 📄 Description ## ELI5 We keep a list of which repos get automatic code reviews. That list lives in a safe. You can put a **new** list in, but you can never open the safe to read the current one. Someone wants one repo added. To add one, you have to put in a whole new list — and since we can't read the old one, we'd be writing it from memory. Guess wrong and repos silently drop off. Nobody finds out until their automation has quietly been dead for weeks. The safe does shout one thing every time it's used: *"I'm holding 19 things, and here's a fingerprint of them."* That's enough to **check** a guess, but not enough to **make** one. This PR adds a button that asks the safe a few harmless questions about the **shape** of what's inside — how many items, which optional fields each item has, what type of value sits in each field. Never the names. Never the label text. Not even how long they are. Shape + fingerprint together let us rebuild the list exactly and **prove** it matches before writing anything. Then we add the new repo, and the safe's next fingerprint confirms it landed. Then we delete the button. It also expires on its own on 2026-09-30, in case we forget. ## Why A request came in to enrol another caller in `CURSOR_REVIEW_CALLERS`. That turns out to be blocked, and this PR is the unblock. The rosters live only inside **write-only** Actions secrets: `gh secret set` overwrites wholesale and there is no `gh secret get`. So adding one caller means rewriting all of them, and we cannot currently prove what "all of them" is. Live `CURSOR_REVIEW_CALLERS` is **19 callers, `ad58945c…`** (confirmed on run `32929348118`, 2026-08-26). The private ops repo has recovered membership, order and filenames from the fleet's own bump PRs — but **not the entry shape**; ~116 candidate serializations have failed to match that digest. Writing an unverified roster is how you silently unenroll 19 repos. ## What this does A `workflow_dispatch`-only, **self-expiring** probe that prints the missing structural facts and nothing else. One run should close the digest. ``` count: 19 roster_sha256: ad58945c… <- the bumper's own audit digest keys: [["file","label","repo","wire_bot"],…] <- per position; unknown names -> "<unlisted-key>" wire_bot: [true,false,null,"absent",…] <- absent / null / bool kept distinct label: ["<string>",null,"absent",…] <- presence/type only; never text, never length files: [".github/workflows/ci-cursor-review.yml","absent",null,"<unlisted>",…] ``` For a roster with fewer than **8** entries only `count` + `roster_sha256` print (both already public via the bumper's audit line) — see "The disclosure argument". ## The disclosure argument — this is the bit to review This repo is public and so are its run logs. **Actions masks a secret's exact value but not anything derived from it**, so "it came from a secret" buys nothing here. Each line is therefore non-disclosing *by construction*: | printed | why it discloses nothing | |---|---| | `keys` | only the four documented field names; any other key prints `<unlisted-key>`, re-sorted after substitution so its position doesn't hint at the name | | `wire_bot`, `label` | booleans, `null`, `"absent"` and type markers (`"<string>"`); no string content and **no lengths** — over a `""`/`ci`/`Core` vocabulary a length *is* the label | | `files` | printed verbatim **only** when drawn from the public vocabulary of reusables this repo ships (`(ci-)?<stem>.ya?ml`, anchored); an off-vocabulary string prints `<unlisted>`; absent / null / non-string use the same four-state encoding as the other fields | | `roster_sha256` | the whole-roster digest `bump-callers.sh` already logs on every run; verifies a reconstruction we hold. Its residual (a one-entry roster collapses to "is it this repo?") is pre-existing and stated in the header | | *(nothing below 8 entries)* | the shape lines pin every preimage component but the ordered repo list; for a handful of entries that list is small enough to search offline against the already-public digest, so below `MIN_SHAPE_COUNT` the operator does the inverse search locally over the tiny per-entry shape space instead | **Deliberately not printed:** any repo name, any label text **or length**, any repo name **length**, any key name outside the documented four, any filename outside the public vocabulary, and — after review — **no per-field digests**: `sha256(map(.repo))` with count and filenames beside it is an unsalted membership oracle over an enumerable `Comfy-Org/*` namespace, and `sha256(map(.label))` over a three-word vocabulary is just the labels in a wrapper. If you add a line here, re-answer that question for it. If you can't state why it discloses nothing, it doesn't go in. ## One subtlety worth flagging The per-position vectors use `has()`, not `// null`. In jq only `null` and `false` are falsy, so `.wire_bot // null` reports a present-and-**false** `wire_bot` identically to an **absent** one — and those two serialize differently under `jq -cS`, which would leave the reconstruction unfalsifiable against the very digest we're trying to match. Absent (`"absent"`), explicit `null`, the boolean, and `"<type>"` for anything else are four distinct outputs on every field, `file` included. ## Blast radius `workflow_dispatch` only; `permissions: {}`; 5-minute timeout. A first, secret-free step **fails** (rather than silently skipping) on a non-main ref, re-checks the roster name against a literal allowlist before it is used to index `secrets`, and refuses to run after `EXPIRES: 2026-09-30` so a forgotten copy on `main` goes dead on its own — the date is shape-validated so a typo fails closed, not open. The header says plainly that the ref check is not a security boundary (write access is secrets access, as for every `bump-*-callers.yml`). The roster gate is slurped and pinned to exactly one JSON array of objects, so a concatenated paste is a red run rather than a green one with a digest nothing can match. It touches no roster, opens no PR, and writes nothing. The live bumper is **not modified** — this is a separate file specifically so it cannot affect the fleet, and so it can't trip `test-bump-callers.yml`'s static `bump-*-callers.yml` guard. ## Verification - `actionlint .github/workflows/roster-shape-probe.yml` — passes - `shellcheck` on both extracted step bodies — passes - `python3 .github/workflow-pins/check_workflow_pins.py` — passes - `bash .github/bump-callers/tests/test_bump_callers.sh` — 437 passed, 0 failed - Both step bodies run locally against synthetic rosters: a 9-entry roster mixing every field state (absent / null / false / non-string per field, listed and unlisted filenames, trailing newline, unanchored prefix, unknown keys) whose printed digest matches `jq -cS . | sha256sum`; a 3-entry roster (shape withheld) and the 7→8 threshold boundary; an array of strings, a bare object, two concatenated documents, empty, non-JSON; no `sha256sum` on PATH (exits non-zero, nothing printed); non-main ref, unknown roster name, malformed `EXPIRES`, expired, expires-today ## After this merges 1. Dispatch the probe for `CURSOR_REVIEW_CALLERS` 2. Reconstruct exactly; confirm `jq -cS . | sha256sum == ad58945c…` **locally** 3. Commit the verified roster to the private ops repo, then write the secret with the new caller added (20) 4. Confirm the next bump run's audit line reports 20 and the new digest 5. Delete this workflow — or keep it briefly to capture the other uncaptured rosters (all 12 fleets are selectable) — before it expires on 2026-09-30 (tracked: BE-9651) ## Provenance **Verified:** actionlint, shellcheck, check_workflow_pins.py and test_bump_callers.sh (437/0) green locally at d7f71c2; both probe step bodies exercised against synthetic rosters and every guard path (see Verification). **Deviations:** round-1 panel fixes (d5c901d) dropped the two per-field digests, allowlisted the printed filenames, and replaced the job-level ref `if:` with a failing, self-expiring guard step; round-2 fixes (d7f71c2) allowlist key names, drop label lengths for presence/type, give `file` the four-state encoding, withhold the shape lines below 8 entries, validate `EXPIRES`, slurp the JSON gate, and set `permissions: {}`. A protected `environment:` gate was not adopted (repo-settings change, not warranted for a self-expiring throwaway); `FILE_ALLOW` stays a static snapshot rather than a run-time derivation (that would need a checkout or token in a job meant to hold only the secret) — the output flags possible staleness instead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:37:40 +00:00