[PR #9] [MERGED] Add torch-index-stacks.json manifest with validated R2 publishing #14

New issue

Closed

opened 2026-10-26 22:22:56 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:22:56 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/pull/9
Author: @Kosinkadink
Created: 7/25/2026
Status: ✅ Merged
Merged: 7/26/2026
Merged by: @Kosinkadink

Base: main ← Head: feat/torch-index-stacks-manifest

📝 Commits (10+)

1f9294d feat: add torch-index-stacks.json manifest with validated R2 publishing
eb7d9af Reject PEP 440 dev (nightly) versions in schema 1
e891355 feat: automated nightly torch stack refresh (cu132)
19887ba review: address CodeRabbit feedback
e91fb5b Expand stable manifest: cu126/cu128/cu130/cu132, rocm7.1, xpu, cpu, mps stacks
303cc29 Require torchaudio in every stable stack; drop stable cu132 entries
4900fd1 Add torch 2.9.1 fallback to cu126, mirroring cu128
c6027e9 Add series metadata to the torch index stacks manifest
2534bd8 Pin actions/checkout to a commit SHA; drop persisted credentials where unused
f04b2f1 Run the nightly refresh on the hour (06:00 UTC)

📊 Changes

9 files changed (+1453 additions, -1 deletions)

View changed files

➕ .github/workflows/publish-torch-index-stacks.yml (+90 -0)
➕ .github/workflows/refresh-nightly-stacks.yml (+99 -0)
➕ .gitignore (+2 -0)
📝 README.md (+36 -1)
➕ scripts/nightly_policy.py (+19 -0)
➕ scripts/refresh_nightly_stacks.py (+208 -0)
➕ scripts/test_torch_index_scripts.py (+452 -0)
➕ scripts/validate_torch_index_stacks.py (+345 -0)
➕ torch-index-stacks.json (+202 -0)

📄 Description

ComfyUI Desktop 2.0 (Comfy-Org/Comfy-Desktop#1248) fetches standalone-environments/torch-index-stacks.json from R2 to populate its PyTorch picker with index-served stacks - pip-applied tuples with no bundle artifact (e.g. legacy CUDA builds for GPUs the current bundles dropped). This PR makes a checked-in file in this repo the source of truth for that manifest, so new stacks ship by PR here with no app release.

What's included

torch-index-stacks.json - initial manifest carrying the two stacks the app currently hardcodes (cu126 and cu128 for torch 2.11.0 with their compute-capability ranges), so remote and built-in start identical.
scripts/validate_torch_index_stacks.py - stdlib-only validator mirroring the app's default-deny remote-manifest validation (index-tag/accel coherence, pip-safe version strings, platform rules, duplicate stack-ID rejection, non-empty pythonAbis). The app silently drops invalid entries by design, so a typo would just make a stack vanish from the picker; this fails loudly at authoring time instead. Not a security boundary - the app stays default-deny regardless.
publish-torch-index-stacks workflow - pull requests touching the manifest run validation only; pushes to main validate then upload to R2 (same secrets as the env build workflow) with --content-type application/json and --cache-control max-age=300. The short TTL matters because unlike the immutable archives this object is replaced in-place, and stack withdrawals (explicit-empty stacks: []) must propagate quickly through the Cloudflare edge. workflow_dispatch allows manual republish. A concurrency group serializes publishes so two merges cannot race on the object.
Optional top-level series map - per-index-tag display metadata: a note explaining the backend line (noteKey for localized clients, plain note as fallback) and optional per-platform minimum NVIDIA driver versions. The app shows the note on the series dropdown of the grouped PyTorch picker and warns (informationally, never blocking) when the detected driver is below the minimum. Minimums follow NVIDIA's minor-version compatibility floors (CUDA 12.x: 527.41 win / 525.60.13 linux; 13.x: 580.88 win / 580.65.06 linux). The map is validated default-deny, and the nightly refresher round-trips the whole document so it survives automated cu132 updates.
README section documenting the flow.

Validation

The checked-in manifest passes the validator (now 12 stacks including the elaborated stable catalog and the nightly cu132 entry) and the scripts' unit tests (37).
Negative cases exercised locally: duplicate stack IDs, torch local tag / indexTag mismatch, AMD+win32 (no Windows ROCm wheels on pytorch.org in schema 1), pip-unsafe version strings, tagged MPS tuples, empty pythonAbis, unknown schemaVersion - all fail with exit 1; explicit-empty stacks: [] passes with a loud withdrawal warning.

Note for reviewers

Nothing in this repo currently syncs the GCS mirror (storage.googleapis.com/comfy-desktop-public/...) that the app falls back to when R2 is unreachable - the env build workflow only writes R2, so the mirror is synced by some outside process. That process needs to include torch-index-stacks.json, otherwise the mirror serves a stale manifest indefinitely to users who rely on the fallback.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/pull/9 **Author:** [@Kosinkadink](https://github.com/Kosinkadink) **Created:** 7/25/2026 **Status:** ✅ Merged **Merged:** 7/26/2026 **Merged by:** [@Kosinkadink](https://github.com/Kosinkadink) **Base:** `main` ← **Head:** `feat/torch-index-stacks-manifest` --- ### 📝 Commits (10+) - [`1f9294d`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/1f9294d196670fe0f3afc7af92278d355842498d) feat: add torch-index-stacks.json manifest with validated R2 publishing - [`eb7d9af`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/eb7d9afac33ff3a3283465de8b215adf74e8c0f8) Reject PEP 440 dev (nightly) versions in schema 1 - [`e891355`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/e8913555b0287ea638497ec9451794c448f11922) feat: automated nightly torch stack refresh (cu132) - [`19887ba`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/19887bade4e19e3798e2a8b436f79a4a8864525d) review: address CodeRabbit feedback - [`e91fb5b`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/e91fb5bc5ad86efba1ce5d8df548d57c07cccc09) Expand stable manifest: cu126/cu128/cu130/cu132, rocm7.1, xpu, cpu, mps stacks - [`303cc29`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/303cc29dba78a918ca2ba9319119b9b736865a14) Require torchaudio in every stable stack; drop stable cu132 entries - [`4900fd1`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/4900fd12f78f6d144c90e55bf2ec73277e3a1a89) Add torch 2.9.1 fallback to cu126, mirroring cu128 - [`c6027e9`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/c6027e9b24c46c2aa66c94fe754e3ef4cfa813c8) Add series metadata to the torch index stacks manifest - [`2534bd8`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/2534bd852a475f5beacd35942ea12a24bf49740a) Pin actions/checkout to a commit SHA; drop persisted credentials where unused - [`f04b2f1`](https://github.com/Comfy-Org/ComfyUI-Standalone-Environments/commit/f04b2f1ce1bdc1255ed750553110b3fe079caf7f) Run the nightly refresh on the hour (06:00 UTC) ### 📊 Changes **9 files changed** (+1453 additions, -1 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/publish-torch-index-stacks.yml` (+90 -0) ➕ `.github/workflows/refresh-nightly-stacks.yml` (+99 -0) ➕ `.gitignore` (+2 -0) 📝 `README.md` (+36 -1) ➕ `scripts/nightly_policy.py` (+19 -0) ➕ `scripts/refresh_nightly_stacks.py` (+208 -0) ➕ `scripts/test_torch_index_scripts.py` (+452 -0) ➕ `scripts/validate_torch_index_stacks.py` (+345 -0) ➕ `torch-index-stacks.json` (+202 -0) </details> ### 📄 Description ComfyUI Desktop 2.0 (Comfy-Org/Comfy-Desktop#1248) fetches `standalone-environments/torch-index-stacks.json` from R2 to populate its PyTorch picker with index-served stacks - pip-applied tuples with no bundle artifact (e.g. legacy CUDA builds for GPUs the current bundles dropped). This PR makes a checked-in file in this repo the source of truth for that manifest, so new stacks ship by PR here with no app release. ## What's included - **`torch-index-stacks.json`** - initial manifest carrying the two stacks the app currently hardcodes (cu126 and cu128 for torch 2.11.0 with their compute-capability ranges), so remote and built-in start identical. - **`scripts/validate_torch_index_stacks.py`** - stdlib-only validator mirroring the app's default-deny remote-manifest validation (index-tag/accel coherence, pip-safe version strings, platform rules, duplicate stack-ID rejection, non-empty `pythonAbis`). The app silently *drops* invalid entries by design, so a typo would just make a stack vanish from the picker; this fails loudly at authoring time instead. Not a security boundary - the app stays default-deny regardless. - **`publish-torch-index-stacks` workflow** - pull requests touching the manifest run validation only; pushes to `main` validate then upload to R2 (same secrets as the env build workflow) with `--content-type application/json` and `--cache-control max-age=300`. The short TTL matters because unlike the immutable archives this object is replaced in-place, and stack *withdrawals* (explicit-empty `stacks: []`) must propagate quickly through the Cloudflare edge. `workflow_dispatch` allows manual republish. A concurrency group serializes publishes so two merges cannot race on the object. - **Optional top-level `series` map** - per-index-tag display metadata: a note explaining the backend line (`noteKey` for localized clients, plain `note` as fallback) and optional per-platform minimum NVIDIA driver versions. The app shows the note on the series dropdown of the grouped PyTorch picker and warns (informationally, never blocking) when the detected driver is below the minimum. Minimums follow NVIDIA's minor-version compatibility floors (CUDA 12.x: 527.41 win / 525.60.13 linux; 13.x: 580.88 win / 580.65.06 linux). The map is validated default-deny, and the nightly refresher round-trips the whole document so it survives automated cu132 updates. - README section documenting the flow. ## Validation - The checked-in manifest passes the validator (now 12 stacks including the elaborated stable catalog and the nightly cu132 entry) and the scripts' unit tests (37). - Negative cases exercised locally: duplicate stack IDs, torch local tag / indexTag mismatch, AMD+win32 (no Windows ROCm wheels on pytorch.org in schema 1), pip-unsafe version strings, tagged MPS tuples, empty `pythonAbis`, unknown `schemaVersion` - all fail with exit 1; explicit-empty `stacks: []` passes with a loud withdrawal warning. ## Note for reviewers Nothing in this repo currently syncs the GCS mirror (`storage.googleapis.com/comfy-desktop-public/...`) that the app falls back to when R2 is unreachable - the env build workflow only writes R2, so the mirror is synced by some outside process. That process needs to include `torch-index-stacks.json`, otherwise the mirror serves a stale manifest indefinitely to users who rely on the fallback. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:22:56 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Comfy-Org/ComfyUI-Standalone-Environments#14

No description provided.

Rows
Columns