fallback_provider to models.run #159

New issue

Closed

opened 2026-10-31 22:01:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 22:01:54 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-python-sdk/pull/159
Author: @deepme987
Created: 9/16/2026
Status: ✅ Merged
Merged: 9/18/2026
Merged by: @mattmillerai

Base: main ← Head: deepme987/router-provider-params

📝 Commits (7)

e108e96 chore: sync vendored Comfy Router spec from cloud@996a63c
2af9be2 feat(models): add model_provider, strict_mode, fallback_provider to models.run
a60de73 test(integration): add Router model-run e2e for the alt-provider controls
0438c1f fix(models): normalise a bool fallback_provider to the spec's true/false
ea9b995 feat(models): surface Router's serving-provider and dropped-params disclosure
3dde973 test(integration): require an explicit opt-in before the e2e tests bill
e273132 docs(models): resend the alt-provider controls with a recovery idempotency key

📊 Changes

9 files changed (+1185 additions, -70 deletions)

View changed files

📝 spec/router-openapi.yaml (+398 -35)
📝 src/comfy_low/transport.py (+93 -7)
📝 src/comfy_sdk/__init__.py (+4 -0)
📝 src/comfy_sdk/models.py (+262 -6)
📝 src/comfy_sdk/router_exceptions.py (+80 -9)
➕ tests/integration/test_router_models_e2e.py (+152 -0)
📝 tests/test_models_run.py (+150 -0)
📝 tests/test_models_run_retry.py (+33 -9)
📝 tests/test_router_exceptions.py (+13 -4)

📄 Description

What

Adds Comfy Router's alt-provider controls to client.models.run (sync + async):

model_provider — select an alternate serving provider for a model (e.g. "fal"); omitted runs the model's default provider.
strict_mode — False (default) sends this model's native input and Router translates to/from the provider's schema; True sends and returns the provider's own raw shape unchanged.
fallback_provider — pass "false" to opt out of the retry-against-another-provider behavior.

Each is a query param on the run route, built by _router_run_query and sent only when set, so a call that names none is byte-for-byte the request this route has always made.

Also reconciles the vendored Router spec sync (supersedes #156, which is spec-only): reblesses the 8 error buckets whose meaning moved in cloud@996a63c, and types the 3 the spec now declares (cancelled, queue_timeout, request_not_found). The first two leave DEFERRED_ERROR_TYPES now that the contract carries them (file_download_error stays deferred, still not in the spec).

Scope

These controls are on the synchronous run route only — the backend accepts them on runRouterModel alone, not on the queue routes (submit/subscribe), confirmed in the vendored spec and cloud's router_queue.go. Queue-surface support needs a cloud change first.

Testing

Unit: uv run pytest — 884 passed, 4 skipped.
Live against staging (COMFY_ROUTER_BASE_URL=https://stagingapi.comfy.org, model_provider=fal), all passing:
- gpt-image-2 (direct) sync + async, native round-trip
- vertexai/gemini-3.1-flash-image (nano-banana-2, Gemini translator, direct)
- byteplus/dreamina-seedance-2-0-260128 (seedance-2.0, BytePlus translator, submit-poll video) → real video_url
- strict_mode=true → fal's raw {"images":[...]} passthrough
- fallback_provider="false" accepted

Companion PRs

Same feature across the other SDKs:

comfy-typescript-sdk: (to follow)
comfy-swift-sdk: (to follow)
comfy-go-sdk: (to follow)

Summary by CodeRabbit

New Features
- Added queued model execution with request submission, status polling, result retrieval, cancellation, queue positions, and polling guidance.
- Added alternate-provider selection, optional fallback providers, and strict mode for synchronous and asynchronous model runs.
- Added reporting for translated requests, dropped parameters, fallback providers, and queue outcomes.
- Added specific errors for cancelled requests, queue timeouts, and missing requests.
Documentation
- Expanded API guidance for model schemas, native payloads, billing, pagination, validation, authentication, idempotency, and response headers.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-python-sdk/pull/159 **Author:** [@deepme987](https://github.com/deepme987) **Created:** 9/16/2026 **Status:** ✅ Merged **Merged:** 9/18/2026 **Merged by:** [@mattmillerai](https://github.com/mattmillerai) **Base:** `main` ← **Head:** `deepme987/router-provider-params` --- ### 📝 Commits (7) - [`e108e96`](https://github.com/Comfy-Org/comfy-python-sdk/commit/e108e966919677230ff2fe40458a379f9f880898) chore: sync vendored Comfy Router spec from cloud@996a63c - [`2af9be2`](https://github.com/Comfy-Org/comfy-python-sdk/commit/2af9be2a8759e63b4af9ae53dbf9dce12003ee89) feat(models): add model_provider, strict_mode, fallback_provider to models.run - [`a60de73`](https://github.com/Comfy-Org/comfy-python-sdk/commit/a60de73a061b108e227480c53a38bf32d48b96c3) test(integration): add Router model-run e2e for the alt-provider controls - [`0438c1f`](https://github.com/Comfy-Org/comfy-python-sdk/commit/0438c1fe7706357795b8e7e60dacab15cdd858fb) fix(models): normalise a bool fallback_provider to the spec's true/false - [`ea9b995`](https://github.com/Comfy-Org/comfy-python-sdk/commit/ea9b995e95a42821b4de20cb42066fe4a4dd247c) feat(models): surface Router's serving-provider and dropped-params disclosure - [`3dde973`](https://github.com/Comfy-Org/comfy-python-sdk/commit/3dde973dd4287f2955e5266b6e71c5955738b7cb) test(integration): require an explicit opt-in before the e2e tests bill - [`e273132`](https://github.com/Comfy-Org/comfy-python-sdk/commit/e273132e54b4fed8e140905b4856e688490aeed3) docs(models): resend the alt-provider controls with a recovery idempotency key ### 📊 Changes **9 files changed** (+1185 additions, -70 deletions) <details> <summary>View changed files</summary> 📝 `spec/router-openapi.yaml` (+398 -35) 📝 `src/comfy_low/transport.py` (+93 -7) 📝 `src/comfy_sdk/__init__.py` (+4 -0) 📝 `src/comfy_sdk/models.py` (+262 -6) 📝 `src/comfy_sdk/router_exceptions.py` (+80 -9) ➕ `tests/integration/test_router_models_e2e.py` (+152 -0) 📝 `tests/test_models_run.py` (+150 -0) 📝 `tests/test_models_run_retry.py` (+33 -9) 📝 `tests/test_router_exceptions.py` (+13 -4) </details> ### 📄 Description ## What Adds Comfy Router's alt-provider controls to `client.models.run` (sync + async): - `model_provider` — select an alternate serving provider for a model (e.g. `"fal"`); omitted runs the model's default provider. - `strict_mode` — `False` (default) sends this model's native input and Router translates to/from the provider's schema; `True` sends and returns the provider's own raw shape unchanged. - `fallback_provider` — pass `"false"` to opt out of the retry-against-another-provider behavior. Each is a query param on the run route, built by `_router_run_query` and sent only when set, so a call that names none is byte-for-byte the request this route has always made. Also reconciles the vendored Router spec sync (**supersedes #156**, which is spec-only): reblesses the 8 error buckets whose `meaning` moved in `cloud@996a63c`, and types the 3 the spec now declares (`cancelled`, `queue_timeout`, `request_not_found`). The first two leave `DEFERRED_ERROR_TYPES` now that the contract carries them (`file_download_error` stays deferred, still not in the spec). ## Scope These controls are on the **synchronous run route only** — the backend accepts them on `runRouterModel` alone, not on the queue routes (`submit`/`subscribe`), confirmed in the vendored spec and cloud's `router_queue.go`. Queue-surface support needs a cloud change first. ## Testing - Unit: `uv run pytest` — **884 passed, 4 skipped**. - Live against staging (`COMFY_ROUTER_BASE_URL=https://stagingapi.comfy.org`, `model_provider=fal`), all passing: - gpt-image-2 (direct) sync + async, native round-trip - vertexai/gemini-3.1-flash-image (nano-banana-2, Gemini translator, direct) - byteplus/dreamina-seedance-2-0-260128 (seedance-2.0, BytePlus translator, submit-poll video) → real `video_url` - `strict_mode=true` → fal's raw `{"images":[...]}` passthrough - `fallback_provider="false"` accepted ## Companion PRs Same feature across the other SDKs: - comfy-typescript-sdk: _(to follow)_ - comfy-swift-sdk: _(to follow)_ - comfy-go-sdk: _(to follow)_  ## Summary by CodeRabbit - **New Features** - Added queued model execution with request submission, status polling, result retrieval, cancellation, queue positions, and polling guidance. - Added alternate-provider selection, optional fallback providers, and strict mode for synchronous and asynchronous model runs. - Added reporting for translated requests, dropped parameters, fallback providers, and queue outcomes. - Added specific errors for cancelled requests, queue timeouts, and missing requests. - **Documentation** - Expanded API guidance for model schemas, native payloads, billing, pagination, validation, authentication, idempotency, and response headers.  --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 22:01:54 +00:00