[PR #204] [MERGED] feat(jobs): label jobs with metadata and list jobs by label #204

New issue

Closed

opened 2026-10-31 22:02:19 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 22:02:19 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-python-sdk/pull/204
Author: @vqt123
Created: 10/5/2026
Status: ✅ Merged
Merged: 10/8/2026
Merged by: @vqt123

Base: main ← Head: vinh/dplat-2463-job-metadata

📝 Commits (10+)

8008597 feat(jobs): job metadata on submit, Job.metadata, and list_jobs
c3cac1f feat(jobs): map metadata_invalid to InvalidWorkflow; JobSummary carries create/update time and deployment id
c4ecefd fix(jobs): refused metadata raises the base ComfyError; JobSummary times are datetimes
9de4aec docs(jobs): note invalid_cursor and what JobSummary.deployment_id names
35d95bd docs(jobs): Comfy Cloud refuses labels for now (metadata_not_supported)
848938c fix(jobs): read odd job metadata leniently; retry a paced 429 on each list page
969d2a7 fix(jobs): check list filters on each item; wait at least a second on a 429
b17d4ad fix(jobs): compare list filter values as the text the query sends
7cbdbbe fix(jobs): pickle a job with metadata; check list filters as the query sends them
45d610a fix(jobs): raise ComfyError for a job list item without id or status

📊 Changes

11 files changed (+1466 additions, -48 deletions)

View changed files

📝 CHANGELOG.md (+37 -0)
📝 README.md (+84 -2)
📝 src/comfy_low/__init__.py (+4 -2)
📝 src/comfy_low/transport.py (+137 -17)
📝 src/comfy_sdk/__init__.py (+2 -1)
📝 src/comfy_sdk/client.py (+183 -7)
📝 src/comfy_sdk/jobs.py (+117 -3)
📝 tests/conftest.py (+79 -6)
📝 tests/test_async.py (+2 -2)
➕ tests/test_job_metadata.py (+814 -0)
📝 tests/test_jobs.py (+7 -8)

📄 Description

What changes for a developer

Before: a job had no labels. To find "all the jobs I ran for customer acme", a developer had to keep their own list of job ids.

Now: submit takes string labels, the job returns them, and list_jobs finds jobs by them.

job = client.submit(wf, metadata={"client": "acme"})
job.metadata  # {"client": "acme"}

for summary in client.list_jobs(metadata={"client": "acme"}, limit=50):
    print(summary.id, summary.status)

submit(..., metadata=...) on Comfy and AsyncComfy sends the map as the metadata field of POST /api/v2/jobs. Without it, the request body is the same as before.
Labels work on jobs sent to a deployment (point the client at the deployment's address with COMFY_BASE_URL). Comfy Cloud refuses them for now: submit gets 422 metadata_not_supported and list_jobs gets 501 not_implemented, each raised as ComfyError with that code.
A self-hosted comfy-api-proxy does not keep labels. Its own job metadata is a single string, so it refuses a label map with 422 invalid_request. The SDK reads its string metadata as no labels, so a filtered list_jobs yields nothing there, and an unfiltered one yields the proxy's newest jobs (50 by default, up to 100 with limit), each with metadata {}, and stops there, since the proxy sends no next cursor. Its items carry created_at, not create_time, so create_time reads as None there.
Job.metadata and AsyncJob.metadata return the labels, or {} when the job has none. A metadata that is not a map of strings reads as {}, and a value that is not a string is dropped. It never raises, so an odd answer to a submit cannot hide a job the server already created.
list_jobs(metadata=None, limit=None) calls GET /api/v2/jobs with one metadata[<key>]=<value> query parameter per filter. It follows next_cursor until the last page and yields every match. limit is the page size. The SDK also checks the filters on each item and skips one whose labels do not match, comparing each key and value as the text the query sends ({"run": 7} matches the label "7", {1: "a"} the key "1"). The query and the check build their pairs with one helper, so they cannot disagree. So a host that ignores the filters yields only real matches, though one step of the iteration can then read several pages, or every page, before it yields or ends. Each page retries a 429 that carries Retry-After, the same way submit does, with its own 60-second budget.
A 429 retry on submit and list_jobs now waits at least one second, so Retry-After: 0 cannot drive a tight loop. Before, submit retried at once on Retry-After: 0. On AsyncComfy it is an async iterator.
Each list item is a new JobSummary (id, status, create_time, update_time, deployment_id, metadata, and data, the item as the server sent it). create_time and update_time are datetime values. The fields match the TypeScript SDK's list item. A list item has no outputs or links, so it is not a Job handle; client.jobs.get(summary.id) returns the full job.
The SDK does not check the label limits. The server checks them. Its 422 metadata_invalid raises ComfyError with that code and the server's message. The message names the key when one key or value breaks a rule, and gives the count when there are more than 16 pairs. A bad filter raises ComfyError with code invalid_metadata_filter, and a cursor the server did not issue raises ComfyError with code invalid_cursor. deployment_id is the deployment copy that ran the job, so after a deployment update it can differ from the deployment's current id.

Generated code is untouched

spec/openapi.yaml and src/comfy_low/models/_generated.py do not have these fields yet, and this PR does not edit them. The transport is hand-written, so the change lives there:

The transport validates job responses into a small subclass of the generated Job with a lenient metadata field. It overrides the field rather than adding one, so it keeps reading leniently after the spec sync adds a strict metadata to Job. A test simulates that strict field.
list_jobs on the transport returns each page as a dict. It is not in OPERATION_IDS yet. When the spec sync adds the list operation, tests/test_spec_coverage.py will ask for it to be added there, mapped to list_jobs.

Depends on

The server-side job-labels change, which adds job metadata and the job list filters. It is in production.

Tests

New file tests/test_job_metadata.py, against the existing stub server in tests/conftest.py (extended to echo metadata or serve any given shape of it, serve GET /api/v2/jobs in pages with real JobStatus values, answer chosen list requests 429, and record each list path and query):

submit with metadata puts it in the body beside workflow, and the returned job exposes it (sync and async).
submit with no metadata, or with {}, sends exactly {"workflow": ...} (sync and async).
A fetched job exposes its labels; a job without labels gives {}; the returned dict is a copy; refresh() keeps the labels (async).
list_jobs sends metadata[client]=acme, metadata[run]=... and limit=2, follows the cursor across three pages, yields all five jobs in order, and makes no request after the last page (sync). The async client follows two pages with the same filters.
list_jobs() with no arguments requests the bare /api/v2/jobs, not /jobs?.
cancel() keeps metadata on the returned job (sync and async).
A job whose metadata is {"n": 1}, {"n": null, ...}, the string "batch-7" or a list reads as labels on submit, jobs.get, refresh, wait and cancel (sync and async), and on a list item. A Job with a strict metadata: dict[str, str] field, as the spec sync will add, still reads them.
A label keeps its key whatever it is: __proto__, constructor, an empty key, punctuation, non-ASCII.
A filtered list skips items whose labels do not match, including one whose metadata is a string, and keeps paging (sync and async). metadata={"run": 7} yields the job labelled run "7". On a proxy-shaped list a filtered call yields nothing.
Retry-After: 0 on a list page waits 1 second (sync and async); the existing submit tests for Retry-After: 0 and -5 now expect 1 second instead of 0.
hash(summary) works. A job model from post, get or cancel pickles and unpickles with its metadata (sync and async).
Filters {1: "a"} and {"1": b"a"} send metadata[1]=a and keep the job labelled "1": "a".
A list page still answered 429 after its retry budget raises RateLimited; a list 429 without Retry-After raises at once with no wait. A page with no jobs key yields nothing, and paging continues past it (sync and async).
A 429 with Retry-After: 3 on the first request for each page is retried after 3 seconds with the same query (sync and async). A 501 not_implemented raises ComfyError with that code and is not retried.
A JobSummary keeps id, status, create_time, update_time, deployment_id and the raw item; missing optional fields read as None and missing labels as {}. Nanosecond times parse; an unreadable time reads as None and stays in data.
A 422 metadata_invalid raises ComfyError (not InvalidWorkflow) with that code, status 422 and the server's real message (the key for a bad key, the count for 17 pairs); the submit is not retried (sync and async).
A 400 invalid_metadata_filter from the list raises ComfyError with that code (sync and async); a 400 invalid_cursor does the same.
Printing a JobSummary shows its id and status but not data, the raw item; equality and the id-only hash are unchanged.
A list item whose id or status is missing or null, or that is not a JSON object, raises ComfyError with code invalid_response naming the problem, and is not skipped; so does a page that is not an object, a page whose jobs is not an array (a missing or null jobs reads as an empty page), an id, status or deployment_id of the wrong type, and a next_cursor that is not a string (0, false and [] included) or repeats one already sent; a null or empty next_cursor ends the list (sync and async).
A 422 metadata_not_supported (Comfy Cloud today) raises ComfyError with that code, not InvalidWorkflow, and is not retried.

Run locally, each with its own exit status: ruff check . passed, ruff format --check . passed, mypy src passed, pytest 1236 passed and 9 skipped (the skips are the live-server tests), scripts/check_drift.py passed, scripts/check_public_repo_hygiene.py passed.

Not run: this SDK against a live server. The server-side change passed its own test on a staging deployment, covering the same request and response shapes.

Risk: low. It was rated high only for the release order, which is now met: the server-side job-labels change is in production. The code itself is low risk: Every new argument is optional and defaults to the old request. The generated models are not edited. Job responses now go through a subclass that adds one optional field, so a response without metadata reads exactly as before. One existing behaviour changes: submit waits at least 1 s before re-sending after a 429, even when Retry-After is 0.

Summary by CodeRabbit

New Features
- Submit jobs with optional metadata labels and view normalized metadata on job details.
- List jobs with metadata filters and pagination in synchronous and asynchronous clients. Results include job status, timestamps, and deployment IDs.
Improvements
- Eligible rate-limit responses during job submission and listing are retried with a minimum one-second delay, subject to the retry time limit.
Documentation
- Added guidance on metadata and job listing, including platform-specific availability and behavior.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-python-sdk/pull/204 **Author:** [@vqt123](https://github.com/vqt123) **Created:** 10/5/2026 **Status:** ✅ Merged **Merged:** 10/8/2026 **Merged by:** [@vqt123](https://github.com/vqt123) **Base:** `main` ← **Head:** `vinh/dplat-2463-job-metadata` --- ### 📝 Commits (10+) - [`8008597`](https://github.com/Comfy-Org/comfy-python-sdk/commit/80085973309582dcf7f5e3f6ebe34c1df2c57bda) feat(jobs): job metadata on submit, Job.metadata, and list_jobs - [`c3cac1f`](https://github.com/Comfy-Org/comfy-python-sdk/commit/c3cac1fea8682651c639f653b2bc7ff3b446ec68) feat(jobs): map metadata_invalid to InvalidWorkflow; JobSummary carries create/update time and deployment id - [`c4ecefd`](https://github.com/Comfy-Org/comfy-python-sdk/commit/c4ecefd29ed2aa6d8a8b15f4fe3d97035f7f0ac6) fix(jobs): refused metadata raises the base ComfyError; JobSummary times are datetimes - [`9de4aec`](https://github.com/Comfy-Org/comfy-python-sdk/commit/9de4aec311375f5d24a97fa5db50e5556bfe8e80) docs(jobs): note invalid_cursor and what JobSummary.deployment_id names - [`35d95bd`](https://github.com/Comfy-Org/comfy-python-sdk/commit/35d95bd0174b6872833d4117e4c9c3824daf8d8d) docs(jobs): Comfy Cloud refuses labels for now (metadata_not_supported) - [`848938c`](https://github.com/Comfy-Org/comfy-python-sdk/commit/848938c52605f513db0c262f276ba766b85f8aa1) fix(jobs): read odd job metadata leniently; retry a paced 429 on each list page - [`969d2a7`](https://github.com/Comfy-Org/comfy-python-sdk/commit/969d2a761790d2e98f60d6cf4d6b4fc35f8798b5) fix(jobs): check list filters on each item; wait at least a second on a 429 - [`b17d4ad`](https://github.com/Comfy-Org/comfy-python-sdk/commit/b17d4ad1124087f64c5e61564132337b538a9faf) fix(jobs): compare list filter values as the text the query sends - [`7cbdbbe`](https://github.com/Comfy-Org/comfy-python-sdk/commit/7cbdbbec6868b5b88f2e14c46f7fcdb57c1003d5) fix(jobs): pickle a job with metadata; check list filters as the query sends them - [`45d610a`](https://github.com/Comfy-Org/comfy-python-sdk/commit/45d610a2d0bb1a5b68d64e35d7dbcc3f29d21f0a) fix(jobs): raise ComfyError for a job list item without id or status ### 📊 Changes **11 files changed** (+1466 additions, -48 deletions) <details> <summary>View changed files</summary> 📝 `CHANGELOG.md` (+37 -0) 📝 `README.md` (+84 -2) 📝 `src/comfy_low/__init__.py` (+4 -2) 📝 `src/comfy_low/transport.py` (+137 -17) 📝 `src/comfy_sdk/__init__.py` (+2 -1) 📝 `src/comfy_sdk/client.py` (+183 -7) 📝 `src/comfy_sdk/jobs.py` (+117 -3) 📝 `tests/conftest.py` (+79 -6) 📝 `tests/test_async.py` (+2 -2) ➕ `tests/test_job_metadata.py` (+814 -0) 📝 `tests/test_jobs.py` (+7 -8) </details> ### 📄 Description ## What changes for a developer Before: a job had no labels. To find "all the jobs I ran for customer acme", a developer had to keep their own list of job ids. Now: `submit` takes string labels, the job returns them, and `list_jobs` finds jobs by them. ```python job = client.submit(wf, metadata={"client": "acme"}) job.metadata # {"client": "acme"} for summary in client.list_jobs(metadata={"client": "acme"}, limit=50): print(summary.id, summary.status) ``` - `submit(..., metadata=...)` on `Comfy` and `AsyncComfy` sends the map as the `metadata` field of `POST /api/v2/jobs`. Without it, the request body is the same as before. - Labels work on jobs sent to a deployment (point the client at the deployment's address with `COMFY_BASE_URL`). Comfy Cloud refuses them for now: `submit` gets 422 `metadata_not_supported` and `list_jobs` gets 501 `not_implemented`, each raised as `ComfyError` with that code. - A self-hosted comfy-api-proxy does not keep labels. Its own job `metadata` is a single string, so it refuses a label map with 422 `invalid_request`. The SDK reads its string `metadata` as no labels, so a filtered `list_jobs` yields nothing there, and an unfiltered one yields the proxy's newest jobs (50 by default, up to 100 with `limit`), each with `metadata` `{}`, and stops there, since the proxy sends no next cursor. Its items carry `created_at`, not `create_time`, so `create_time` reads as `None` there. - `Job.metadata` and `AsyncJob.metadata` return the labels, or `{}` when the job has none. A `metadata` that is not a map of strings reads as `{}`, and a value that is not a string is dropped. It never raises, so an odd answer to a submit cannot hide a job the server already created. - `list_jobs(metadata=None, limit=None)` calls `GET /api/v2/jobs` with one `metadata[<key>]=<value>` query parameter per filter. It follows `next_cursor` until the last page and yields every match. `limit` is the page size. The SDK also checks the filters on each item and skips one whose labels do not match, comparing each key and value as the text the query sends (`{"run": 7}` matches the label "7", `{1: "a"}` the key "1"). The query and the check build their pairs with one helper, so they cannot disagree. So a host that ignores the filters yields only real matches, though one step of the iteration can then read several pages, or every page, before it yields or ends. Each page retries a 429 that carries `Retry-After`, the same way `submit` does, with its own 60-second budget. - A 429 retry on `submit` and `list_jobs` now waits at least one second, so `Retry-After: 0` cannot drive a tight loop. Before, `submit` retried at once on `Retry-After: 0`. On `AsyncComfy` it is an async iterator. - Each list item is a new `JobSummary` (`id`, `status`, `create_time`, `update_time`, `deployment_id`, `metadata`, and `data`, the item as the server sent it). `create_time` and `update_time` are `datetime` values. The fields match the TypeScript SDK's list item. A list item has no outputs or links, so it is not a `Job` handle; `client.jobs.get(summary.id)` returns the full job. - The SDK does not check the label limits. The server checks them. Its 422 `metadata_invalid` raises `ComfyError` with that code and the server's message. The message names the key when one key or value breaks a rule, and gives the count when there are more than 16 pairs. A bad filter raises `ComfyError` with code `invalid_metadata_filter`, and a cursor the server did not issue raises `ComfyError` with code `invalid_cursor`. `deployment_id` is the deployment copy that ran the job, so after a deployment update it can differ from the deployment's current id. ## Generated code is untouched `spec/openapi.yaml` and `src/comfy_low/models/_generated.py` do not have these fields yet, and this PR does not edit them. The transport is hand-written, so the change lives there: - The transport validates job responses into a small subclass of the generated `Job` with a lenient `metadata` field. It overrides the field rather than adding one, so it keeps reading leniently after the spec sync adds a strict `metadata` to `Job`. A test simulates that strict field. - `list_jobs` on the transport returns each page as a dict. It is not in `OPERATION_IDS` yet. When the spec sync adds the list operation, `tests/test_spec_coverage.py` will ask for it to be added there, mapped to `list_jobs`. ## Depends on The server-side job-labels change, which adds job metadata and the job list filters. It is in production. ## Tests New file `tests/test_job_metadata.py`, against the existing stub server in `tests/conftest.py` (extended to echo `metadata` or serve any given shape of it, serve `GET /api/v2/jobs` in pages with real `JobStatus` values, answer chosen list requests 429, and record each list path and query): - `submit` with `metadata` puts it in the body beside `workflow`, and the returned job exposes it (sync and async). - `submit` with no `metadata`, or with `{}`, sends exactly `{"workflow": ...}` (sync and async). - A fetched job exposes its labels; a job without labels gives `{}`; the returned dict is a copy; `refresh()` keeps the labels (async). - `list_jobs` sends `metadata[client]=acme`, `metadata[run]=...` and `limit=2`, follows the cursor across three pages, yields all five jobs in order, and makes no request after the last page (sync). The async client follows two pages with the same filters. - `list_jobs()` with no arguments requests the bare `/api/v2/jobs`, not `/jobs?`. - `cancel()` keeps `metadata` on the returned job (sync and async). - A job whose `metadata` is `{"n": 1}`, `{"n": null, ...}`, the string `"batch-7"` or a list reads as labels on submit, `jobs.get`, `refresh`, `wait` and `cancel` (sync and async), and on a list item. A `Job` with a strict `metadata: dict[str, str]` field, as the spec sync will add, still reads them. - A label keeps its key whatever it is: `__proto__`, `constructor`, an empty key, punctuation, non-ASCII. - A filtered list skips items whose labels do not match, including one whose `metadata` is a string, and keeps paging (sync and async). `metadata={"run": 7}` yields the job labelled run "7". On a proxy-shaped list a filtered call yields nothing. - `Retry-After: 0` on a list page waits 1 second (sync and async); the existing submit tests for `Retry-After: 0` and `-5` now expect 1 second instead of 0. - `hash(summary)` works. A job model from post, get or cancel pickles and unpickles with its `metadata` (sync and async). - Filters `{1: "a"}` and `{"1": b"a"}` send `metadata[1]=a` and keep the job labelled "1": "a". - A list page still answered 429 after its retry budget raises `RateLimited`; a list 429 without `Retry-After` raises at once with no wait. A page with no `jobs` key yields nothing, and paging continues past it (sync and async). - A 429 with `Retry-After: 3` on the first request for each page is retried after 3 seconds with the same query (sync and async). A 501 `not_implemented` raises `ComfyError` with that code and is not retried. - A `JobSummary` keeps `id`, `status`, `create_time`, `update_time`, `deployment_id` and the raw item; missing optional fields read as `None` and missing labels as `{}`. Nanosecond times parse; an unreadable time reads as `None` and stays in `data`. - A 422 `metadata_invalid` raises `ComfyError` (not `InvalidWorkflow`) with that code, status 422 and the server's real message (the key for a bad key, the count for 17 pairs); the submit is not retried (sync and async). - A 400 `invalid_metadata_filter` from the list raises `ComfyError` with that code (sync and async); a 400 `invalid_cursor` does the same. - Printing a `JobSummary` shows its id and status but not `data`, the raw item; equality and the id-only hash are unchanged. - A list item whose `id` or `status` is missing or `null`, or that is not a JSON object, raises `ComfyError` with code `invalid_response` naming the problem, and is not skipped; so does a page that is not an object, a page whose `jobs` is not an array (a missing or `null` `jobs` reads as an empty page), an `id`, `status` or `deployment_id` of the wrong type, and a `next_cursor` that is not a string (`0`, `false` and `[]` included) or repeats one already sent; a `null` or empty `next_cursor` ends the list (sync and async). - A 422 `metadata_not_supported` (Comfy Cloud today) raises `ComfyError` with that code, not `InvalidWorkflow`, and is not retried. Run locally, each with its own exit status: `ruff check .` passed, `ruff format --check .` passed, `mypy src` passed, `pytest` 1236 passed and 9 skipped (the skips are the live-server tests), `scripts/check_drift.py` passed, `scripts/check_public_repo_hygiene.py` passed. Not run: this SDK against a live server. The server-side change passed its own test on a staging deployment, covering the same request and response shapes. **Risk: low.** It was rated high only for the release order, which is now met: the server-side job-labels change is in production. The code itself is low risk: Every new argument is optional and defaults to the old request. The generated models are not edited. Job responses now go through a subclass that adds one optional field, so a response without `metadata` reads exactly as before. One existing behaviour changes: `submit` waits at least 1 s before re-sending after a 429, even when `Retry-After` is 0.  ## Summary by CodeRabbit - **New Features** - Submit jobs with optional metadata labels and view normalized metadata on job details. - List jobs with metadata filters and pagination in synchronous and asynchronous clients. Results include job status, timestamps, and deployment IDs. - **Improvements** - Eligible rate-limit responses during job submission and listing are retried with a minimum one-second delay, subject to the retry time limit. - **Documentation** - Added guidance on metadata and job listing, including platform-specific availability and behavior.  --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 22:02:19 +00:00