[PR #1915] [MERGED] [feat] Implement comprehensive batch tracking and OpenAPI-driven data models #2147

New issue

Closed

opened 2026-10-28 21:07:28 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-28 21:07:28 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/ComfyUI-Manager/pull/1915
Author: @christian-byrne
Created: 6/8/2025
Status: ✅ Merged
Merged: 6/22/2025
Merged by: @christian-byrne

Base: draft-v4 ← Head: feat/implement-batch-tracking-clean

📝 Commits (10+)

601f1bf [feat] Add client_id support to task queue system
c8882dc [feat] Implement comprehensive batch tracking and OpenAPI-driven data models
7f1ebbe [cleanup] Remove completed TODO comments and fix ruff issues
884b503 [feat] Add comprehensive Pydantic validation to all API endpoints
3afafdb remove dist dir
0be0a2e migrate to data models for all routes
b55e140 return installed pack list on status update
b73dc61 refresh cache before reporting status
4d89c69 add installed packs to openapi
98af8dc add claude memory

📊 Changes

15 files changed (+3760 additions, -1841 deletions)

View changed files

➕ .github/workflows/ci.yml (+70 -0)
📝 .gitignore (+1 -0)
➕ comfyui_manager/data_models/README.md (+67 -0)
➕ comfyui_manager/data_models/__init__.py (+125 -0)
➕ comfyui_manager/data_models/generated_models.py (+537 -0)
➕ comfyui_manager/glob/CLAUDE.md (+11 -0)
➕ comfyui_manager/glob/constants.py (+54 -0)
📝 comfyui_manager/glob/manager_server.py (+1411 -1348)
➕ comfyui_manager/glob/utils/__init__.py (+0 -0)
➕ comfyui_manager/glob/utils/environment_utils.py (+142 -0)
➕ comfyui_manager/glob/utils/formatting_utils.py (+60 -0)
➕ comfyui_manager/glob/utils/model_utils.py (+161 -0)
➕ comfyui_manager/glob/utils/node_pack_utils.py (+65 -0)
➕ comfyui_manager/glob/utils/security_utils.py (+54 -0)
📝 openapi.yaml (+1002 -493)

📄 Description

Four main goals of this PR:

Improved Task Tracking

Fixed inconsistencies in the progress tracker dialog during installations. Previously, task state wasn't tracked server-side, forcing the frontend to manually track queue state.

Tasks now have UUIDs and client IDs for proper identification
Maintain task history and queue server-side with thread-safe operations
WebSocket notifications when tasks start/complete keep clients synchronized
Failed tasks are tracked without interrupting the queue
Tasks can be cancelled by ID
Frontend receives consistent stdout logs for each task (previously buggy due to manual tracking)

Remove Legacy Features

Cleaning up features being replaced by other projects (native model management, subgraphs)
Removed unsafe routes like git URL downloads and pip installs - as part of core HTTP API, only registry-validated (security scanned) packages should be downloadable
Legacy features still available in legacy sub-package, accessible via --enable_manager_legacy_ui CLI flag

Type-Safe API-Driven Development

Generated OpenAPI specification for Manager HTTP API
Both frontend and backend generate types/SDKs from the same spec, enforcing contracts
Frontend uses TypeScript checking, backend uses Pydantic validation
OpenAPI validated against actual server in CI/CD
Auto-generated documentation

Set Groundwork for Restore Points

Captures environment state before/after task batches (ComfyUI version, frontend version, custom nodes, python environment)
State schemas defined in OpenAPI with validation
HTTP routes for accessing states (later, migration will be possible as part of snapshot/restore feature)

Additional refactoring includes splitting very large files out into more modules and applying ruff/pylint.

Test plan

Test task queue operations with client_id tracking
Verify batch history files are created correctly
Test OpenAPI model generation with datamodel-codegen
Verify all REST endpoints work with new models
Test queue status filtering by client_id
Verify tasks are partitioned by failed and succeeded

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/ComfyUI-Manager/pull/1915 **Author:** [@christian-byrne](https://github.com/christian-byrne) **Created:** 6/8/2025 **Status:** ✅ Merged **Merged:** 6/22/2025 **Merged by:** [@christian-byrne](https://github.com/christian-byrne) **Base:** `draft-v4` ← **Head:** `feat/implement-batch-tracking-clean` --- ### 📝 Commits (10+) - [`601f1bf`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/601f1bf452514f0cbefb6d43d3d81f03b6d866fc) [feat] Add client_id support to task queue system - [`c8882dc`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/c8882dcb7c439af0c2130722cbffd03be1c1d972) [feat] Implement comprehensive batch tracking and OpenAPI-driven data models - [`7f1ebbe`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/7f1ebbe0814da3f4bee3d07dd1bdfcfb271fda19) [cleanup] Remove completed TODO comments and fix ruff issues - [`884b503`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/884b5037280cbab0a08d333ac1614e29a3cbd855) [feat] Add comprehensive Pydantic validation to all API endpoints - [`3afafdb`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/3afafdb8843ff50c15528715e358e8bfb3601bdb) remove dist dir - [`0be0a2e`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/0be0a2e6d74c4daf26d30ef1550d609b57512962) migrate to data models for all routes - [`b55e140`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/b55e1404b19132c71705dc03da8c7c5a0b1299f5) return installed pack list on status update - [`b73dc61`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/b73dc6121f1c5f38973b7707a9f097b3d1c909f2) refresh cache before reporting status - [`4d89c69`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/4d89c6910947dc8aa5bc0c3169cfa7df4f367ad8) add installed packs to openapi - [`98af8dc`](https://github.com/Comfy-Org/ComfyUI-Manager/commit/98af8dc849e5b581ea29accffe23fa90b1270566) add claude memory ### 📊 Changes **15 files changed** (+3760 additions, -1841 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/ci.yml` (+70 -0) 📝 `.gitignore` (+1 -0) ➕ `comfyui_manager/data_models/README.md` (+67 -0) ➕ `comfyui_manager/data_models/__init__.py` (+125 -0) ➕ `comfyui_manager/data_models/generated_models.py` (+537 -0) ➕ `comfyui_manager/glob/CLAUDE.md` (+11 -0) ➕ `comfyui_manager/glob/constants.py` (+54 -0) 📝 `comfyui_manager/glob/manager_server.py` (+1411 -1348) ➕ `comfyui_manager/glob/utils/__init__.py` (+0 -0) ➕ `comfyui_manager/glob/utils/environment_utils.py` (+142 -0) ➕ `comfyui_manager/glob/utils/formatting_utils.py` (+60 -0) ➕ `comfyui_manager/glob/utils/model_utils.py` (+161 -0) ➕ `comfyui_manager/glob/utils/node_pack_utils.py` (+65 -0) ➕ `comfyui_manager/glob/utils/security_utils.py` (+54 -0) 📝 `openapi.yaml` (+1002 -493) </details> ### 📄 Description Four main goals of this PR: ### Improved Task Tracking Fixed inconsistencies in the progress tracker dialog during installations. Previously, task state wasn't tracked server-side, forcing the frontend to manually track queue state. - Tasks now have UUIDs and client IDs for proper identification - Maintain task history and queue server-side with thread-safe operations - WebSocket notifications when tasks start/complete keep clients synchronized - Failed tasks are tracked without interrupting the queue - Tasks can be cancelled by ID - Frontend receives consistent stdout logs for each task (previously buggy due to manual tracking) ### Remove Legacy Features - Cleaning up features being replaced by other projects (native model management, subgraphs) - Removed unsafe routes like git URL downloads and pip installs - as part of core HTTP API, only registry-validated (security scanned) packages should be downloadable - Legacy features still available in `legacy` sub-package, accessible via `--enable_manager_legacy_ui` CLI flag ### Type-Safe API-Driven Development - Generated OpenAPI specification for Manager HTTP API - Both frontend and backend generate types/SDKs from the same spec, enforcing contracts - Frontend uses TypeScript checking, backend uses Pydantic validation - OpenAPI validated against actual server in CI/CD - Auto-generated documentation ### Set Groundwork for Restore Points - Captures environment state before/after task batches (ComfyUI version, frontend version, custom nodes, python environment) - State schemas defined in OpenAPI with validation - HTTP routes for accessing states (later, migration will be possible as part of snapshot/restore feature) Additional refactoring includes splitting very large files out into more modules and applying ruff/pylint. ## Test plan - [x] Test task queue operations with client_id tracking - [x] Verify batch history files are created correctly - [x] Test OpenAPI model generation with datamodel-codegen - [x] Verify all REST endpoints work with new models - [x] Test queue status filtering by client_id - [x] Verify tasks are partitioned by failed and succeeded --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-28 21:07:28 +00:00