[PR #22] [MERGED] fix(encoder): recurse into nested AUTOGROW / DYNAMIC_COMBO runtime dicts #21

New issue

Closed

opened 2026-10-26 22:29:00 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 22:29:00 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/Comfy-Org/comfy-remote-nodes/pull/22
Author: @Kosinkadink
Created: 5/17/2026
Status: ✅ Merged
Merged: 5/17/2026
Merged by: @Kosinkadink

Base: main ← Head: feat/encode-nested-autogrow-dicts

📝 Commits (1)

866b7e3 fix(encoder): recurse into nested AUTOGROW / DYNAMIC_COMBO runtime dicts

📊 Changes

2 files changed (+431 additions, -7 deletions)

View changed files

➕ notes/run_encode_nested_dicts.py (+360 -0)
📝 proxy_node.py (+71 -7)

📄 Description

ComfyUI's V3 build_nested_inputs (comfy_api/latest/_io.py L1722) rebuilds AUTOGROW / DYNAMIC_COMBO runtime kwargs into nested dicts before they reach the proxy_node's execute(). For an AUTOGROW input named reference_images, the value arrives as {"image1": <tensor>, "image2": <tensor>, ...}; for a DYNAMIC_COMBO branch with a nested AUTOGROW (e.g. the new Wan2ReferenceVideoApi model DC with nested video1..video3 + image1..image5), the value arrives as {"<dc-name>": branch_key, "video_refs": {"video1": <VideoInput>}, "reference_images": {"image1": <tensor>}, ...}.

Before this change _encode_one only encoded values where the top-level value was itself a torch.Tensor / VideoInput / audio dict / model3d input. A non-audio dict (the AG / DC runtime shape) failed all four duck-type checks and was returned unchanged, so per-slot tensors / VideoInputs survived past encoding and either crashed at json.dumps time (no custom JSONEncoder for torch.Tensor) or violated the server's contract (comfy-rnp-server's _ordered_wan_videoedit_image_envelopes and the WAN2 / HappyHorse / Grok / Bria / Luma AG providers shipped over the last 5 PRs all require each slot value to already be an RNP envelope before they reach the upload step).

The same gap also defeated the externalization cap: _encode_inputs only ran maybe_externalize when the top-level encoded value was itself an envelope, so per-slot envelopes inside an AG dict would ship inline regardless of max_inline_bytes.

Fix

Structural — match the runtime data model:

_encode_one recurses into plain (non-envelope, non-audio) dict values, calling itself for each entry. Envelope dicts pass through so we don't double-encode; audio dicts still route to encode_audio_input because their dict shape (waveform + sample_rate) is a heavy leaf, not a container. The recursion passes the parent's accepted_encodings downward — single-image AG slots and VIDEO / AUDIO / MODEL_3D leaves don't depend on it, and gating per-child policy on descriptor metadata adds plumbing for no concrete provider need today.
New _externalize_nested helper walks the encoded tree the same way: envelope leaves go through serialization.maybe_externalize; plain dicts recurse; everything else passes through. _encode_inputs replaces the old "if is_envelope: externalize" guard with one call to the recursive walker, so per-slot envelopes inside AG / DC dicts get the same presigned-PUT treatment top-level envelopes get.

Lists are intentionally NOT traversed — AG / DC runtime values always materialize as dicts, never lists, and recursing into arbitrary lists would touch opaque payloads. Recursion depth is bounded by the upstream Autogrow max-slot cap (100) and the maximum DC branch nesting depth observed in any provider today (1).

Tests

New notes/run_encode_nested_dicts.py standalone smoke test exercises 9 scenarios:

Top-level AG-of-IMAGE: each image<N> tensor becomes an image envelope; wrapping dict preserved.
Top-level AG-of-VIDEO: each video<N> VideoInput becomes a video envelope.
DC-wrapping-AG: branch key untouched, nested AG dict encoded at every level.
AUDIO dict still routes to encode_audio_input (not swallowed by dict-recursion branch).
Already-encoded envelope passes through unchanged (no double-encoding).
Plain scalar config dict passes through unchanged.
_externalize_nested walks per-slot envelopes (cap=3 forces every slot to externalize via the stubbed maybe_externalize).
JSON-serializability post-condition: json.dumps(...) round-trips the full payload (regression guard for the exact bug fixed).
Leaf-path regression: single-tensor / single-video / single-audio top-level inputs still encode unchanged.

Loads _encode_one / _encode_inputs / _externalize_nested out of proxy_node.py via AST extraction so it runs without importing the full comfy_api / comfy_api_nodes packages, same pattern as run_image_max_batch.py.

Runs clean against the ComfyUI venv:

$ python notes/run_encode_nested_dicts.py
ok: top-level AG-of-IMAGE encoded per slot
ok: top-level AG-of-VIDEO encoded per slot
ok: DC-wrapping-AG encoded leaves at every level
ok: AUDIO dict still routes to encode_audio_input
ok: pre-encoded envelope passed through unchanged
ok: scalar config dict passed through unchanged
ok: nested AG envelopes externalized per slot
ok: scalar siblings preserved alongside externalized envelopes
ok: full encoded payload is JSON-serializable
ok: top-level leaf path unchanged
ALL OK

Unblocks the next batch of comfy-rnp-server providers that need AG-of-VIDEO or DC-wrapping-AG-of-VIDEO support (WanReferenceVideoApi_RNP character1..character3 and Wan2ReferenceVideoApi_RNP video1..video3 + image1..image5).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Comfy-Org/comfy-remote-nodes/pull/22 **Author:** [@Kosinkadink](https://github.com/Kosinkadink) **Created:** 5/17/2026 **Status:** ✅ Merged **Merged:** 5/17/2026 **Merged by:** [@Kosinkadink](https://github.com/Kosinkadink) **Base:** `main` ← **Head:** `feat/encode-nested-autogrow-dicts` --- ### 📝 Commits (1) - [`866b7e3`](https://github.com/Comfy-Org/comfy-remote-nodes/commit/866b7e32a3bc049d7a1aa7c1f822898184be92f4) fix(encoder): recurse into nested AUTOGROW / DYNAMIC_COMBO runtime dicts ### 📊 Changes **2 files changed** (+431 additions, -7 deletions) <details> <summary>View changed files</summary> ➕ `notes/run_encode_nested_dicts.py` (+360 -0) 📝 `proxy_node.py` (+71 -7) </details> ### 📄 Description ComfyUI's V3 `build_nested_inputs` (`comfy_api/latest/_io.py` L1722) rebuilds AUTOGROW / DYNAMIC_COMBO runtime kwargs into nested dicts before they reach the proxy_node's `execute()`. For an AUTOGROW input named `reference_images`, the value arrives as `{"image1": <tensor>, "image2": <tensor>, ...}`; for a DYNAMIC_COMBO branch with a nested AUTOGROW (e.g. the new Wan2ReferenceVideoApi `model` DC with nested `video1..video3` + `image1..image5`), the value arrives as `{"<dc-name>": branch_key, "video_refs": {"video1": <VideoInput>}, "reference_images": {"image1": <tensor>}, ...}`. Before this change `_encode_one` only encoded values where the **top-level** value was itself a `torch.Tensor` / `VideoInput` / audio dict / model3d input. A non-audio dict (the AG / DC runtime shape) failed all four duck-type checks and was returned unchanged, so per-slot tensors / VideoInputs survived past encoding and either crashed at `json.dumps` time (no custom JSONEncoder for `torch.Tensor`) or violated the server's contract (`comfy-rnp-server`'s `_ordered_wan_videoedit_image_envelopes` and the WAN2 / HappyHorse / Grok / Bria / Luma AG providers shipped over the last 5 PRs all require each slot value to already be an RNP envelope before they reach the upload step). The same gap also defeated the externalization cap: `_encode_inputs` only ran `maybe_externalize` when the *top-level* encoded value was itself an envelope, so per-slot envelopes inside an AG dict would ship inline regardless of `max_inline_bytes`. ## Fix Structural — match the runtime data model: 1. `_encode_one` recurses into plain (non-envelope, non-audio) dict values, calling itself for each entry. Envelope dicts pass through so we don't double-encode; audio dicts still route to `encode_audio_input` because their dict shape (`waveform` + `sample_rate`) is a heavy leaf, not a container. The recursion passes the parent's `accepted_encodings` downward — single-image AG slots and VIDEO / AUDIO / MODEL_3D leaves don't depend on it, and gating per-child policy on descriptor metadata adds plumbing for no concrete provider need today. 2. New `_externalize_nested` helper walks the encoded tree the same way: envelope leaves go through `serialization.maybe_externalize`; plain dicts recurse; everything else passes through. `_encode_inputs` replaces the old "if `is_envelope`: externalize" guard with one call to the recursive walker, so per-slot envelopes inside AG / DC dicts get the same presigned-PUT treatment top-level envelopes get. Lists are intentionally NOT traversed — AG / DC runtime values always materialize as dicts, never lists, and recursing into arbitrary lists would touch opaque payloads. Recursion depth is bounded by the upstream Autogrow max-slot cap (100) and the maximum DC branch nesting depth observed in any provider today (1). ## Tests New `notes/run_encode_nested_dicts.py` standalone smoke test exercises 9 scenarios: 1. Top-level AG-of-IMAGE: each `image<N>` tensor becomes an image envelope; wrapping dict preserved. 2. Top-level AG-of-VIDEO: each `video<N>` `VideoInput` becomes a video envelope. 3. DC-wrapping-AG: branch key untouched, nested AG dict encoded at every level. 4. AUDIO dict still routes to `encode_audio_input` (not swallowed by dict-recursion branch). 5. Already-encoded envelope passes through unchanged (no double-encoding). 6. Plain scalar config dict passes through unchanged. 7. `_externalize_nested` walks per-slot envelopes (cap=3 forces every slot to externalize via the stubbed `maybe_externalize`). 8. JSON-serializability post-condition: `json.dumps(...)` round-trips the full payload (regression guard for the exact bug fixed). 9. Leaf-path regression: single-tensor / single-video / single-audio top-level inputs still encode unchanged. Loads `_encode_one` / `_encode_inputs` / `_externalize_nested` out of `proxy_node.py` via AST extraction so it runs without importing the full `comfy_api` / `comfy_api_nodes` packages, same pattern as `run_image_max_batch.py`. Runs clean against the ComfyUI venv: ``` $ python notes/run_encode_nested_dicts.py ok: top-level AG-of-IMAGE encoded per slot ok: top-level AG-of-VIDEO encoded per slot ok: DC-wrapping-AG encoded leaves at every level ok: AUDIO dict still routes to encode_audio_input ok: pre-encoded envelope passed through unchanged ok: scalar config dict passed through unchanged ok: nested AG envelopes externalized per slot ok: scalar siblings preserved alongside externalized envelopes ok: full encoded payload is JSON-serializable ok: top-level leaf path unchanged ALL OK ``` Unblocks the next batch of `comfy-rnp-server` providers that need AG-of-VIDEO or DC-wrapping-AG-of-VIDEO support (`WanReferenceVideoApi_RNP` `character1..character3` and `Wan2ReferenceVideoApi_RNP` `video1..video3` + `image1..image5`). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-26 22:29:00 +00:00