[PR #4] [MERGED] [Bugfix][EPD] Fix DeepStack encoder cache reservation sizing #2

New issue

Closed

opened 2026-10-29 21:06:01 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:06:01 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/vllm/pull/4
Author: @jiaran-king
Created: 8/23/2026
Status: ✅ Merged
Merged: 8/24/2026
Merged by: @gty111

Base: add-mooncake-ec-connector ← Head: codex/ecmooncake-deepstack-reservation

📝 Commits (1)

96d9975 [Bugfix][EPD] Fix DeepStack encoder cache reservation sizing

📊 Changes

2 files changed (+36 additions, -2 deletions)

View changed files

📝 tests/v1/ec_connector/unit/test_ec_mooncake_connector.py (+32 -1)
📝 vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py (+4 -1)

📄 Description

Summary

Fix encoder-cache reservation sizing for Qwen3-VL DeepStack and variants using the same cache layout.
Qwen3-VL DeepStack encoder cache width is out_hidden_size * (1 + len(deepstack_visual_indexes)).
The old get_hidden_size() path made the spec/reservation 4x too small; for example, [6144, 2560] was reserved for a [6144, 10240] tensor.
Reuse the existing _get_encoder_cache_hidden_dim helper; the ordinary-model fallback is unchanged.

This is an incremental child PR against the #41567 source branch add-mooncake-ec-connector, not a duplicate implementation against main.

Runtime evidence for the sizing path is consistent with the fix: the candidate path produced a 125,829,120-byte producer push with zero producer failures. This is not a claim of complete end-to-end correctness.

Validation

Targeted CPU test:

.venv/bin/python -m pytest tests/v1/ec_connector/unit/test_ec_mooncake_connector.py -k 'producer_builds_push_metadata_after_preprocessing or producer_uses_deepstack_encoder_cache_width'

Result: 2 passed, 40 deselected.

AI assistance was used to prepare this child PR. A human reviewer is responsible for reviewing and validating the change.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/vllm/pull/4 **Author:** [@jiaran-king](https://github.com/jiaran-king) **Created:** 8/23/2026 **Status:** ✅ Merged **Merged:** 8/24/2026 **Merged by:** [@gty111](https://github.com/gty111) **Base:** `add-mooncake-ec-connector` ← **Head:** `codex/ecmooncake-deepstack-reservation` --- ### 📝 Commits (1) - [`96d9975`](https://github.com/kvcache-ai/vllm/commit/96d99751df9053b5d4ead905d05a47beca6ccde3) [Bugfix][EPD] Fix DeepStack encoder cache reservation sizing ### 📊 Changes **2 files changed** (+36 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `tests/v1/ec_connector/unit/test_ec_mooncake_connector.py` (+32 -1) 📝 `vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py` (+4 -1) </details> ### 📄 Description ## Summary - Fix encoder-cache reservation sizing for Qwen3-VL DeepStack and variants using the same cache layout. - Qwen3-VL DeepStack encoder cache width is `out_hidden_size * (1 + len(deepstack_visual_indexes))`. - The old `get_hidden_size()` path made the spec/reservation 4x too small; for example, `[6144, 2560]` was reserved for a `[6144, 10240]` tensor. - Reuse the existing `_get_encoder_cache_hidden_dim` helper; the ordinary-model fallback is unchanged. This is an incremental child PR against the #41567 source branch `add-mooncake-ec-connector`, not a duplicate implementation against `main`. Runtime evidence for the sizing path is consistent with the fix: the candidate path produced a 125,829,120-byte producer push with zero producer failures. This is not a claim of complete end-to-end correctness. ## Validation Targeted CPU test: ```text .venv/bin/python -m pytest tests/v1/ec_connector/unit/test_ec_mooncake_connector.py -k 'producer_builds_push_metadata_after_preprocessing or producer_uses_deepstack_encoder_cache_width' ``` Result: `2 passed, 40 deselected`. AI assistance was used to prepare this child PR. A human reviewer is responsible for reviewing and validating the change. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:06:01 +00:00