mirror of
https://github.com/kvcache-ai/Mooncake.git
synced 2026-11-05 01:17:55 +00:00
[GH-ISSUE #261] two H20 nodes to run DeepSeek671b 1P1D occur RuntimeError: shape '[-1, 16, 56]' is invalid for input of size 36864 #106
Labels
No labels
ascend
auto-closed
bug
bug
CI/Build
Common
documentation
enhancement
feature request
good first issue
help wanted
high priority
MetaX
nightly-failure
pull-request
PyTorch Backend
question
RFC
Roadmap
stale
Store
Tests
Transfer Engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/Mooncake#106
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @RockGo on GitHub (Apr 16, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/261
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] output = self.model_runner.execute_model(
(VllmWorkerProcess pid=2491010) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] RuntimeError: shape '[-1, 16, 56]' is invalid for input of size 36864
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1787, in execute_model
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] get_kv_transfer_group().send_kv_caches_and_hidden_states(
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 61, in send_kv_caches_and_hidden_states
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] self.connector.send_kv_caches_and_hidden_states(
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_connector/mooncake_store_connector.py", line 101, in send_kv_caches_and_hidden_states
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] key_cache = kv_cache[0].reshape(-1, num_heads, head_size)
(VllmWorkerProcess pid=2491016) ERROR 04-16 20:47:15 [multiproc_worker_utils.py:238] RuntimeError: shape '[-1, 16, 56]' is invalid for input of size 36864
@ShangmingCai commented on GitHub (Apr 17, 2025):
Can you try the vllm main branch? There is a fix to support MLA with mooncake.
@ShangmingCai commented on GitHub (Apr 17, 2025):
Maybe try this https://github.com/vllm-project/vllm/pull/16020 ?
@RockGo commented on GitHub (Apr 17, 2025):
it's works ok