kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #3540] [Bug]: Error occurs when Mooncake runs with multiple instances or DP on a single node under the same IP #905

New issue

Closed

opened 2026-10-29 21:20:45 +00:00 by bsakaguchi · 5 comments

bsakaguchi commented

2026-10-29 21:20:45 +00:00

Owner

Originally created by @absoluteword on GitHub (Aug 20, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3540

Bug Report

When using Mooncake on the 6000D and B300, I encountered TCP connection failures when running vLLM with DP or multiple‑instance shared pool within the same Docker container. This prevents content from being stored into the KV cache pool. However, it works properly after isolation using separate containers. Could you please advise whether DP is not yet adapted for this scenario, or if special configuration is required?

Problem Description:
When running multiple instances on a single node, the services start successfully. Both instances can connect to the Mooncake master, and TCP listening works as expected. However, TCP connection‑failure errors occur once inference starts, and subsequent attempts to access the original TCP listening port also fail.
After consulting relevant large‑model materials, my analysis suggests the root cause may be port‑list contamination among multiple Mooncake‑connected instances under the same‑IP network, which results in attempts to access incorrect ports. I would like to confirm whether this hypothesis is correct.

mooncake_config:
{
"metadata_server": "http://127.0.0.1:8092/metadata",
"master_server_address": "127.0.0.1:50063",
"global_segment_size": "256gb",
"local_buffer_size": "5gb",
"protocol": "tcp",
"device_name": ""
}

when I start Mooncake Master:

mooncake_master
--enable_http_metadata_server=true
--http_metadata_server_port=8092
--metrics_port=9003
--http_metadata_server_host=0.0.0.0 --rpc_port=50063 --default_kv_lease_ttl=1000000

when I start vllm LLM instance:

export MOONCAKE_CONFIG_PATH=/path/mooncake_config.json

PYTHONHASHSEED=0 vllm serve /models/models/Qwen3-8B --port 8000 --tensor-parallel-size 1 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}' --enable-prompt-tokens-details --enable-prefix-caching --gpu-memory-utilization 0.85

And the wrong message is like:

E0807 01:52:51.106648 17959 tcp_transport.cpp:886] TcpTransport::startTransfer failed to get connection to 10.66.6.30:16499
E0807 01:52:51.108152 17959 tcp_transport.cpp:708] TcpTransport::getConnection failed to create connection to 10.66.6.30:16499. Error: connect: Cannot assign requested address
E0807 01:52:51.108158 17959 tcp_transport.cpp:886] TcpTransport::startTransfer failed to get connection to 10.66.6.30:16499
E0807 01:52:51.109604 17959 tcp_transport.cpp:708] TcpTransport::getConnection failed to create connection to 10.66.6.30:16499. Error: connect: Cannot assign requested address

Is there anything wrong when I use Mooncake? Thanks for your help.

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @absoluteword on GitHub (Aug 20, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3540 ### Bug Report When using Mooncake on the 6000D and B300, I encountered TCP connection failures when running vLLM with DP or multiple‑instance shared pool within the same Docker container. This prevents content from being stored into the KV cache pool. However, it works properly after isolation using separate containers. Could you please advise whether DP is not yet adapted for this scenario, or if special configuration is required? Problem Description: When running multiple instances on a single node, the services start successfully. Both instances can connect to the Mooncake master, and TCP listening works as expected. However, TCP connection‑failure errors occur once inference starts, and subsequent attempts to access the original TCP listening port also fail. After consulting relevant large‑model materials, my analysis suggests the root cause may be port‑list contamination among multiple Mooncake‑connected instances under the same‑IP network, which results in attempts to access incorrect ports. I would like to confirm whether this hypothesis is correct. mooncake_config: { "metadata_server": "http://127.0.0.1:8092/metadata", "master_server_address": "127.0.0.1:50063", "global_segment_size": "256gb", "local_buffer_size": "5gb", "protocol": "tcp", "device_name": "" } when I start Mooncake Master: mooncake_master \ --enable_http_metadata_server=true \ --http_metadata_server_port=8092 \ --metrics_port=9003 \ --http_metadata_server_host=0.0.0.0 --rpc_port=50063 --default_kv_lease_ttl=1000000 when I start vllm LLM instance: export MOONCAKE_CONFIG_PATH=/path/mooncake_config.json PYTHONHASHSEED=0 vllm serve /models/models/Qwen3-8B --port 8000 --tensor-parallel-size 1 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}' --enable-prompt-tokens-details --enable-prefix-caching --gpu-memory-utilization 0.85 And the wrong message is like: E0807 01:52:51.106648 17959 tcp_transport.cpp:886] TcpTransport::startTransfer failed to get connection to 10.66.6.30:16499 E0807 01:52:51.108152 17959 tcp_transport.cpp:708] TcpTransport::getConnection failed to create connection to 10.66.6.30:16499. Error: connect: Cannot assign requested address E0807 01:52:51.108158 17959 tcp_transport.cpp:886] TcpTransport::startTransfer failed to get connection to 10.66.6.30:16499 E0807 01:52:51.109604 17959 tcp_transport.cpp:708] TcpTransport::getConnection failed to create connection to 10.66.6.30:16499. Error: connect: Cannot assign requested address Is there anything wrong when I use Mooncake? Thanks for your help. ### Before submitting... - [x] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:20:45 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-29 21:20:46 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Aug 20, 2026):

Thanks for opening this issue, @absoluteword!

Field	Value
Issue	#3540
GitHub user ID	`108380691`
Reporter	@absoluteword

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Aug 20, 2026): Thanks for opening this issue, @absoluteword! | Field | Value | |-------|-------| | **Issue** | #3540 | | **GitHub user ID** | `108380691` | | **Reporter** | @absoluteword | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:20:46 +00:00

Author

Owner

@absoluteword commented on GitHub (Aug 20, 2026):

Add . I use Mooncake 0.3.10.post2 and vllm 0.27.1, ubuntu 24.04.4 ,CUDA 13.2 driver 595.71.05

@absoluteword commented on GitHub (Aug 20, 2026): Add . I use Mooncake 0.3.10.post2 and vllm 0.27.1, ubuntu 24.04.4 ,CUDA 13.2 driver 595.71.05

bsakaguchi commented

2026-10-29 21:20:46 +00:00

Author

Owner

@alogfans commented on GitHub (Aug 20, 2026):

This seems a known issue of TCP transport. Could you use the latest release with option export MC_TCP_ENABLE_CONNECTION_POOL=1 enabled?

@alogfans commented on GitHub (Aug 20, 2026): This seems a known issue of TCP transport. Could you use the latest release with option `export MC_TCP_ENABLE_CONNECTION_POOL=1` enabled?

bsakaguchi commented

2026-10-29 21:20:47 +00:00

Author

Owner

@ykwd commented on GitHub (Aug 20, 2026):

@alogfans Thanks for the suggestions. By the way, if this option (MC_TCP_ENABLE_CONNECTION_POOL) isn’t enabled, the service won’t start at all. Why isn’t it enabled by default?

@ykwd commented on GitHub (Aug 20, 2026): @alogfans Thanks for the suggestions. By the way, if this option (MC_TCP_ENABLE_CONNECTION_POOL) isn’t enabled, the service won’t start at all. Why isn’t it enabled by default?

bsakaguchi commented

2026-10-29 21:20:47 +00:00

Author

Owner

@absoluteword commented on GitHub (Aug 20, 2026):

This seems a known issue of TCP transport. Could you use the latest release with option export MC_TCP_ENABLE_CONNECTION_POOL=1 enabled?

Thank you for your suggestion, it is now working properly.

@absoluteword commented on GitHub (Aug 20, 2026): > This seems a known issue of TCP transport. Could you use the latest release with option `export MC_TCP_ENABLE_CONNECTION_POOL=1` enabled? Thank you for your suggestion, it is now working properly.