[GH-ISSUE #421] VLLM0.6.4.post1 test error #145

New issue

Closed

opened 2026-10-29 21:10:08 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:10:08 +00:00

Owner

Originally created by @LLJING-ER on GitHub (May 29, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/421

I install mooncake according to build docs, and can import mooncake_vllm_adaptor,but I cannot run the following command,
MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model /home/model/deepseek-1.5b/ --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}'
it has error
api_server.py: error: unrecognized arguments: --kv-transfer-config {"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}
and I tried another command in Prefill and Decode node,
VLLM_HOST_IP="10.105.118.15" VLLM_PORT="13003" MASTER_ADDR="10.105.118.15" MASTER_PORT="50051" MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_DISTRIBUTED_KV_ROLE=producer VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model /home/model/deepseek-1.5b/ --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.95
then run proxy_server.py

  ....
     async for _ in forward_request('http://10.105.118.200:8100/v1/completions', # our proxy server IP
                                       prefill_request):
            continue

        # return decode
        generator = forward_request('http://10.105.118.15:13003/v1/completions', # Be sure to change the IP address for your machine
                                    original_request_data)
  ....

moreover, in the Prefill node, nothing shows up after running the test below

curl -s http://109.105.118.200:8000/v1/completions -H "Content-Type: application/json" -d '{
  "model": "/home/model/deepseek-1.5b/",
  "prompt": "San Francisco is a",
  "max_tokens": 1000
}'

Here's what it reports in the Decode and Prefill node, it looks like there's no data transfer, can you give me some advice?
Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%

Originally created by @LLJING-ER on GitHub (May 29, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/421 1. I install mooncake according to **build docs**, and can **import mooncake_vllm_adaptor**,but I cannot run the following command, `MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model /home/model/deepseek-1.5b/ --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}'` **it has error** `api_server.py: error: unrecognized arguments: --kv-transfer-config {"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}` 2. and I tried another command in Prefill and Decode node, `VLLM_HOST_IP="10.105.118.15" VLLM_PORT="13003" MASTER_ADDR="10.105.118.15" MASTER_PORT="50051" MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_DISTRIBUTED_KV_ROLE=producer VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model /home/model/deepseek-1.5b/ --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.95` then run proxy_server.py ``` .... async for _ in forward_request('http://10.105.118.200:8100/v1/completions', # our proxy server IP prefill_request): continue # return decode generator = forward_request('http://10.105.118.15:13003/v1/completions', # Be sure to change the IP address for your machine original_request_data) .... ``` moreover, in the Prefill node, nothing shows up after running the test below ``` curl -s http://109.105.118.200:8000/v1/completions -H "Content-Type: application/json" -d '{ "model": "/home/model/deepseek-1.5b/", "prompt": "San Francisco is a", "max_tokens": 1000 }' ``` Here's what it reports in the Decode and Prefill node, it looks like there's no data transfer, can you give me some advice? `Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%`

bsakaguchi

2026-10-29 21:10:08 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:10:09 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 9, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jul 9, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:10:09 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 17, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 17, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:26:41 +00:00

[PR #145] [MERGED] fix compilation errors #1353