[GH-ISSUE #31] Error in running vllm，install with cpu #20

New issue

Closed

opened 2026-10-29 21:07:57 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:07:57 +00:00

Owner

Originally created by @ygf85611 on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/31

MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8100 --max-model-len 10000 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}'

api_server.py: error: argument --kv-transfer-config: invalid from_cli value: '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}'

Originally created by @ygf85611 on GitHub (Dec 11, 2024). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/31 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8100 --max-model-len 10000 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}' api_server.py: error: argument --kv-transfer-config: invalid from_cli value: '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}'

bsakaguchi closed this issue

2026-10-29 21:07:57 +00:00

bsakaguchi referenced this issue

2026-10-29 21:26:03 +00:00

[PR #21] [MERGED] [TransferEngine][feature] support multiple metadata servers #1285