[GH-ISSUE #901] [Usage]: mooncake的优势在哪？ #284

New issue

Closed

opened 2026-10-29 21:12:01 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:12:01 +00:00

Owner

Originally created by @lijufeng2016 on GitHub (Sep 29, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/901

Describe your usage question

VLLM_USE_V1=0
nohup python3 -m vllm.entrypoints.openai.api_server
--model Qwen/Qwen2.5-7B-Instruct
--served-model-name Qwen/Qwen2.5-7B-Instruct
--port 8100
--max-model-len 10000
--gpu-memory-utilization 0.92
--max-num-seqs 64
--max-num-batched-tokens 32768

evalscope perf
--api openai
--url http://127.0.0.1:8088/v1/completions
--model "Qwen/Qwen2.5-7B-Instruct"
--tokenizer-path Qwen/Qwen2.5-7B-Instruct
--dataset openqa
--parallel 100
--number 400
--max-tokens 1024
--temperature 0.0
--extra-args '{"ignore_eos": true}'

上面我的vllm启动命令和压测命令，总共3个节点做kv_both节点，没有使用mooncake的情况下，3个节点做了负载均衡的请求代理，模型是Qwen2.5-7B，数据集是openqa；

然后测试使用mooncake，一台kv_both,两台单独的decoder，测之前做了大量的预热，给足了了DRAM内存，最后测试的结果并没有前面不使用mooncake的好，这是为什么？

或者说，什么情况下，能体现mooncake的优势？

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @lijufeng2016 on GitHub (Sep 29, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/901 ### Describe your usage question VLLM_USE_V1=0 \ nohup python3 -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen2.5-7B-Instruct \ --served-model-name Qwen/Qwen2.5-7B-Instruct \ --port 8100 \ --max-model-len 10000 \ --gpu-memory-utilization 0.92 \ --max-num-seqs 64 \ --max-num-batched-tokens 32768 evalscope perf \ --api openai \ --url http://127.0.0.1:8088/v1/completions \ --model "Qwen/Qwen2.5-7B-Instruct" \ --tokenizer-path Qwen/Qwen2.5-7B-Instruct \ --dataset openqa \ --parallel 100 \ --number 400 \ --max-tokens 1024 \ --temperature 0.0 \ --extra-args '{"ignore_eos": true}' 上面我的vllm启动命令和压测命令，总共3个节点做kv_both节点，没有使用mooncake的情况下，3个节点做了负载均衡的请求代理，模型是Qwen2.5-7B，数据集是openqa；然后测试使用mooncake，一台kv_both,两台单独的decoder，测之前做了大量的预热，给足了了DRAM内存，最后测试的结果并没有前面不使用mooncake的好，这是为什么？或者说，什么情况下，能体现mooncake的优势？ ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:12:01 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:12:02 +00:00

Author

Owner

@lijufeng2016 commented on GitHub (Sep 29, 2025):

想要知道mooncake的正确使用姿势，比如参数不对？还是模型不对？还是数据集不对？

@lijufeng2016 commented on GitHub (Sep 29, 2025): 想要知道mooncake的正确使用姿势，比如参数不对？还是模型不对？还是数据集不对？

bsakaguchi commented

2026-10-29 21:12:02 +00:00

Author

Owner

@alogfans commented on GitHub (Oct 13, 2025):

PD分离模式的性能不一定优于非PD分离模式，这是因为前者在关键路径上有数据传输需求。我们试验后发现一般需要（包括但不限于）下列条件：

多路 RDMA 网络
长输入，短输出 —— 这主要是因为非PD分离模式会产生明显的流水线气泡
合理的P/D数量配置

@alogfans commented on GitHub (Oct 13, 2025): PD分离模式的性能不一定优于非PD分离模式，这是因为前者在关键路径上有数据传输需求。我们试验后发现一般需要（包括但不限于）下列条件： - 多路 RDMA 网络 - 长输入，短输出 —— 这主要是因为非PD分离模式会产生明显的流水线气泡 - 合理的P/D数量配置

bsakaguchi commented

2026-10-29 21:12:02 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 27, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jun 27, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:12:02 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 5, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 5, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.