[GH-ISSUE #510] Have the KVCache-centric Scheduling Algorithm implemented in SGLang or Vllm? #177

New issue

Closed

opened 2026-10-29 21:10:32 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:10:32 +00:00

Owner

Originally created by @Zhangmj0621 on GitHub (Jun 17, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/510

Dear Experts,
I would like to inquire whether there are plans to implement and open-source the KVCache-centric Scheduling Algorithm in frameworks such as SGLang or vLLM in the future. I am particularly interested in the details of its implementation.
Specifically, when a node is chosen as prefill node with a prefix length that is not the best, it seems that KVCache data must be transferred between prefill nodes. Since this KVCache likely resides in DRAM not VRAM, I assume this transfer cannot utilize GDR (GPU Direct RDMA). I am also curious about how the connection between prefill nodes is established in such cases.
Any suggestion will be highly appreciated!

Originally created by @Zhangmj0621 on GitHub (Jun 17, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/510 Dear Experts, I would like to inquire whether there are plans to implement and open-source the KVCache-centric Scheduling Algorithm in frameworks such as SGLang or vLLM in the future. I am particularly interested in the details of its implementation. Specifically, when a node is chosen as prefill node with a prefix length that is not the best, it seems that KVCache data must be transferred between prefill nodes. Since this KVCache likely resides in DRAM not VRAM, I assume this transfer cannot utilize GDR (GPU Direct RDMA). I am also curious about how the connection between prefill nodes is established in such cases. Any suggestion will be highly appreciated!

bsakaguchi

2026-10-29 21:10:32 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:10:32 +00:00

Author

Owner

@zhaoyongke commented on GitHub (Jun 18, 2025):

Look into this thread: https://github.com/kvcache-ai/Mooncake/issues/381

@zhaoyongke commented on GitHub (Jun 18, 2025): Look into this thread: https://github.com/kvcache-ai/Mooncake/issues/381

bsakaguchi commented

2026-10-29 21:10:33 +00:00

Author

Owner

@Zhangmj0621 commented on GitHub (Jun 19, 2025):

Look into this thread: #381

It seems that a Cache-aware scheduler has not been open-source. Do you have some open-source plan?

@Zhangmj0621 commented on GitHub (Jun 19, 2025): > Look into this thread: [#381](https://github.com/kvcache-ai/Mooncake/issues/381) It seems that a Cache-aware scheduler has not been open-source. Do you have some open-source plan?

bsakaguchi commented

2026-10-29 21:10:33 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 4, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jul 4, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:10:33 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 13, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 13, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:26:52 +00:00

[PR #177] [MERGED] Package mooncake_vllm_adaptor and MooncakeDistributedStore into a wheel #1373