[GH-ISSUE #1225] [RFC]: Future plans for improving Mooncake EP #414

New issue

Closed

opened 2026-10-29 21:14:12 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:14:12 +00:00

Owner

Originally created by @UNIDY2002 on GitHub (Dec 17, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1225

This issue outlines the planned improvements for Mooncake EP, organized by category and priority.

Functionalities

Support torch.distributed.send / recv (P0) → #1236
Support dynamic membership for the EP Buffer (P1) → #1630
Support additional collective primitives (e.g., gather, scatter, reduce) (P2) → #1469
Support full reduction ops for allreduce (e.g., product, min, max) (P2) → #1440

Performance

Improve performance of EP dispatch/combine (P0)
Improve performance of isend/irecv collective primitives (P1) → #1533

Maintainability

Make CUDA support future-proof (e.g. support CUDA 13) (P0)
Split the Torch Distributed backend from Mooncake EP into a separate directory (Mooncake PG, i.e., process group) (P1) → #1387, #1401
Avoid indexing SegmentDesc::buffers to obtain peer memory locations; transfer them through Torch's rendezvous store instead (P2)

Maintained by UNIDY2002's OpenClaw

Originally created by @UNIDY2002 on GitHub (Dec 17, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1225 This issue outlines the planned improvements for Mooncake EP, organized by category and priority. ## Functionalities - [x] Support `torch.distributed.send` / `recv` (P0) → [#1236](https://github.com/kvcache-ai/Mooncake/pull/1236) - [x] Support dynamic membership for the EP Buffer (P1) → [#1630](https://github.com/kvcache-ai/Mooncake/pull/1630) - [x] Support additional collective primitives (e.g., gather, scatter, reduce) (P2) → [#1469](https://github.com/kvcache-ai/Mooncake/pull/1469) - [x] Support full reduction ops for `allreduce` (e.g., product, min, max) (P2) → [#1440](https://github.com/kvcache-ai/Mooncake/pull/1440)  ## Performance - [x] Improve performance of EP dispatch/combine (P0) - [x] Improve performance of `isend/irecv` collective primitives (P1) → [#1533](https://github.com/kvcache-ai/Mooncake/pull/1533) ## Maintainability - [x] Make CUDA support future-proof (e.g. support CUDA 13) (P0) - [x] Split the Torch Distributed backend from Mooncake EP into a separate directory (Mooncake PG, i.e., process group) (P1) → [#1387](https://github.com/kvcache-ai/Mooncake/pull/1387), [#1401](https://github.com/kvcache-ai/Mooncake/pull/1401) - [x] Avoid indexing `SegmentDesc::buffers` to obtain peer memory locations; transfer them through Torch's rendezvous store instead (P2) --- *Maintained by UNIDY2002's OpenClaw*

bsakaguchi closed this issue

2026-10-29 21:14:12 +00:00