[PR #1173] [MERGED] [EP] Implement elastic scaling up

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2050

bsakaguchi commented

2026-10-29 21:32:24 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1173
Author: @UNIDY2002
Created: 12/5/2025
Status: ✅ Merged
Merged: 12/22/2025
Merged by: @UNIDY2002

Base: main ← Head: sunxun/extend-dev

📝 Commits (10+)

4b53d6c Implement elastic scaling up
342cbc7 Fixed a bug
b13b293 Debug
9a636b4 Debug
28c2e5c Fixed a bug
e7d8f26 Enable debug
1be2443 Fix
b25dd61 Fix
d02e227 Fix
bbf2936 Format

📊 Changes

9 files changed (+533 additions, -139 deletions)

View changed files

📝 .github/workflows/ci.yml (+1 -0)
📝 docs/source/python-api-reference/ep-backend.md (+38 -1)
📝 mooncake-ep/include/mooncake_backend.h (+21 -1)
📝 mooncake-ep/include/mooncake_worker.cuh (+9 -6)
📝 mooncake-ep/src/mooncake_backend.cpp (+214 -102)
📝 mooncake-ep/src/mooncake_worker.cu (+62 -28)
📝 mooncake-ep/src/mooncake_worker_thread.cpp (+18 -0)
📝 mooncake-integration/ep/ep_py.cpp (+33 -1)
➕ mooncake-wheel/tests/test_mooncake_backend_elastic.py (+137 -0)

📄 Description

Description

This PR introduces support for elastically adding new ranks and recovering failed ranks to existing collective process groups, without recreating the process group.

There is a sample test file in mooncake-wheel/tests/test_mooncake_backend_elastic.py, showing the usage.

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

The unit test mooncake-wheel/tests/test_mooncake_backend_elastic.py passes.
An ongoing work on SGLang integration verifies its e2e correctness (at https://github.com/HanHan009527/sglang/tree/sunxun/pr3-fix).

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1173 **Author:** [@UNIDY2002](https://github.com/UNIDY2002) **Created:** 12/5/2025 **Status:** ✅ Merged **Merged:** 12/22/2025 **Merged by:** [@UNIDY2002](https://github.com/UNIDY2002) **Base:** `main` ← **Head:** `sunxun/extend-dev` --- ### 📝 Commits (10+) - [`4b53d6c`](https://github.com/kvcache-ai/Mooncake/commit/4b53d6cf2ae97581aefd62b9ac9ab5e2bce398f7) Implement elastic scaling up - [`342cbc7`](https://github.com/kvcache-ai/Mooncake/commit/342cbc720ee51210f50c51bee30ddef7c0544225) Fixed a bug - [`b13b293`](https://github.com/kvcache-ai/Mooncake/commit/b13b2935a4fe60b5ba9d004f7ff796de795dc40b) Debug - [`9a636b4`](https://github.com/kvcache-ai/Mooncake/commit/9a636b49c1819655218b5850ac68c506fb6808e2) Debug - [`28c2e5c`](https://github.com/kvcache-ai/Mooncake/commit/28c2e5cf35225365fe9748df4f72598d82decd5c) Fixed a bug - [`e7d8f26`](https://github.com/kvcache-ai/Mooncake/commit/e7d8f2677e13236109cbb06d7f8299e9e9443461) Enable debug - [`1be2443`](https://github.com/kvcache-ai/Mooncake/commit/1be2443095d5ab00ab8e8a636f1990056b3231f3) Fix - [`b25dd61`](https://github.com/kvcache-ai/Mooncake/commit/b25dd61c6716eab04b2934cb53092b7962390e55) Fix - [`d02e227`](https://github.com/kvcache-ai/Mooncake/commit/d02e2274ab5efe64d81713c2fa5804d8a03f15c1) Fix - [`bbf2936`](https://github.com/kvcache-ai/Mooncake/commit/bbf29366e00a566b5be1b59f351141cd99dae8d5) Format ### 📊 Changes **9 files changed** (+533 additions, -139 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/ci.yml` (+1 -0) 📝 `docs/source/python-api-reference/ep-backend.md` (+38 -1) 📝 `mooncake-ep/include/mooncake_backend.h` (+21 -1) 📝 `mooncake-ep/include/mooncake_worker.cuh` (+9 -6) 📝 `mooncake-ep/src/mooncake_backend.cpp` (+214 -102) 📝 `mooncake-ep/src/mooncake_worker.cu` (+62 -28) 📝 `mooncake-ep/src/mooncake_worker_thread.cpp` (+18 -0) 📝 `mooncake-integration/ep/ep_py.cpp` (+33 -1) ➕ `mooncake-wheel/tests/test_mooncake_backend_elastic.py` (+137 -0) </details> ### 📄 Description ## Description  This PR introduces support for elastically adding new ranks and recovering failed ranks to existing collective process groups, without recreating the process group. There is a sample test file in mooncake-wheel/tests/test_mooncake_backend_elastic.py, showing the usage. ## Type of Change * Types - [ ] Bug fix - [ ] New feature - [ ] Transfer Engine - [ ] Mooncake Store - [x] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested?  1. The unit test mooncake-wheel/tests/test_mooncake_backend_elastic.py passes. 2. An ongoing work on SGLang integration verifies its e2e correctness (at https://github.com/HanHan009527/sglang/tree/sunxun/pr3-fix). ## Checklist - [x] I have performed a self-review of my own code. - [x] I have updated the documentation. - [x] I have added tests to prove my changes are effective. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

closed this issue
added the
pull-request
label

Rows
Columns

[PR #1173] [MERGED] [EP] Implement elastic scaling up #2050