kvcache-ai/Mooncake

Fork

You've already forked Mooncake

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

Code Issues 368 Projects Packages Wiki Activity

[PR #1326] [MERGED] [Store][Feature]: add task manager component #2160

New issue

Closed

opened 2026-10-29 21:33:15 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:33:15 +00:00

Owner

Copy link

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/1326
Author: @zhongzhouTan-coder
Created: 1/5/2026
Status: ✅ Merged
Merged: 1/7/2026
Merged by: @stmatengss

Base: main ← Head: feature/task-manager

📝 Commits (10+)

2c53b9b [Store]: initialize the basic task data structure
b4d39a7 [Store]: implement the task manager
b15ad60 [Store]: change the mutex to custom mutext
26a8fd9 [Store]: implement copy and move, query task api
7dec5a0 [Store]: change the task manager lock to be shared
f219b3b [Store]: change the task to struct when submit
078aacc [Store]: add fetch tasks api
4cb60be [Store]: add metrics for query and fetch task api
b533286 [Store]: add update task status api
cad1934 [Store]: rename the updateTask api and add pending and processing task limit

📊 Changes

31 files changed (+2441 additions, -52 deletions)

View changed files

📝 mooncake-integration/store/store_py.cpp (+104 -1)
📝 mooncake-store/include/client_service.h (+48 -0)
📝 mooncake-store/include/dummy_client.h (+9 -0)
📝 mooncake-store/include/master_client.h (+50 -0)
📝 mooncake-store/include/master_config.h (+110 -1)
📝 mooncake-store/include/master_metric_manager.h (+36 -0)
📝 mooncake-store/include/master_service.h (+71 -1)
📝 mooncake-store/include/pyclient.h (+10 -0)
📝 mooncake-store/include/real_client.h (+32 -0)
📝 mooncake-store/include/rpc_service.h (+14 -0)
📝 mooncake-store/include/rpc_types.h (+67 -1)
📝 mooncake-store/include/segment.h (+11 -1)
➕ mooncake-store/include/task_manager.h (+234 -0)
📝 mooncake-store/include/types.h (+14 -0)
📝 mooncake-store/include/utils.h (+0 -1)
📝 mooncake-store/src/CMakeLists.txt (+1 -0)
📝 mooncake-store/src/client_service.cpp (+26 -0)
📝 mooncake-store/src/dummy_client.cpp (+16 -0)
📝 mooncake-store/src/master.cpp (+101 -40)
📝 mooncake-store/src/master_client.cpp (+81 -0)

...and 11 more files

📄 Description

Description

For the review purpose, we just split the large pr to three small pr.
refer to the origin pr: https://github.com/kvcache-ai/Mooncake/pull/1278

In order to support the dynamic replica adjustment, we need to support replica move and copy first, so we implement the basic replica copy and move features.

The detailed architecture design in the RFC: https://github.com/kvcache-ai/Mooncake/issues/1159

Features in this pr:

external api (python api)
- create_copy_task api to create copy task
- create_move_task api to create move task
- query_task api to monitor task status
internal api
- fetchTasks api for client to poll tasks
- markTaskToComplete api for client to report task status
task manager component
- limited client task queue (default capacity is 10000)
- finished task cleanup logic
- pending and processing task timeout logic

Type of Change

Types
- Bug fix
- New feature
  - Transfer Engine
  - Mooncake Store
  - Mooncake EP
  - Integration
  - P2P Store
  - Python Wheel
- Breaking change
- CI/CD
- Documentation update
- Other

How Has This Been Tested?

unit test
api test

Checklist

I have performed a self-review of my own code.
I have updated the documentation.
I have added tests to prove my changes are effective.

Co-authors

Co-authored-by: Chen Jinlong chenjinlong.cjl@alibaba-inc.com
Co-authored-by: Vincent Gao vincentbo@linux.alibaba.com

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/1326 **Author:** [@zhongzhouTan-coder](https://github.com/zhongzhouTan-coder) **Created:** 1/5/2026 **Status:** ✅ Merged **Merged:** 1/7/2026 **Merged by:** [@stmatengss](https://github.com/stmatengss) **Base:** `main` ← **Head:** `feature/task-manager` --- ### 📝 Commits (10+) - [`2c53b9b`](https://github.com/kvcache-ai/Mooncake/commit/2c53b9b86a7decd42c7fc3de449e41537f06d859) [Store]: initialize the basic task data structure - [`b4d39a7`](https://github.com/kvcache-ai/Mooncake/commit/b4d39a75756696d2ce87b5640115954d21a4110f) [Store]: implement the task manager - [`b15ad60`](https://github.com/kvcache-ai/Mooncake/commit/b15ad6077283c6d13a97812ff1e8d7da3fdc1060) [Store]: change the mutex to custom mutext - [`26a8fd9`](https://github.com/kvcache-ai/Mooncake/commit/26a8fd9b10c4272c8a0f3250f59ff455c857f6fc) [Store]: implement copy and move, query task api - [`7dec5a0`](https://github.com/kvcache-ai/Mooncake/commit/7dec5a0f213806e001a7297d22bbbf70b1e54f8b) [Store]: change the task manager lock to be shared - [`f219b3b`](https://github.com/kvcache-ai/Mooncake/commit/f219b3b5cf652d27346d258db28f3245f01282be) [Store]: change the task to struct when submit - [`078aacc`](https://github.com/kvcache-ai/Mooncake/commit/078aaccca04945ff2dc72187e4235adc9fdca2b7) [Store]: add fetch tasks api - [`4cb60be`](https://github.com/kvcache-ai/Mooncake/commit/4cb60be4a02aa6aa3d78a86ed5d5c19445e657cb) [Store]: add metrics for query and fetch task api - [`b533286`](https://github.com/kvcache-ai/Mooncake/commit/b5332868892dbc45db1c4eed46f3e95538e2a602) [Store]: add update task status api - [`cad1934`](https://github.com/kvcache-ai/Mooncake/commit/cad1934c6ef583a504a5cc41929347be83c86982) [Store]: rename the updateTask api and add pending and processing task limit ### 📊 Changes **31 files changed** (+2441 additions, -52 deletions) <details> <summary>View changed files</summary> 📝 `mooncake-integration/store/store_py.cpp` (+104 -1) 📝 `mooncake-store/include/client_service.h` (+48 -0) 📝 `mooncake-store/include/dummy_client.h` (+9 -0) 📝 `mooncake-store/include/master_client.h` (+50 -0) 📝 `mooncake-store/include/master_config.h` (+110 -1) 📝 `mooncake-store/include/master_metric_manager.h` (+36 -0) 📝 `mooncake-store/include/master_service.h` (+71 -1) 📝 `mooncake-store/include/pyclient.h` (+10 -0) 📝 `mooncake-store/include/real_client.h` (+32 -0) 📝 `mooncake-store/include/rpc_service.h` (+14 -0) 📝 `mooncake-store/include/rpc_types.h` (+67 -1) 📝 `mooncake-store/include/segment.h` (+11 -1) ➕ `mooncake-store/include/task_manager.h` (+234 -0) 📝 `mooncake-store/include/types.h` (+14 -0) 📝 `mooncake-store/include/utils.h` (+0 -1) 📝 `mooncake-store/src/CMakeLists.txt` (+1 -0) 📝 `mooncake-store/src/client_service.cpp` (+26 -0) 📝 `mooncake-store/src/dummy_client.cpp` (+16 -0) 📝 `mooncake-store/src/master.cpp` (+101 -40) 📝 `mooncake-store/src/master_client.cpp` (+81 -0) _...and 11 more files_ </details> ### 📄 Description ## Description For the review purpose, we just split the large pr to three small pr. refer to the origin pr: https://github.com/kvcache-ai/Mooncake/pull/1278 In order to support the dynamic replica adjustment, we need to support replica move and copy first, so we implement the basic replica copy and move features. The detailed architecture design in the RFC: https://github.com/kvcache-ai/Mooncake/issues/1159 Features in this pr: 1. external api (python api) - `create_copy_task` api to create copy task - `create_move_task` api to create move task - `query_task` api to monitor task status 2. internal api - `fetchTasks` api for client to poll tasks - `markTaskToComplete` api for client to report task status 3. task manager component - limited client task queue (default capacity is 10000) - finished task cleanup logic - pending and processing task timeout logic ## Type of Change * Types - [ ] Bug fix - [x] New feature - [ ] Transfer Engine - [x] Mooncake Store - [ ] Mooncake EP - [ ] Integration - [ ] P2P Store - [ ] Python Wheel - [ ] Breaking change - [ ] CI/CD - [ ] Documentation update - [ ] Other ## How Has This Been Tested? - unit test - api test ## Checklist - [x] I have performed a self-review of my own code. - [x] I have updated the documentation. - [x] I have added tests to prove my changes are effective. ## Co-authors Co-authored-by: Chen Jinlong [chenjinlong.cjl@alibaba-inc.com](mailto:chenjinlong.cjl@alibaba-inc.com) Co-authored-by: Vincent Gao [vincentbo@linux.alibaba.com](mailto:vincentbo@linux.alibaba.com) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:33:15 +00:00

closed this issue
added the
pull-request
label

No Branch/Tag specified

main

te-coro

staryxchen-tent-capability-research

feature/musa-ep-jit

zxpdemonio/fix-tent-bootstrap-admission

add-store-rs-tone-tests

scatter-rdma-completion-pr2

P2P-Mooncake-Store

release/v0.3.14

feat/tent-rdma-adaptive-adapter

fix/ep-local-counter-gpu-scope

codex/kvcs-oplog-eviction

give-me-a-star

cruz/mooncake-kvcs-lowlevel-pr1-20261008

ylt-handshake-limits

ci/rocm-e2e-tent

sima/docs-pypi-downloads

fix-ci/hicache-mooncake-metadata-loopback

sima/cuda-driver-loading

sima/ci-performance-gate

fix/tent-rdma-unknown-topology-rail-pairing

ci/tent-e2e-dual-job

mc-patch-1

codex/p2p-merge-prerequisites

sima/store-hca-numa-placement

codex/p2p-merge-cli

codex/p2p-merge-ipv6

copilot/fix-ctests-job-failure

czh/pg-device-collectives-pr6

dependabot/cargo/mooncake-store-rs/pyo3-0.29.0

czh/pg-device-collectives-pr5

codex/segment-pool-master-cutover

sima/nightly-ci-recovery

ci/tent-params

stmatengss-patch-1

dependabot/cargo/mooncake-store-rs/rand-0.8.8

dependabot/cargo/mooncake-store-rs/opentelemetry_sdk-0.32.1

dependabot/cargo/mooncake-store-rs/xxhash-rust-0.8.16

codex/migrate-store-rest-service

fix/tone-wheel-arch-selection

czh/pg-device-collectives-pr4

czh/pg-device-collectives-pr3

ketong/fix_moe_tc

copilot/fix-test-wheel-ubuntu-job

fix/ci-wheel-torch-pg-pin

codex/migrate-structured-object-storage

codex/migrate-allocator-python

codex/migrate-vllm-python

codex/migrate-shared-segment

ci/fix_sgl_eval_install

codex/segment-pool-nof-placement-cleanup

feat/pip-native-release

copilot/bugfix-ha-standby-promotion

copilot/mooncake-store-transport-settings

copilot/fix-tensor-support-issues

copilot/optimize-rl-direction

copilot/mooncake-storeagentic-workflow

copilot/fix-client-reconnect-timeout

copilot/mooncake-error-debugging

copilot/mooncake-ipv6-support

feat/nccl-host-python-api

dependabot/go_modules/mooncake-p2p-store/src/p2pstore/google.golang.org/grpc-1.82.1

copilot/migrate-and-move-operations

copilot/migrate-and-move-operations-again

refactor/centralize-env-vars

idhanani/mooncake-retain-groups

copilot/codeownerreviewer

copilot/give-feedback-on-issue-2123

copilot/create-issue-rfc

copilot/break-cuda-monopoly-logic

copilot/compare-cudamemcpy-cu12-cu13

copilot/report-ci-error-issue

copilot/research-commit-history-analysis

copilot/explain-mooncake-store-service

copilot/analyze-project-functionality

copilot/sub-pr-1680

copilot/review-pr-1680

copilot/sub-pr-1597

copilot/fix-github-actions-job-failure

copilot/move-test-sglang-integration-job

v0.3.14

v0.3.14-pre-4

v0.3.14-pre-3

v0.3.14-pre-2

v0.3.14-pre

v0.3.14-rc1

v0.3.13.post1

v0.3.13-rc2

v0.3.13-rc1

v0.3.13

v0.3.12.post1

v0.3.12

v0.3.11-rc1

v0.3.11.post1

v0.3.11

v0.3.10.post2

v0.3.10.post1

v0.3.10

v0.3.9

v0.3.8.post1

v0.3.8

v0.3.7.post2

v0.3.7.post1

v0.3.7

v0.3.6.post1

v0.3.6

v0.3.5

v0.3.4.post2

v0.3.4.post1

v0.3.4

v0.3.3.post2

v0.3.3.post1

v0.3.3

v0.3.2.post1

v0.3.2

v0.3.1

v0.3.0

v0.3.0-beta

v0.3.0-alpha

v0.2.0

Labels

Clear labels

Mirrored from GitHub Pull Request

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/Mooncake#2160

Reference in a new issue

Repository

kvcache-ai/Mooncake

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns