[GH-ISSUE #3363] [SegmentPool] Refactor Memory and CXL management into SegmentPool #855

New issue

Open

opened 2026-10-29 21:20:13 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:20:13 +00:00

Owner

Originally created by @Aionw on GitHub (Aug 10, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3363

Parent RFC: #3360
Depends on: #3362

Goal

Turn the remaining Memory/CXL segment manager into the region-oriented SegmentPool boundary.

Scope

Refactor or rename the post-LocalSSD SegmentManager into SegmentPool.
Define a mounted pool element as a client-mounted, addressable allocation region.
Keep Memory and CXL catalog, indexes, allocator ownership, status, and two-phase unmount inside the pool.
Introduce stateful Memory/CXL region drivers that own physical allocator resources.
Extract pointer-based placement indexing and concrete replica allocation while preserving all existing policy behavior.
Separate allocator and pool snapshot codecs while preserving the current wire shape.
Update internal MasterService call sites and tests while retaining external RPC contracts.

Out of scope

Moving NoF state into the pool; that is tracked separately.
Changing placement-policy behavior, segment-name grouping semantics, locality ordering, or CXL replica representation.
Generalizing promotion targets.

Implementation stack

Mechanisms land before the final caller switch:

Order	PR	Review focus
1	#3703	Region-driver resource ownership, import, and rollback
2	#3704	Placement pointer stability, policy feedback, and hot-path allocation
3	#3705	SegmentPool catalog/index invariants and lifecycle transactions
4	#3706	Snapshot validation, allocator adoption, and fork safety
5	#3707	MasterService/NoF call-site cutover and legacy-path deletion

Each PR is based on the preceding PR. Merge in order; do not retarget later layers to main until their dependency has landed.

Acceptance criteria

SegmentPool is the sole owner of Memory/CXL mounted-region and allocator state.
CXL remains represented as a MEMORY replica and keeps its existing global allocator behavior.
Same-name segment grouping and existing mount/remount/unmount error behavior are preserved.
Snapshot wire shape and LocalSSD/NoF ownership boundaries are preserved.
Release PutStart comparisons remain within 3% single-thread QPS, 5% multi-thread QPS, and 10% p99 regression thresholds.
Focused driver, placement, SegmentPool, snapshot, NoF mapping, LocalSSD, offload, promotion, and HA tests pass.

Contribution notes

Review the stack in the listed order. Keep feedback on resource ownership, placement execution, pool lifecycle, snapshot boundaries, and caller cutover attached to the corresponding layer. NoF catalog/resource migration remains in #3364.

Originally created by @Aionw on GitHub (Aug 10, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3363 Parent RFC: #3360 Depends on: #3362 ## Goal Turn the remaining Memory/CXL segment manager into the region-oriented `SegmentPool` boundary. ## Scope - Refactor or rename the post-LocalSSD `SegmentManager` into `SegmentPool`. - Define a mounted pool element as a client-mounted, addressable allocation region. - Keep Memory and CXL catalog, indexes, allocator ownership, status, and two-phase unmount inside the pool. - Introduce stateful Memory/CXL region drivers that own physical allocator resources. - Extract pointer-based placement indexing and concrete replica allocation while preserving all existing policy behavior. - Separate allocator and pool snapshot codecs while preserving the current wire shape. - Update internal MasterService call sites and tests while retaining external RPC contracts. ## Out of scope - Moving NoF state into the pool; that is tracked separately. - Changing placement-policy behavior, segment-name grouping semantics, locality ordering, or CXL replica representation. - Generalizing promotion targets. ## Implementation stack Mechanisms land before the final caller switch: | Order | PR | Review focus | | ---: | --- | --- | | 1 | #3703 | Region-driver resource ownership, import, and rollback | | 2 | #3704 | Placement pointer stability, policy feedback, and hot-path allocation | | 3 | #3705 | SegmentPool catalog/index invariants and lifecycle transactions | | 4 | #3706 | Snapshot validation, allocator adoption, and fork safety | | 5 | #3707 | MasterService/NoF call-site cutover and legacy-path deletion | Each PR is based on the preceding PR. Merge in order; do not retarget later layers to `main` until their dependency has landed. ## Acceptance criteria - `SegmentPool` is the sole owner of Memory/CXL mounted-region and allocator state. - CXL remains represented as a MEMORY replica and keeps its existing global allocator behavior. - Same-name segment grouping and existing mount/remount/unmount error behavior are preserved. - Snapshot wire shape and LocalSSD/NoF ownership boundaries are preserved. - Release PutStart comparisons remain within 3% single-thread QPS, 5% multi-thread QPS, and 10% p99 regression thresholds. - Focused driver, placement, SegmentPool, snapshot, NoF mapping, LocalSSD, offload, promotion, and HA tests pass. ## Contribution notes Review the stack in the listed order. Keep feedback on resource ownership, placement execution, pool lifecycle, snapshot boundaries, and caller cutover attached to the corresponding layer. NoF catalog/resource migration remains in #3364.

bsakaguchi commented

2026-10-29 21:20:14 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Aug 10, 2026):

Thanks for opening this issue, @Aionw!

Field	Value
Issue	#3363
GitHub user ID	`41376987`
Reporter	@Aionw

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Aug 10, 2026): Thanks for opening this issue, @Aionw! | Field | Value | |-------|-------| | **Issue** | #3363 | | **GitHub user ID** | `41376987` | | **Reporter** | @Aionw | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi referenced this issue

2026-10-29 21:30:58 +00:00

[PR #855] [CLOSED] [WIP]feat(rdma): add parallel memory region registration support #1869