[GH-ISSUE #1407] [RFC]: ERDMA Support with HugeTLB Memory Allocation

bsakaguchi commented

2026-10-29 21:14:48 +00:00

Owner

Originally created by @lichang98 on GitHub (Jan 20, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1407

Changes proposed

Changes Proposed

Summary

Add support for Alibaba Cloud Elastic RDMA (ERDMA) by implementing HugeTLB-based memory allocation and pre-registered buffer management. This enables large-scale memory registration (100s of GBs to TBs) required for LLM inference deployments on ERDMA.

Motivation

Alibaba Cloud Elastic Compute Service (ECS) provides Elastic RDMA (ERDMA) as a high-performance network solution for cloud-native workloads. ERDMA offers RDMA capabilities in virtualized environments but has specific hardware constraints that differ from traditional InfiniBand/RoCE deployments.

Problem Statement

Constraint	Impact
Limited `max_mr_size`	Restricts individual memory region size
Limited `max_ep_per_ctx`	Caps endpoints per context
Limited `max_sge`	Restricts scatter-gather elements

Why HugeTLB is Required

With standard 4KB pages, ERDMA can register no more than 128GB of memory—insufficient for large-scale LLM inference deployments with large KV cache pools.

Using huge pages (2MB or 1GB) significantly reduces the number of Memory Regions (MRs) needed:

Page Size	MRs for 256GB	ERDMA Feasibility
4KB	~67M pages	Exceeds MR limits
2MB	~128K pages	Within limits
1GB	256 pages	Well within limits

Reference: Alibaba Cloud ERDMA Best Practices

Proposed Changes

1. New Configuration Options

Environment Variable	Config Field	Default	Description
`MC_USE_HUGETLB`	`use_hugetlb`	`false`	Enable `MAP_HUGETLB` for memory allocations
`MC_CLIENT_BUFFER_SIZE`	`client_buffer_size`	`2GB`	Size of pre-allocated RDMA buffer for batch operations
`MC_MAX_MR_SIZE`	`max_mr_size`	`0x10000000000` (1TB)	Maximum memory region size

2. Memory Allocation Strategy

Current Behavior:

Uses aligned_alloc() for segment allocation
User buffers are used directly for RDMA operations

Proposed Behavior (when MC_USE_HUGETLB=1):

Allocate memory using mmap() with MAP_HUGETLB flag
Fallback to regular mmap() if huge pages are unavailable
Copy user data to pre-registered HugeTLB buffer before ERDMA transfer (required because inference framework buffers are not HugeTLB-backed)

Allocation Flow:

3. Batch Operation Buffer Management

Since inference framework KV buffers are not created with MAP_HUGETLB, data must be staged through pre-registered buffers although it may bring extra overhead:

batch_put_from flow:

batch_get_into flow:

4. Files Modified

File	Changes
`mooncake-transfer-engine/include/config.h`	Add `use_hugetlb`, `client_buffer_size` fields
`mooncake-transfer-engine/src/config.cpp`	Add environment variable parsing
`mooncake-store/include/real_client.h`	Add `client_mmap_buf_`, `client_mmap_buf_size_`, `SegmentAlloc` struct
`mooncake-store/src/real_client.cpp`	Implement ERDMA buffer management with bounds checking
`mooncake-store/src/utils.cpp`	Add HugeTLB support to `allocate_buffer_allocator_memory()`
`mooncake-store/src/dummy_client.cpp`	Add HugeTLB support for shared memory mapping

Usage Example

# 1. Configure huge pages on the system
echo 1024 | sudo tee /proc/sys/vm/nr_hugepages

# 2. Enable ERDMA support
export MC_USE_HUGETLB=1
export MC_CLIENT_BUFFER_SIZE=2147483648  # 2GB

# 3. Adjust for ERDMA hardware limits (optional)
export MC_MAX_MR_SIZE=0x1000000000       # 64GB
export MC_MAX_EP_PER_CTX=20000
export MC_MAX_SGE=1

# 4. Run Mooncake
./transfer_engine_bench --device_name=erdma_0 ...

Compatibility

Aspect	Guarantee
Backward Compatible	Default behavior unchanged (`MC_USE_HUGETLB=0`)
Graceful Degradation	Falls back to regular `mmap()` if huge pages unavailable
Bounds Checking	Validates buffer sizes before RDMA operations

Design Considerations

Consideration	Impact	Mitigation
Extra memcpy	Adds latency to batch operations	Necessary for ERDMA compatibility; async copy could be explored later
Memory overhead	Pre-allocated buffer consumes memory when idle	User-configurable via `MC_CLIENT_BUFFER_SIZE`
Huge page availability	Requires system configuration	Graceful fallback to regular pages

Testing Plan

Unit tests for configuration loading
Integration tests on ECS with ERDMA device
Fallback behavior when huge pages unavailable
Memory leak tests for buffer management
Stress performance tests with large batch operations

Documentation Updates

Add ERDMA deployment guide to docs/source/deployment/
Update environment variable reference in Transfer Engine docs
Add troubleshooting section for common ERDMA issues

References

https://developer.aliyun.com/article/1661045

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @lichang98 on GitHub (Jan 20, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1407 ### Changes proposed ## Changes Proposed ### Summary Add support for Alibaba Cloud Elastic RDMA (ERDMA) by implementing HugeTLB-based memory allocation and pre-registered buffer management. This enables large-scale memory registration (100s of GBs to TBs) required for LLM inference deployments on ERDMA. ### Motivation Alibaba Cloud Elastic Compute Service (ECS) provides Elastic RDMA (ERDMA) as a high-performance network solution for cloud-native workloads. ERDMA offers RDMA capabilities in virtualized environments but has specific hardware constraints that differ from traditional InfiniBand/RoCE deployments. #### Problem Statement | Constraint | Impact | |------------|--------| | Limited `max_mr_size` | Restricts individual memory region size | | Limited `max_ep_per_ctx` | Caps endpoints per context | | Limited `max_sge` | Restricts scatter-gather elements | #### Why HugeTLB is Required With standard 4KB pages, ERDMA can register **no more than 128GB of memory**—insufficient for large-scale LLM inference deployments with large KV cache pools. Using huge pages (2MB or 1GB) significantly reduces the number of Memory Regions (MRs) needed: | Page Size | MRs for 256GB | ERDMA Feasibility | |-----------|---------------|-------------------| | 4KB | ~67M pages | Exceeds MR limits | | 2MB | ~128K pages | Within limits | | 1GB | 256 pages | Well within limits | **Reference:** [Alibaba Cloud ERDMA Best Practices](https://developer.aliyun.com/article/1661045) --- ### Proposed Changes #### 1. New Configuration Options | Environment Variable | Config Field | Default | Description | |---------------------|--------------|---------|-------------| | `MC_USE_HUGETLB` | `use_hugetlb` | `false` | Enable `MAP_HUGETLB` for memory allocations | | `MC_CLIENT_BUFFER_SIZE` | `client_buffer_size` | `2GB` | Size of pre-allocated RDMA buffer for batch operations | | `MC_MAX_MR_SIZE` | `max_mr_size` | `0x10000000000` (1TB) | Maximum memory region size | #### 2. Memory Allocation Strategy **Current Behavior:** - Uses `aligned_alloc()` for segment allocation - User buffers are used directly for RDMA operations **Proposed Behavior (when `MC_USE_HUGETLB=1`):** - Allocate memory using `mmap()` with `MAP_HUGETLB` flag - Fallback to regular `mmap()` if huge pages are unavailable - Copy user data to pre-registered HugeTLB buffer before ERDMA transfer (required because inference framework buffers are not HugeTLB-backed) **Allocation Flow:** <img width="606" height="638" alt="Image" src="https://github.com/user-attachments/assets/aab2474a-9997-451d-9a27-66477aa4b53a" /> #### 3. Batch Operation Buffer Management Since inference framework KV buffers are not created with `MAP_HUGETLB`, data must be staged through pre-registered buffers although it may bring extra overhead: **`batch_put_from` flow:** <img width="1188" height="110" alt="Image" src="https://github.com/user-attachments/assets/6b2f84af-a22e-467c-b172-e0a21f4cbf44" /> **`batch_get_into` flow:** <img width="1196" height="114" alt="Image" src="https://github.com/user-attachments/assets/0a0e5e57-4646-4921-8d05-f5dbc2f23a59" /> #### 4. Files Modified | File | Changes | |------|---------| | `mooncake-transfer-engine/include/config.h` | Add `use_hugetlb`, `client_buffer_size` fields | | `mooncake-transfer-engine/src/config.cpp` | Add environment variable parsing | | `mooncake-store/include/real_client.h` | Add `client_mmap_buf_`, `client_mmap_buf_size_`, `SegmentAlloc` struct | | `mooncake-store/src/real_client.cpp` | Implement ERDMA buffer management with bounds checking | | `mooncake-store/src/utils.cpp` | Add HugeTLB support to `allocate_buffer_allocator_memory()` | | `mooncake-store/src/dummy_client.cpp` | Add HugeTLB support for shared memory mapping | --- ### Usage Example ```bash # 1. Configure huge pages on the system echo 1024 | sudo tee /proc/sys/vm/nr_hugepages # 2. Enable ERDMA support export MC_USE_HUGETLB=1 export MC_CLIENT_BUFFER_SIZE=2147483648 # 2GB # 3. Adjust for ERDMA hardware limits (optional) export MC_MAX_MR_SIZE=0x1000000000 # 64GB export MC_MAX_EP_PER_CTX=20000 export MC_MAX_SGE=1 # 4. Run Mooncake ./transfer_engine_bench --device_name=erdma_0 ... ``` --- Compatibility | Aspect | Guarantee | |--------|-----------| | **Backward Compatible** | Default behavior unchanged (`MC_USE_HUGETLB=0`) | | **Graceful Degradation** | Falls back to regular `mmap()` if huge pages unavailable | | **Bounds Checking** | Validates buffer sizes before RDMA operations | --- Design Considerations | Consideration | Impact | Mitigation | |---------------|--------|------------| | **Extra memcpy** | Adds latency to batch operations | Necessary for ERDMA compatibility; async copy could be explored later | | **Memory overhead** | Pre-allocated buffer consumes memory when idle | User-configurable via `MC_CLIENT_BUFFER_SIZE` | | **Huge page availability** | Requires system configuration | Graceful fallback to regular pages | --- Testing Plan - Unit tests for configuration loading - Integration tests on ECS with ERDMA device - Fallback behavior when huge pages unavailable - Memory leak tests for buffer management - Stress performance tests with large batch operations --- Documentation Updates - Add ERDMA deployment guide to docs/source/deployment/ - Update environment variable reference in Transfer Engine docs - Add troubleshooting section for common ERDMA issues --- References - https://developer.aliyun.com/article/1661045 ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi

2026-10-29 21:14:48 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@ykwd commented on GitHub (Jan 20, 2026):

Thanks, this proposal is very useful. Looking forward to the PR.
One request: please make sure it also works correctly in environments without ERDMA or without huge pages.

@ykwd commented on GitHub (Jan 20, 2026): Thanks, this proposal is very useful. Looking forward to the PR. One request: please make sure it also works correctly in environments without ERDMA or without huge pages.

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@stmatengss commented on GitHub (Jan 20, 2026):

FYI: https://github.com/kvcache-ai/Mooncake/pull/1300 I think this proposal can improve the overall performance.

@stmatengss commented on GitHub (Jan 20, 2026): FYI: https://github.com/kvcache-ai/Mooncake/pull/1300 I think this proposal can improve the overall performance.

bsakaguchi commented

2026-10-29 21:14:49 +00:00

Author

Owner

@lichang98 commented on GitHub (Jan 21, 2026):

FYI: #1300 I think this proposal can improve the overall performance.

Thank you for the information. The main part covers what I need, however ERDMA's restrictions don't seem to be taken into account. I'll add a comment below it.

@lichang98 commented on GitHub (Jan 21, 2026): > FYI: [#1300](https://github.com/kvcache-ai/Mooncake/pull/1300) I think this proposal can improve the overall performance. Thank you for the information. The main part covers what I need, however ERDMA's restrictions don't seem to be taken into account. I'll add a comment below it.

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@YiXR commented on GitHub (Jan 22, 2026):

Right now, inference framework like SGLang will use MooncakeHostMemAllocator to create L2 cache no matter if dummy client mode is chosen or not. If MC_USE_HUGEPAGE is set, the inference framework's buffer is created with MAP_HUGETLB.

@YiXR commented on GitHub (Jan 22, 2026): Right now, inference framework like SGLang will use MooncakeHostMemAllocator to create L2 cache no matter if dummy client mode is chosen or not. If MC_USE_HUGEPAGE is set, the inference framework's buffer is created with MAP_HUGETLB.

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@YiXR commented on GitHub (Jan 22, 2026):

max_mr_size will be updated by device_attr automatically.

@YiXR commented on GitHub (Jan 22, 2026): max_mr_size will be updated by device_attr automatically. <img width="1076" height="538" alt="Image" src="https://github.com/user-attachments/assets/7c852e27-f976-433b-bed1-548d6c5f15ae" />

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@easonyang1010 commented on GitHub (Mar 27, 2026):

hi, I also encountered this problem, I notice a PR for mooncake-store has been merged. So for mooncake-transfer-engine, this is also work?

@easonyang1010 commented on GitHub (Mar 27, 2026): hi, I also encountered this problem, I notice a PR for mooncake-store has been merged. So for mooncake-transfer-engine, this is also work?

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@lichang98 commented on GitHub (Apr 18, 2026):

hi, I also encountered this problem, I notice a PR for mooncake-store has been merged. So for mooncake-transfer-engine, this is also work?

Sorry for the late reply. Have you solved the problem?

@lichang98 commented on GitHub (Apr 18, 2026): > hi, I also encountered this problem, I notice a PR for mooncake-store has been merged. So for mooncake-transfer-engine, this is also work? Sorry for the late reply. Have you solved the problem?

bsakaguchi commented

2026-10-29 21:14:50 +00:00

Author

Owner

@stmatengss commented on GitHub (Apr 18, 2026):

For the transfer engine, users must register memory themselves, allocating it via Hugepage mmap.

@stmatengss commented on GitHub (Apr 18, 2026): For the transfer engine, users must register memory themselves, allocating it via Hugepage mmap.

bsakaguchi commented

2026-10-29 21:14:51 +00:00

Author

Owner

@easonyang1010 commented on GitHub (Apr 20, 2026):

@lichang98 yes, I already solved this. It was proved that the OFED version was too low, resulting in the GDR not using a 64KB-sized page. In eRDMA GDR, page size is 64KB, and has nothing to do with hugepage.

@easonyang1010 commented on GitHub (Apr 20, 2026): @lichang98 yes, I already solved this. It was proved that the OFED version was too low, resulting in the GDR not using a 64KB-sized page. In eRDMA GDR, page size is 64KB, and has nothing to do with hugepage.

bsakaguchi commented

2026-10-29 21:14:51 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 20, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jul 20, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi commented

2026-10-29 21:14:51 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 27, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 27, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:28:35 +00:00

[PR #451] [MERGED] [Store] High Availability 1: Master Failover #1574

Rows
Columns

[GH-ISSUE #1407] [RFC]: ERDMA Support with HugeTLB Memory Allocation #451

Changes proposed

Changes Proposed

Summary

Motivation

Problem Statement

Why HugeTLB is Required

Proposed Changes

1. New Configuration Options

2. Memory Allocation Strategy

3. Batch Operation Buffer Management

4. Files Modified

Usage Example

Before submitting a new issue...