kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #5015] [Bug][TE]: GH200: DMA-BUF-registered GPU MRs whose start is 2^k-aligned (k ≥ 32) wrap every 2^k bytes, giving silent data corruption #1270

New issue

Open

opened 2026-10-29 21:24:40 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:24:40 +00:00

Owner

Originally created by @0z5a on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/5015

Describe the bug

On GH200, a GPU buffer registered through Mooncake's DMA-BUF path (WITH_NVIDIA_PEERMEM=0, i.e. exportDmabuf() → cuMemGetHandleForAddressRange → ibv_reg_dmabuf_mr) misbehaves when the MR's start address is exactly 2^k-aligned with k ≥ 32:

The MR behaves as if it wraps every 2^k bytes.
RDMA READ/WRITE at offset X ≥ 2^k actually reaches X mod 2^k.
Every transfer completes with rc = 0, and nothing is logged.

The same buffers registered with ibv_reg_mr (nvidia-peermem, the default WITH_NVIDIA_PEERMEM=1) are correct.

Large cudaMalloc allocations land on such boundaries in ordinary use. A 16 GiB allocation at 0xf1b700000000, which is 2^32-aligned, corrupted a plain run with no forced alignment. So any process on the DMA-BUF path can hit this, intermittently, depending on where its allocation lands.

Results

Setup:

GH200, driver 595.58.03, CUDA 13.2, kernel 6.8.0-136-generic, mlx5 NIC (MT41692, firmware 32.43.1014), upstream main 829d95ff.
RDMA loopback through the NIC, two processes, each with a 12 GiB GPU MR.
The start is forced to exactly 2^k (not 2^(k+1)) by carving it out of a larger cudaMalloc.
Each run moves 1152 scattered 64 KiB blocks by READ and by WRITE, plus transfers that end at the MR's last byte. Everything is compared byte by byte.

registration	aligned side	k	result
DMA-BUF	remote	31	clean
DMA-BUF	remote	32	remote offsets 0–4 GiB: 0 bad; 4–12 GiB: 100% bad (READ 781/1152 blocks bad, 776 all zero)
DMA-BUF	remote	33	remote offsets 0–8 GiB: 0 bad; 8–12 GiB: 100% bad
DMA-BUF	local	32	local offsets ≥ 4 GiB: 100% bad
nvidia-peermem	remote	32, 33	clean
DMA-BUF + `MC_MAX_MR_SIZE=2147483648`	remote	32	clean

The wrap follows whichever side's MR start is aligned. The same buffers registered with ibv_reg_mr are correct, and Mooncake passes the region straight to ibv_reg_dmabuf_mr. So the cause is likely below Mooncake: the kernel's DMA-BUF umem / mlx5 page-size selection for ≥ 4 GiB pages, the NIC firmware, or the GPU driver's DMA-BUF export. It does not look like a Mooncake logic error, but Mooncake users on this path are exposed to it.

ai-dynamo/nixl#2512: the same wrap through UCX's DMA-BUF path, clean with UCX_CUDA_COPY_DMABUF=no. That report's environment lists NIC firmware 32.38.3056; the runs above used 32.43.1014.
vllm-project/vllm#64048: in vLLM disaggregated serving, when the decode KV cache starts on a 2^32 boundary, only the first layers arrive intact.

Why it matters for Mooncake

WITH_NVIDIA_PEERMEM=0 and Data Direct registration both go through this path today.
#4987 would make DMA-BUF the default on hosts without a peer-memory module.
Store segments and KV caches are often registered as large single allocations.

Workaround and possible mitigation

Workaround: MC_MAX_MR_SIZE at or below 4 GiB, which splits large buffers into several MRs (Mooncake#2017). It removed the corruption here. A wrap period of 2^32 or more can't show up inside a 2 GiB MR, but this only avoids the symptom; it isn't a root-cause fix.
Possible mitigation in Mooncake: when registering through DMA-BUF, cap each MR below 4 GiB by default (at least on GH200 / ATS hosts), or warn when a DMA-BUF MR starts on a 2^32 boundary and is longer than 4 GiB.

I can turn either into a PR and test it on the same hardware if that direction is acceptable.

AI assistance disclosure

The investigation and test scripts were prepared with AI assistance (Claude); I reviewed them and ran the tests on the hardware above.

Originally created by @0z5a on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/5015 ### Describe the bug On GH200, a GPU buffer registered through Mooncake's DMA-BUF path (`WITH_NVIDIA_PEERMEM=0`, i.e. `exportDmabuf()` → `cuMemGetHandleForAddressRange` → `ibv_reg_dmabuf_mr`) misbehaves when the MR's start address is exactly 2^k-aligned with k ≥ 32: - The MR behaves as if it wraps every 2^k bytes. - RDMA READ/WRITE at offset X ≥ 2^k actually reaches X mod 2^k. - Every transfer completes with rc = 0, and nothing is logged. The same buffers registered with `ibv_reg_mr` (nvidia-peermem, the default `WITH_NVIDIA_PEERMEM=1`) are correct. Large `cudaMalloc` allocations land on such boundaries in ordinary use. A 16 GiB allocation at `0xf1b700000000`, which is 2^32-aligned, corrupted a plain run with no forced alignment. So any process on the DMA-BUF path can hit this, intermittently, depending on where its allocation lands. ### Results Setup: - GH200, driver 595.58.03, CUDA 13.2, kernel 6.8.0-136-generic, mlx5 NIC (MT41692, firmware 32.43.1014), upstream `main` `829d95ff`. - RDMA loopback through the NIC, two processes, each with a 12 GiB GPU MR. - The start is forced to exactly 2^k (not 2^(k+1)) by carving it out of a larger `cudaMalloc`. - Each run moves 1152 scattered 64 KiB blocks by READ and by WRITE, plus transfers that end at the MR's last byte. Everything is compared byte by byte. | registration | aligned side | k | result | |---|---|---|---| | DMA-BUF | remote | 31 | clean | | DMA-BUF | remote | 32 | remote offsets 0–4 GiB: 0 bad; 4–12 GiB: 100% bad (READ 781/1152 blocks bad, 776 all zero) | | DMA-BUF | remote | 33 | remote offsets 0–8 GiB: 0 bad; 8–12 GiB: 100% bad | | DMA-BUF | local | 32 | local offsets ≥ 4 GiB: 100% bad | | nvidia-peermem | remote | 32, 33 | clean | | DMA-BUF + `MC_MAX_MR_SIZE=2147483648` | remote | 32 | clean | The wrap follows whichever side's MR start is aligned. The same buffers registered with `ibv_reg_mr` are correct, and Mooncake passes the region straight to `ibv_reg_dmabuf_mr`. So the cause is likely below Mooncake: the kernel's DMA-BUF umem / mlx5 page-size selection for ≥ 4 GiB pages, the NIC firmware, or the GPU driver's DMA-BUF export. It does not look like a Mooncake logic error, but Mooncake users on this path are exposed to it. ### Related reports - ai-dynamo/nixl#2512: the same wrap through UCX's DMA-BUF path, clean with `UCX_CUDA_COPY_DMABUF=no`. That report's environment lists NIC firmware 32.38.3056; the runs above used 32.43.1014. - vllm-project/vllm#64048: in vLLM disaggregated serving, when the decode KV cache starts on a 2^32 boundary, only the first layers arrive intact. ### Why it matters for Mooncake - `WITH_NVIDIA_PEERMEM=0` and Data Direct registration both go through this path today. - #4987 would make DMA-BUF the default on hosts without a peer-memory module. - Store segments and KV caches are often registered as large single allocations. ### Workaround and possible mitigation - **Workaround:** `MC_MAX_MR_SIZE` at or below 4 GiB, which splits large buffers into several MRs (Mooncake#2017). It removed the corruption here. A wrap period of 2^32 or more can't show up inside a 2 GiB MR, but this only avoids the symptom; it isn't a root-cause fix. - **Possible mitigation in Mooncake:** when registering through DMA-BUF, cap each MR below 4 GiB by default (at least on GH200 / ATS hosts), or warn when a DMA-BUF MR starts on a 2^32 boundary and is longer than 4 GiB. I can turn either into a PR and test it on the same hardware if that direction is acceptable. ### AI assistance disclosure The investigation and test scripts were prepared with AI assistance (Claude); I reviewed them and ran the tests on the hardware above.

bsakaguchi commented

2026-10-29 21:24:41 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 27, 2026):

Thanks for opening this issue, @0z5a!

Field	Value
Issue	#5015
GitHub user ID	`192209249`
Reporter	@0z5a

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Oct 27, 2026): Thanks for opening this issue, @0z5a! | Field | Value | |-------|-------| | **Issue** | #5015 | | **GitHub user ID** | `192209249` | | **Reporter** | @0z5a | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:24:41 +00:00

Author

Owner

@0z5a commented on GitHub (Oct 28, 2026):

Follow-up: I ran the same test on an x86 host. With the same VA alignment, it is clean there.

Setup:

B300 (x86_64), driver 610.57.04, ConnectX-7 firmware 28.46.5500.
The same RDMA software stack as the GH200 run: kernel 6.8.0-136-generic, with the DOCA-OFED 26.01 mlx5_core / mlx5_ib / ib_core modules (DKMS, not the in-tree drivers) and rdma-core 2601.0.7.
No nvidia_peermem loaded on this host.
The Linux CI wheel of main at fe4f297a, which includes #4987 and #4998.
RDMA loopback through one IB port, with a 12 GiB GPU MR on each side. The start is forced exactly 2^k-aligned (not 2^(k+1)), as before.
Each run: 1152 scattered 64 KiB blocks by READ and by WRITE, plus transfers that end at the MR's last byte. Every byte is compared.

registration	aligned side	k	result
DMA-BUF	remote	31	clean
DMA-BUF	remote	32	clean
DMA-BUF	remote	33	clean
DMA-BUF	local	32	clean
default config (`WITH_NVIDIA_PEERMEM` unset; after #4987 this host picks DMA-BUF)	remote	32	clean
DMA-BUF + `MC_MAX_MR_SIZE=2147483648`	remote	32	clean (6 MRs)
`ibv_reg_mr` (`WITH_NVIDIA_PEERMEM=1`)	remote	32	registration fails with `Bad address [14]`, as expected without a peer-memory module

In every DMA-BUF row except the MC_MAX_MR_SIZE one, each side registered a single 12 GiB MR; the log shows no auto-split. So forcing the MR's VA onto a 2^k boundary is not enough to trigger the wrap on this x86 host, on the same kernel and RDMA modules as the GH200 run.

That does not rule out the kernel or mlx5 large-page path:

When the kernel picks the page size for a DMA-BUF MR, it looks at both the VA and the DMA addresses in the exporter's scatter list.
On x86 the NIC reaches GPU memory through BAR1. The DMA addresses behind this MR therefore need not share the VA's alignment, and this test can neither force nor observe them.
On GH200 the coherent export may hand back addresses that do share it.

So the trigger probably needs the DMA addresses to be 2^k-aligned as well. Today's exposure looks narrowed to GH200-class hosts; the root cause is still open.

#4987 is relevant here: a GH200 host without nvidia_peermem now takes the DMA-BUF path by default. That makes the cap or warning suggested above more useful for GH200.

AI assistance: the test scripts and this summary were prepared with Claude; I reviewed them and ran the tests on the hardware above.

@0z5a commented on GitHub (Oct 28, 2026): Follow-up: I ran the same test on an x86 host. With the same VA alignment, it is clean there. Setup: - B300 (x86_64), driver 610.57.04, ConnectX-7 firmware 28.46.5500. - The same RDMA software stack as the GH200 run: kernel 6.8.0-136-generic, with the DOCA-OFED 26.01 `mlx5_core` / `mlx5_ib` / `ib_core` modules (DKMS, not the in-tree drivers) and rdma-core 2601.0.7. - No `nvidia_peermem` loaded on this host. - The Linux CI wheel of `main` at `fe4f297a`, which includes #4987 and #4998. - RDMA loopback through one IB port, with a 12 GiB GPU MR on each side. The start is forced exactly 2^k-aligned (not 2^(k+1)), as before. - Each run: 1152 scattered 64 KiB blocks by READ and by WRITE, plus transfers that end at the MR's last byte. Every byte is compared. | registration | aligned side | k | result | |---|---|---|---| | DMA-BUF | remote | 31 | clean | | DMA-BUF | remote | 32 | clean | | DMA-BUF | remote | 33 | clean | | DMA-BUF | local | 32 | clean | | default config (`WITH_NVIDIA_PEERMEM` unset; after #4987 this host picks DMA-BUF) | remote | 32 | clean | | DMA-BUF + `MC_MAX_MR_SIZE=2147483648` | remote | 32 | clean (6 MRs) | | `ibv_reg_mr` (`WITH_NVIDIA_PEERMEM=1`) | remote | 32 | registration fails with `Bad address [14]`, as expected without a peer-memory module | In every DMA-BUF row except the `MC_MAX_MR_SIZE` one, each side registered a single 12 GiB MR; the log shows no auto-split. So forcing the MR's VA onto a 2^k boundary is not enough to trigger the wrap on this x86 host, on the same kernel and RDMA modules as the GH200 run. That does not rule out the kernel or mlx5 large-page path: - When the kernel picks the page size for a DMA-BUF MR, it looks at both the VA and the DMA addresses in the exporter's scatter list. - On x86 the NIC reaches GPU memory through BAR1. The DMA addresses behind this MR therefore need not share the VA's alignment, and this test can neither force nor observe them. - On GH200 the coherent export may hand back addresses that do share it. So the trigger probably needs the DMA addresses to be 2^k-aligned as well. Today's exposure looks narrowed to GH200-class hosts; the root cause is still open. #4987 is relevant here: a GH200 host without `nvidia_peermem` now takes the DMA-BUF path by default. That makes the cap or warning suggested above more useful for GH200. AI assistance: the test scripts and this summary were prepared with Claude; I reviewed them and ran the tests on the hardware above.

bsakaguchi referenced this issue

2026-10-29 21:32:57 +00:00

[PR #1270] [CLOSED] [TE/topology]Implement PCIe Root Complex detection for optimal GPU-InfiniBand pairing #2122