[GH-ISSUE #2724] [TE] Cross-node RDMA KV transfer SIGSEGVs on AMD under rdma+hip multi-protocol registration #714

New issue

Closed

opened 2026-10-29 21:18:34 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:18:34 +00:00

Owner

Originally created by @Lzy17 on GitHub (Jul 2, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2724

Environment

Transport engine: mooncake at 01d1eb2a (#2682, "Support rdma+hip multi-protocol segments").
HW: AMD MI3xx (gfx950 / gfx942), RoCE NICs. Reproduced on two independent
fabrics: Broadcom bnxt_re (flat L2) and ionic (routed L3).
Workload: SGLang 1P1D disaggregation, prefill and decode on separate
nodes, GPU KV cache transferred cross-node over RDMA. hip protocol is
registered for intra-node GPU-IPC, rdma for cross-node — the normal
disaggregated-inference setup.
Build: USE_HIP=ON, ENABLE_MULTI_PROTOCOL=ON.

Symptom

Cross-node KV transfer of GPU memory crashes with SIGSEGV during the RDMA
submit path, whenever the same GPU buffer is registered under both hip and
rdma. Single-protocol rdma-only deployments are unaffected.

Two related failure modes depending on which sub-bug fires first:

RdmaTransport::submitTransferTask → SIGSEGV at
slice->rdma.source_lkey = local_segment_desc->buffers[buffer_id].lkey[device_id];
(rdma_transport.cpp) — out-of-bounds read on an empty lkey vector.
When transport selection misfires first: cross-node request is routed to
the hip transport and fails with hipIpcOpenMemHandle failed (hip is
intra-node GPU-IPC only).

The assert(lkey.size() == context_list_.size()) that would have caught (1) is
compiled out in Release builds, so it surfaces as a bare segfault.

Root cause

With multi-protocol registration, a single GPU address ends up with two
buffer descriptors at the same address range:

a hip twin — uses IPC handles / shm_name, carries no lkey/rkey
(the hip metadata (de)serialization intentionally omits them);
an rdma twin — the valid RDMA registration, with a populated lkey.

Single-protocol rdma-only deployments only ever have the second one, so every
lookup is correct. #2682 enabled the dual registration, which is the first time
the RDMA data path is handed two twins at one address — and it does not
disambiguate. This surfaces three independent latent issues:

selectDevice() / selectDeviceByLocalHca() are protocol-blind
(rdma_transport.cpp). They scan the segment's buffers by address range
only and return the first match. Because the hip twin aliases the same
address, they return the hip buffer, whose lkey vector is empty → the
OOB read above. (root cause of the crash)
selectTransport() hardcodes hip priority above rdma
(multi_transport.cpp). For a comma-joined multi-protocol segment the
priority table has hip (4) > rdma (2), so a cross-node request selects
the intra-node-only hip transport → hipIpcOpenMemHandle failed.
USE_HIP_DMABUF define never reaches rdma_context.cpp
(src/CMakeLists.txt). The define is applied PRIVATE to the
transfer_engine target, but the dmabuf MR-registration code lives in
rdma_context.cpp, compiled in the separate rdma_transport OBJECT library
(linked via $<TARGET_OBJECTS:transport>). The define therefore never
reaches that code, the dmabuf path is silently compiled out, and GPU MRs
fall back to plain ibv_reg_mr → EINVAL on device memory.

Diagnosis detail

A dump of the local segment at the crash confirms the dual registration:

buffers 0-47: proto=hip, lkey.size=0
buffers 48-95: proto=rdma, lkey.size=1 — the valid RDMA twins of the
same GPU addresses.

selectDevice returns buffer id 0 (hip, empty lkey) → .lkey[0] on an empty
vector → SIGSEGV.

Scope / who is affected

AMD + multi-protocol only. NV is unaffected. rdma-only / single-protocol
deployments are unaffected (they never have a competing hip twin), which is
why this went unnoticed after #2682 merged.

Fix

See the accompanying PR. Three minimal, gated changes:

In selectDevice / selectDeviceByLocalHca, under ENABLE_MULTI_PROTOCOL,
skip any buffer whose protocol is set and is not rdma (guarded with
!protocol.empty() so legacy single-protocol descriptors are untouched).
Allow hip transport priority to be de-prioritized via MC_DISABLE_HIP for
deployments that need the cross-node RDMA path (default behavior unchanged).
Apply USE_HIP_DMABUF (and the hsa-runtime64 usage requirements) to the
rdma_transport target that actually compiles rdma_context.cpp.

Validation

Before: cross-node /generate SIGSEGVs during KV transfer.
After all three fixes: cross-node /generate correct; concurrency sweep
(1/8/32/64) 100% success, zero QP/transfer failures, zero SIGSEGV. Validated
on both bnxt_re and ionic fabrics.
rdma-only build (ENABLE_MULTI_PROTOCOL=OFF) runs the same workload
unaffected, confirming the bug is specific to multi-protocol dual
registration.

Note for routed-RoCE (L3) fabrics

On routed fabrics (e.g. ionic, NICs on point-to-point /31 subnets) the
deployment must also select a routable GID (RoCEv2 IPv4 GID, e.g.
MC_GID_INDEX=1); otherwise the QP falls back to a link-local GID and the RTR
handshake times out. This is deployment config, not part of the fix, but worth
documenting for AMD multi-node users.

Originally created by @Lzy17 on GitHub (Jul 2, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2724 ## Environment - **Transport engine:** mooncake at `01d1eb2a` (#2682, "Support rdma+hip multi-protocol segments"). - **HW:** AMD MI3xx (gfx950 / gfx942), RoCE NICs. Reproduced on two independent fabrics: Broadcom `bnxt_re` (flat L2) and `ionic` (routed L3). - **Workload:** SGLang 1P1D disaggregation, prefill and decode on **separate nodes**, GPU KV cache transferred cross-node over RDMA. `hip` protocol is registered for intra-node GPU-IPC, `rdma` for cross-node — the normal disaggregated-inference setup. - **Build:** `USE_HIP=ON`, `ENABLE_MULTI_PROTOCOL=ON`. ## Symptom Cross-node KV transfer of GPU memory **crashes with SIGSEGV** during the RDMA submit path, whenever the same GPU buffer is registered under both `hip` and `rdma`. Single-protocol rdma-only deployments are unaffected. Two related failure modes depending on which sub-bug fires first: 1. `RdmaTransport::submitTransferTask` → SIGSEGV at `slice->rdma.source_lkey = local_segment_desc->buffers[buffer_id].lkey[device_id];` (`rdma_transport.cpp`) — out-of-bounds read on an empty `lkey` vector. 2. When transport *selection* misfires first: cross-node request is routed to the `hip` transport and fails with `hipIpcOpenMemHandle failed` (hip is intra-node GPU-IPC only). The `assert(lkey.size() == context_list_.size())` that would have caught (1) is compiled out in Release builds, so it surfaces as a bare segfault. ## Root cause With multi-protocol registration, a single GPU address ends up with **two** buffer descriptors at the same address range: - a `hip` twin — uses IPC handles / `shm_name`, carries **no** `lkey`/`rkey` (the `hip` metadata (de)serialization intentionally omits them); - an `rdma` twin — the valid RDMA registration, with a populated `lkey`. Single-protocol rdma-only deployments only ever have the second one, so every lookup is correct. #2682 enabled the dual registration, which is the first time the RDMA data path is handed two twins at one address — and it does not disambiguate. This surfaces three independent latent issues: 1. **`selectDevice()` / `selectDeviceByLocalHca()` are protocol-blind** (`rdma_transport.cpp`). They scan the segment's buffers **by address range only** and return the first match. Because the `hip` twin aliases the same address, they return the `hip` buffer, whose `lkey` vector is empty → the OOB read above. **(root cause of the crash)** 2. **`selectTransport()` hardcodes `hip` priority above `rdma`** (`multi_transport.cpp`). For a comma-joined multi-protocol segment the priority table has `hip` (4) > `rdma` (2), so a cross-node request selects the intra-node-only `hip` transport → `hipIpcOpenMemHandle failed`. 3. **`USE_HIP_DMABUF` define never reaches `rdma_context.cpp`** (`src/CMakeLists.txt`). The define is applied `PRIVATE` to the `transfer_engine` target, but the dmabuf MR-registration code lives in `rdma_context.cpp`, compiled in the separate `rdma_transport` OBJECT library (linked via `$<TARGET_OBJECTS:transport>`). The define therefore never reaches that code, the dmabuf path is silently compiled out, and GPU MRs fall back to plain `ibv_reg_mr` → `EINVAL` on device memory. ## Diagnosis detail A dump of the local segment at the crash confirms the dual registration: - buffers 0-47: `proto=hip`, `lkey.size=0` - buffers 48-95: `proto=rdma`, `lkey.size=1` — the valid RDMA twins of the **same** GPU addresses. `selectDevice` returns buffer id 0 (hip, empty lkey) → `.lkey[0]` on an empty vector → SIGSEGV. ## Scope / who is affected - **AMD + multi-protocol only.** NV is unaffected. rdma-only / single-protocol deployments are unaffected (they never have a competing hip twin), which is why this went unnoticed after #2682 merged. ## Fix See the accompanying PR. Three minimal, gated changes: 1. In `selectDevice` / `selectDeviceByLocalHca`, under `ENABLE_MULTI_PROTOCOL`, skip any buffer whose protocol is set and is not `rdma` (guarded with `!protocol.empty()` so legacy single-protocol descriptors are untouched). 2. Allow `hip` transport priority to be de-prioritized via `MC_DISABLE_HIP` for deployments that need the cross-node RDMA path (default behavior unchanged). 3. Apply `USE_HIP_DMABUF` (and the `hsa-runtime64` usage requirements) to the `rdma_transport` target that actually compiles `rdma_context.cpp`. ## Validation - Before: cross-node `/generate` SIGSEGVs during KV transfer. - After all three fixes: cross-node `/generate` correct; concurrency sweep (1/8/32/64) 100% success, zero QP/transfer failures, zero SIGSEGV. Validated on both `bnxt_re` and `ionic` fabrics. - rdma-only build (`ENABLE_MULTI_PROTOCOL=OFF`) runs the same workload unaffected, confirming the bug is specific to multi-protocol dual registration. ## Note for routed-RoCE (L3) fabrics On routed fabrics (e.g. ionic, NICs on point-to-point `/31` subnets) the deployment must also select a routable GID (RoCEv2 IPv4 GID, e.g. `MC_GID_INDEX=1`); otherwise the QP falls back to a link-local GID and the RTR handshake times out. This is deployment config, not part of the fix, but worth documenting for AMD multi-node users.

bsakaguchi closed this issue

2026-10-29 21:18:35 +00:00

bsakaguchi commented

2026-10-29 21:18:35 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 2, 2026):

Thanks for opening this issue, @Lzy17!

Field	Value
Issue	#2724
GitHub user ID	`36555117`
Reporter	@Lzy17

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jul 2, 2026): Thanks for opening this issue, @Lzy17! | Field | Value | |-------|-------| | **Issue** | #2724 | | **GitHub user ID** | `36555117` | | **Reporter** | @Lzy17 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi referenced this issue

2026-10-29 21:30:10 +00:00

[PR #714] [MERGED] [TransferEngine] Update to support CANN 8.2.RC1 #1766