kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #1766] [Bug]: Failed to modify QP to RTR with Yunsilicon XScale (xsc-pci) RDMA NIC in SGLang PD disaggregation #510

New issue

Closed

opened 2026-10-29 21:15:40 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-29 21:15:40 +00:00

Owner

Originally created by @PaggyZhang on GitHub (Mar 29, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1766

Bug Report

[Bug]: Failed to modify QP to RTR with Yunsilicon XScale (xsc-pci) RDMA NIC in SGLang PD disaggregation

Environment

Mooncake version: Built from source (bundled in SGLang image, sglang:maca-250822-deepep-250917-mcoplib-0923-stepmesh-0.0.1.dev0-source-c77e9e80-driver)
SGLang version: Source build (commit c77e9e80)
OS: Ubuntu (container on Kubernetes / Volcano Job)
Python: 3.10
RDMA NIC: Yunsilicon XScale (driver: xsc-pci)
- 4x NICs per node: xscale_0, xscale_1, xscale_2, xscale_3
- Link layer: Ethernet (RoCEv2)
- max_mtu: 4096, active_mtu: 4096
- GID index 5 → IPv4-mapped addresses (172.23.x.x/22)
GPU: MetaX (Muxi) GPU × 8 per node
Topology: SGLang disaggregated prefill-decode with mooncake transfer backend
- Prefill: 2 nodes, TP=16
- Decode: 4 nodes, TP=16, DP=2
- Metadata: Pure HTTP KV server (P2PHANDSHAKE mode)

ibv_devinfo output

hca_id: xscale_0
    transport:                      InfiniBand (0)
    fw_ver:                         [vendor firmware]
    vendor_id:                      [vendor_id]
    phys_port_cnt:                  1
        port:   1
            state:                  PORT_ACTIVE (4)
            max_mtu:                4096 (5)
            active_mtu:             4096 (5)
            sm_lid:                 0
            port_lid:               0
            link_layer:             Ethernet

GID index 5: 0000:0000:0000:0000:0000:ffff:ac17:abd1
(= ::ffff:172.23.171.209, IPv4-mapped)

Network topology

All 6 nodes have 4x xscale NICs, each on a different /22 subnet:
- xscale_0 → 172.23.168.0/22 (net1)
- xscale_1 → 172.24.168.0/22 (net2)
- xscale_2 → 172.25.168.0/22 (net3)
- xscale_3 → 172.26.168.0/22 (net4)
All 172.23.x.x IPs are present as OS-level interfaces (ip addr show confirms inet 172.23.x.x/22 scope global net1)
Full-mesh RDMA RC connectivity test (480 pairs, 6 nodes × 4 NICs) passed independently using ib_write_bw / custom RC test

Problem

When SGLang uses mooncake Transfer Engine as the KV cache transfer backend for prefill-decode disaggregation, the RDMA QP handshake fails with Invalid argument [22] during ibv_modify_qp(INIT→RTR).

This happens even with single NIC (device_name: "xscale_0") and same subnet (both endpoints on 172.23.x.x/22).

Mooncake configuration

{
    "local_hostname": "172.23.171.209",
    "metadata_server": "http://metadata-host:8001",
    "protocol": "rdma",
    "device_name": "xscale_0",
    "gid_index": 5
}

Environment variables:

MC_LOCAL_HOSTNAME=172.23.171.209
MC_GID_INDEX=5
MC_NUM_WORKER_THREADS=4

Error logs

Decode side (rdma_endpoint.cpp:425)

E0329 14:34:42.310150  2793 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]
E0329 14:34:42.310698  2829 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]
E0329 14:34:42.313689  2820 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]
E0329 14:34:42.317382  2792 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22]

Prefill side (rdma_endpoint.cpp:157)

E0329 14:34:42.310346  2179 rdma_endpoint.cpp:157] Invalid argument: received packet mismatch,
    local.local_nic_path: 172.23.171.209:15473@xscale_0,
    local.peer_nic_path: 172.23.171.154:15006@xscale_0,
    peer.local_nic_path: ,
    peer.peer_nic_path: 
E0329 14:34:42.310905  2183 rdma_endpoint.cpp:157] Invalid argument: received packet mismatch,
    local.local_nic_path: 172.23.171.209:15513@xscale_0,
    local.peer_nic_path: 172.23.171.154:16138@xscale_0,
    peer.local_nic_path: ,
    peer.peer_nic_path:

Key observation: peer.local_nic_path and peer.peer_nic_path are empty, meaning the P2P handshake metadata exchange is incomplete — the decode side fails before it can send its NIC path back to the prefill side.

Earlier in startup — MR registration also fails

W0329 10:35:15.309159  rdma_context.cpp:203] The buffer length exceeds device max_mr_size, shrink it to 1073741824: No space left on device [28]
E0329 10:35:15.541724  rdma_context.cpp:246] Failed to register memory 0x55bdb29ee3d0: Invalid argument [22]
E0329 10:35:15.606554  rdma_context.cpp:246] Failed to register memory 0x7f4fcf9ff010: Invalid argument [22]

Both heap addresses (0x55...) and mmap addresses (0x7f...) fail. The buffer is first shrunk from its original size to 1GB (device max_mr_size limit), but ibv_reg_mr still returns EINVAL.

Note: Despite MR registration failures, the SGLang server starts and serves HTTP requests. The failures surface when actual KV transfer is attempted.

What we've verified

RDMA basic connectivity works: Full-mesh test of 480 RDMA RC connections across 6 nodes × 4 NICs all passed using standalone tools (not mooncake).
Network reachability: 172.23.x.x IPs exist as OS-level interfaces on both prefill and decode pods. ip route confirms same /22 subnet routing.
ulimit: ulimit -l unlimited confirmed — not a memlock limit issue.
MTU: Both sides report active_mtu: 4096, max_mtu: 4096 — MTU is consistent.
GID index: GID index 5 resolves to valid IPv4-mapped addresses on both sides.
Single NIC: Restricted to device_name: "xscale_0" only — same result.
Multi NIC: Also tested with device_name: "xscale_0,xscale_1,xscale_2,xscale_3" — additional cross-subnet QP failures as expected (different subnets), but the same-subnet QP (xscale_0↔xscale_0) also failed.

Analysis

The xsc-pci driver passes standalone ib_write_bw tests but fails when mooncake creates QPs. This suggests that mooncake's ibv_modify_qp call to transition QP from INIT→RTR uses parameters that are not accepted by the xsc-pci driver. Possible causes:

QP attribute incompatibility: The xsc-pci driver may not support certain ibv_qp_attr values that mooncake sets during INIT→RTR transition (e.g., max_rd_atomic, min_rnr_timer, retry_cnt, rnr_retry, timeout, path_mtu, or ah_attr fields).
GID format handling: The driver may expect a specific GID type (RoCEv1 vs RoCEv2) that differs from what mooncake derives from GID index 5.
MR registration rejection: The ibv_reg_mr failure (EINVAL on both heap and mmap) may indicate the driver has stricter requirements for memory registration flags or alignment.

Expected behavior

QP should transition to RTR successfully, enabling RDMA-based KV cache transfer between prefill and decode instances.

Workaround attempted

Using protocol: "tcp" instead of "rdma" in mooncake config as a temporary workaround to bypass RDMA QP creation entirely.

Request

Could you confirm whether Yunsilicon XScale (xsc-pci) is a tested/supported RDMA device for mooncake Transfer Engine?
Are there any known QP attribute requirements or environment variable overrides (beyond MC_MTU, MC_GID_INDEX) that could help with non-Mellanox RDMA NICs?
Would it be possible to add debug logging in rdma_endpoint.cpp around the ibv_modify_qp call to print the exact ibv_qp_attr values being used (mtu, gid, dest_qp_num, rq_psn, max_rd_atomic, etc.)? This would greatly help diagnose driver-specific incompatibilities.

Mooncake Troubleshooting doc mentions checking MTU/GID for RTR failures, but does not cover non-Mellanox NIC compatibility.
SGLang disaggregation with mooncake backend: sglang.srt.disaggregation.mooncake

Before submitting...

Ensure you searched for relevant issues and read the [documentation]

Originally created by @PaggyZhang on GitHub (Mar 29, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/1766 ### Bug Report # [Bug]: Failed to modify QP to RTR with Yunsilicon XScale (xsc-pci) RDMA NIC in SGLang PD disaggregation ## Environment - **Mooncake version**: Built from source (bundled in SGLang image, `sglang:maca-250822-deepep-250917-mcoplib-0923-stepmesh-0.0.1.dev0-source-c77e9e80-driver`) - **SGLang version**: Source build (commit `c77e9e80`) - **OS**: Ubuntu (container on Kubernetes / Volcano Job) - **Python**: 3.10 - **RDMA NIC**: Yunsilicon XScale (driver: `xsc-pci`) - 4x NICs per node: `xscale_0`, `xscale_1`, `xscale_2`, `xscale_3` - Link layer: Ethernet (RoCEv2) - max_mtu: 4096, active_mtu: 4096 - GID index 5 → IPv4-mapped addresses (172.23.x.x/22) - **GPU**: MetaX (Muxi) GPU × 8 per node - **Topology**: SGLang disaggregated prefill-decode with mooncake transfer backend - Prefill: 2 nodes, TP=16 - Decode: 4 nodes, TP=16, DP=2 - Metadata: Pure HTTP KV server (P2PHANDSHAKE mode) ## ibv_devinfo output ``` hca_id: xscale_0 transport: InfiniBand (0) fw_ver: [vendor firmware] vendor_id: [vendor_id] phys_port_cnt: 1 port: 1 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 link_layer: Ethernet GID index 5: 0000:0000:0000:0000:0000:ffff:ac17:abd1 (= ::ffff:172.23.171.209, IPv4-mapped) ``` ## Network topology - All 6 nodes have 4x xscale NICs, each on a different /22 subnet: - `xscale_0` → `172.23.168.0/22` (net1) - `xscale_1` → `172.24.168.0/22` (net2) - `xscale_2` → `172.25.168.0/22` (net3) - `xscale_3` → `172.26.168.0/22` (net4) - All 172.23.x.x IPs are present as OS-level interfaces (`ip addr show` confirms `inet 172.23.x.x/22 scope global net1`) - Full-mesh RDMA RC connectivity test (480 pairs, 6 nodes × 4 NICs) passed independently using `ib_write_bw` / custom RC test ## Problem When SGLang uses mooncake Transfer Engine as the KV cache transfer backend for prefill-decode disaggregation, the RDMA QP handshake fails with `Invalid argument [22]` during `ibv_modify_qp(INIT→RTR)`. This happens even with **single NIC** (`device_name: "xscale_0"`) and **same subnet** (both endpoints on 172.23.x.x/22). ## Mooncake configuration ```json { "local_hostname": "172.23.171.209", "metadata_server": "http://metadata-host:8001", "protocol": "rdma", "device_name": "xscale_0", "gid_index": 5 } ``` Environment variables: ``` MC_LOCAL_HOSTNAME=172.23.171.209 MC_GID_INDEX=5 MC_NUM_WORKER_THREADS=4 ``` ## Error logs ### Decode side (rdma_endpoint.cpp:425) ``` E0329 14:34:42.310150 2793 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22] E0329 14:34:42.310698 2829 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22] E0329 14:34:42.313689 2820 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22] E0329 14:34:42.317382 2792 rdma_endpoint.cpp:425] [Handshake] Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num: Invalid argument [22] ``` ### Prefill side (rdma_endpoint.cpp:157) ``` E0329 14:34:42.310346 2179 rdma_endpoint.cpp:157] Invalid argument: received packet mismatch, local.local_nic_path: 172.23.171.209:15473@xscale_0, local.peer_nic_path: 172.23.171.154:15006@xscale_0, peer.local_nic_path: , peer.peer_nic_path: E0329 14:34:42.310905 2183 rdma_endpoint.cpp:157] Invalid argument: received packet mismatch, local.local_nic_path: 172.23.171.209:15513@xscale_0, local.peer_nic_path: 172.23.171.154:16138@xscale_0, peer.local_nic_path: , peer.peer_nic_path: ``` Key observation: **`peer.local_nic_path` and `peer.peer_nic_path` are empty**, meaning the P2P handshake metadata exchange is incomplete — the decode side fails before it can send its NIC path back to the prefill side. ### Earlier in startup — MR registration also fails ``` W0329 10:35:15.309159 rdma_context.cpp:203] The buffer length exceeds device max_mr_size, shrink it to 1073741824: No space left on device [28] E0329 10:35:15.541724 rdma_context.cpp:246] Failed to register memory 0x55bdb29ee3d0: Invalid argument [22] E0329 10:35:15.606554 rdma_context.cpp:246] Failed to register memory 0x7f4fcf9ff010: Invalid argument [22] ``` Both heap addresses (`0x55...`) and mmap addresses (`0x7f...`) fail. The buffer is first shrunk from its original size to 1GB (device `max_mr_size` limit), but `ibv_reg_mr` still returns EINVAL. Note: Despite MR registration failures, the SGLang server starts and serves HTTP requests. The failures surface when actual KV transfer is attempted. ## What we've verified 1. **RDMA basic connectivity works**: Full-mesh test of 480 RDMA RC connections across 6 nodes × 4 NICs all passed using standalone tools (not mooncake). 2. **Network reachability**: 172.23.x.x IPs exist as OS-level interfaces on both prefill and decode pods. `ip route` confirms same /22 subnet routing. 3. **ulimit**: `ulimit -l unlimited` confirmed — not a memlock limit issue. 4. **MTU**: Both sides report `active_mtu: 4096`, `max_mtu: 4096` — MTU is consistent. 5. **GID index**: GID index 5 resolves to valid IPv4-mapped addresses on both sides. 6. **Single NIC**: Restricted to `device_name: "xscale_0"` only — same result. 7. **Multi NIC**: Also tested with `device_name: "xscale_0,xscale_1,xscale_2,xscale_3"` — additional cross-subnet QP failures as expected (different subnets), but the same-subnet QP (xscale_0↔xscale_0) also failed. ## Analysis The xsc-pci driver passes standalone `ib_write_bw` tests but fails when mooncake creates QPs. This suggests that mooncake's `ibv_modify_qp` call to transition QP from INIT→RTR uses parameters that are not accepted by the xsc-pci driver. Possible causes: 1. **QP attribute incompatibility**: The xsc-pci driver may not support certain `ibv_qp_attr` values that mooncake sets during INIT→RTR transition (e.g., `max_rd_atomic`, `min_rnr_timer`, `retry_cnt`, `rnr_retry`, `timeout`, `path_mtu`, or `ah_attr` fields). 2. **GID format handling**: The driver may expect a specific GID type (RoCEv1 vs RoCEv2) that differs from what mooncake derives from GID index 5. 3. **MR registration rejection**: The `ibv_reg_mr` failure (EINVAL on both heap and mmap) may indicate the driver has stricter requirements for memory registration flags or alignment. ## Expected behavior QP should transition to RTR successfully, enabling RDMA-based KV cache transfer between prefill and decode instances. ## Workaround attempted - Using `protocol: "tcp"` instead of `"rdma"` in mooncake config as a temporary workaround to bypass RDMA QP creation entirely. ## Request 1. Could you confirm whether Yunsilicon XScale (`xsc-pci`) is a tested/supported RDMA device for mooncake Transfer Engine? 2. Are there any known QP attribute requirements or environment variable overrides (beyond `MC_MTU`, `MC_GID_INDEX`) that could help with non-Mellanox RDMA NICs? 3. Would it be possible to add debug logging in `rdma_endpoint.cpp` around the `ibv_modify_qp` call to print the exact `ibv_qp_attr` values being used (mtu, gid, dest_qp_num, rq_psn, max_rd_atomic, etc.)? This would greatly help diagnose driver-specific incompatibilities. ## Related - Mooncake Troubleshooting doc mentions checking MTU/GID for RTR failures, but does not cover non-Mellanox NIC compatibility. - SGLang disaggregation with mooncake backend: `sglang.srt.disaggregation.mooncake` ### Before submitting... - [ ] Ensure you searched for relevant issues and read the [documentation]

bsakaguchi

2026-10-29 21:15:40 +00:00

closed this issue
added the
MetaX

bug
labels

bsakaguchi commented

2026-10-29 21:15:41 +00:00

Author

Owner

@alogfans commented on GitHub (Mar 30, 2026):

If the Yunsilicon XScale's doc has said that xsc-pci driver may not support certain ibv_qp_attr value, the first step is to vibe a minimized prototype that two nodes can communicate with each other. Can you use regular ib_write_bw testsuite?

@alogfans commented on GitHub (Mar 30, 2026): If the Yunsilicon XScale's doc has said that xsc-pci driver may not support certain ibv_qp_attr value, the first step is to vibe a minimized prototype that two nodes can communicate with each other. Can you use regular ib_write_bw testsuite?

bsakaguchi commented

2026-10-29 21:15:41 +00:00

Author

Owner

@PaggyZhang commented on GitHub (Mar 30, 2026):

We have already verified basic RDMA connectivity using ib_write_bw. A full-mesh test across 6 nodes × 4 NICs (480 RDMA RC connection pairs) all passed successfully using standard perftest tools (ib_write_bw -d xscale_0 -x 5).
This confirms that the xsc-pci driver supports standard QP attributes used by ib_write_bw. The issue only occurs when mooncake's Transfer Engine creates QPs — suggesting that mooncake sets certain ibv_qp_attr values during the INIT→RTR transition that differ from what standard perftest tools use, and those specific values are not accepted by the xsc-pci driver.
If it would help narrow down the incompatibility, we can provide a side-by-side comparison of the QP attributes used by ib_write_bw vs. mooncake, or test with specific ibv_qp_attr overrides if mooncake exposes any configuration for them.

@PaggyZhang commented on GitHub (Mar 30, 2026): We have already verified basic RDMA connectivity using ib_write_bw. A full-mesh test across 6 nodes × 4 NICs (480 RDMA RC connection pairs) all passed successfully using standard perftest tools (ib_write_bw -d xscale_0 -x 5). This confirms that the xsc-pci driver supports standard QP attributes used by ib_write_bw. The issue only occurs when mooncake's Transfer Engine creates QPs — suggesting that mooncake sets certain ibv_qp_attr values during the INIT→RTR transition that differ from what standard perftest tools use, and those specific values are not accepted by the xsc-pci driver. If it would help narrow down the incompatibility, we can provide a side-by-side comparison of the QP attributes used by ib_write_bw vs. mooncake, or test with specific ibv_qp_attr overrides if mooncake exposes any configuration for them.

bsakaguchi commented

2026-10-29 21:15:41 +00:00

Author

Owner

@JianDan0212 commented on GitHub (Apr 2, 2026):

The PD disaggregation feature implemented with Mooncake on vLLM-Metax is working normally, and we have not encountered the handshake failure issue mentioned above.
We are using our internally adapted version 0.3.6 of Mooncake, which has been fully tested and validated across vLLM versions 0.14 to 0.18 without any issues.
The network card information used in our testing is shown in the figure below:

Our startup commands are as follows:

export MC_FORCE_RDMA=1
export MC_ENABLE_DEST_DEVICE_AFFINITY=1
export VLLM_DISABLE_REQUEST_ID_RANDOMIZATION=1

vllm serve $MODEL_PATH \
    --port $PORT \
    --tensor-parallel-size 2 \
    --served-model-name qwen \
    --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer"}'

vllm serve $MODEL_PATH \
    --port $PORT \
    --gpu-memory-utilization 0.9 \
    --tensor-parallel-size 2 \
    --served-model-name qwen \
    --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer"}'

@JianDan0212 commented on GitHub (Apr 2, 2026): The PD disaggregation feature implemented with Mooncake on vLLM-Metax is working normally, and we have not encountered the handshake failure issue mentioned above. We are using our internally adapted version 0.3.6 of Mooncake, which has been fully tested and validated across vLLM versions 0.14 to 0.18 without any issues. The network card information used in our testing is shown in the figure below: <img width="459" height="653" alt="Image" src="https://github.com/user-attachments/assets/5d52e350-d67b-41ad-9995-7630720cb2de" /> Our startup commands are as follows: ``` export MC_FORCE_RDMA=1 export MC_ENABLE_DEST_DEVICE_AFFINITY=1 export VLLM_DISABLE_REQUEST_ID_RANDOMIZATION=1 vllm serve $MODEL_PATH \ --port $PORT \ --tensor-parallel-size 2 \ --served-model-name qwen \ --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer"}' vllm serve $MODEL_PATH \ --port $PORT \ --gpu-memory-utilization 0.9 \ --tensor-parallel-size 2 \ --served-model-name qwen \ --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer"}' ```