kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #4979] [Bug][TE]: `nvlink_intra` — `transfer_*_on_cuda` always kills the process; on hosts without GPU P2P a mixed batch fails silently for its valid entries #1261

New issue

Open

opened 2026-10-29 21:24:35 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:24:35 +00:00

Owner

Originally created by @0z5a on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4979

Environment: 2× NVIDIA L40S (PCIe, P2P disabled: nvidia-smi topo -p2p r = NS between the GPUs), driver 580, CUDA 13.3, x86 nightly wheel mooncake-transfer-engine-cuda13 0.3.14.dev20261025 (main d53ddb9f), MC_INTRANODE_NVLINK=1, P2PHANDSHAKE, one initiator process and one target process.

A. `transfer_{write,read}_on_cuda` + `nvlink_intra` → `_exit(1)`, even on a single GPU

Same-GPU write (target buffer and source on GPU 0, no P2P involved):

E intranode_nvlink_transport.cpp:63] IntraNodeNvlinkTransport: cudaSetDevice(0) failed
E transfer_engine_py.cpp:1459] [Mooncake Cuda] Submit failed: Context: Failed to create NVLink CUDA stream | BatchID: ...

then the process exits with status 1 (by design of transfer_on_cuda_callback's error path). The plain transfer_sync_write of the same buffers succeeds.

Cause (main): submitTransfer runs inside the cudaLaunchHostFunc callback. IntraNodeNvlinkTransport gets its copy stream and event from thread-local pools (PerDeviceStreamPool::getOrCreate, tl_device_stream_pool / tl_device_event_pool). The callback runs on the CUDA driver's callback thread, whose pools are always empty, so every call lazily runs cudaSetDevice + cudaStreamCreateWithFlags inside the host function, which CUDA forbids (host functions must not call CUDA APIs). The later cudaEventRecord / cudaEventSynchronize in submitTransferTask would hit the same restriction. So this combination cannot work on any hardware, NVLink or not.

B. No P2P: misleading error, valid entries dropped, batch leaked

Cross-GPU target: cudaIpcOpenMemHandle failed: peer access is not supported between these two devices, but the caller gets Memory: device memory not registered.
Batch [same-GPU target, cross-GPU target] (either order) via batch_transfer_sync_write/read: rc = -1, the valid same-GPU entry is not copied (destination stays all zero), and multi_transport.cpp: Batch ... is still busy; cleanup has been deferred. submitTransferTask returns in the middle of the loop after earlier slices were created but before submitBatchMemcpy.
P2P is never checked or enabled: enableP2PAccess() exists but its only call site is commented out.

Expected: check cudaDeviceCanAccessPeer up front and either fall back (staged copy / another transport) or reject the whole batch before creating any slice, with an error that names the real cause.

Results (one case per process, 40 s watchdog)

case	result
same-GPU write / read (sync API)	OK / OK
in-process GPU0→GPU1 copy, no IPC	OK
cross-GPU write	FAILED (misleading error)
mixed / mixed reversed write, mixed read	FAILED, valid entry not copied, batch cleanup deferred
same-GPU `transfer_write_on_cuda`	process `_exit(1)`
mixed `transfer_write_on_cuda`	process `_exit(1)`

Open PR #4919 (VMM buffers for nvlink_intra) edits the same file but does not touch the thread-local stream/event pools, submitTransferTask's early return, or enableP2PAccess, so these findings are independent of it. (Open-PR search for on_cuda / nvlink_intra P2P / cuda IPC found nothing else.)

AI assistance disclosure

Found by AI-assisted code reading (Claude) and confirmed with test scripts on the hardware listed above.

Originally created by @0z5a on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/4979 Environment: 2× NVIDIA L40S (PCIe, P2P disabled: `nvidia-smi topo -p2p r` = NS between the GPUs), driver 580, CUDA 13.3, x86 nightly wheel `mooncake-transfer-engine-cuda13 0.3.14.dev20261025` (main `d53ddb9f`), `MC_INTRANODE_NVLINK=1`, P2PHANDSHAKE, one initiator process and one target process. ### A. `transfer_{write,read}_on_cuda` + `nvlink_intra` → `_exit(1)`, even on a single GPU Same-GPU write (target buffer and source on GPU 0, no P2P involved): ``` E intranode_nvlink_transport.cpp:63] IntraNodeNvlinkTransport: cudaSetDevice(0) failed E transfer_engine_py.cpp:1459] [Mooncake Cuda] Submit failed: Context: Failed to create NVLink CUDA stream | BatchID: ... ``` then the process exits with status 1 (by design of `transfer_on_cuda_callback`'s error path). The plain `transfer_sync_write` of the same buffers succeeds. Cause (main): `submitTransfer` runs inside the `cudaLaunchHostFunc` callback. `IntraNodeNvlinkTransport` gets its copy stream and event from **thread-local** pools (`PerDeviceStreamPool::getOrCreate`, `tl_device_stream_pool` / `tl_device_event_pool`). The callback runs on the CUDA driver's callback thread, whose pools are always empty, so every call lazily runs `cudaSetDevice` + `cudaStreamCreateWithFlags` inside the host function, which CUDA forbids (host functions must not call CUDA APIs). The later `cudaEventRecord` / `cudaEventSynchronize` in `submitTransferTask` would hit the same restriction. So this combination cannot work on any hardware, NVLink or not. ### B. No P2P: misleading error, valid entries dropped, batch leaked - Cross-GPU target: `cudaIpcOpenMemHandle failed: peer access is not supported between these two devices`, but the caller gets `Memory: device memory not registered`. - Batch [same-GPU target, cross-GPU target] (either order) via `batch_transfer_sync_write`/`read`: rc = -1, the **valid** same-GPU entry is not copied (destination stays all zero), and `multi_transport.cpp: Batch ... is still busy; cleanup has been deferred`. `submitTransferTask` returns in the middle of the loop after earlier slices were created but before `submitBatchMemcpy`. - P2P is never checked or enabled: `enableP2PAccess()` exists but its only call site is commented out. Expected: check `cudaDeviceCanAccessPeer` up front and either fall back (staged copy / another transport) or reject the whole batch before creating any slice, with an error that names the real cause. ### Results (one case per process, 40 s watchdog) | case | result | |---|---| | same-GPU write / read (sync API) | OK / OK | | in-process GPU0→GPU1 copy, no IPC | OK | | cross-GPU write | FAILED (misleading error) | | mixed / mixed reversed write, mixed read | FAILED, valid entry not copied, batch cleanup deferred | | same-GPU `transfer_write_on_cuda` | process `_exit(1)` | | mixed `transfer_write_on_cuda` | process `_exit(1)` | ### Related Open PR #4919 (VMM buffers for `nvlink_intra`) edits the same file but does not touch the thread-local stream/event pools, `submitTransferTask`'s early return, or `enableP2PAccess`, so these findings are independent of it. (Open-PR search for on_cuda / nvlink_intra P2P / cuda IPC found nothing else.) ### AI assistance disclosure Found by AI-assisted code reading (Claude) and confirmed with test scripts on the hardware listed above.

bsakaguchi commented

2026-10-29 21:24:36 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 27, 2026):

Thanks for opening this issue, @0z5a!

Field	Value
Issue	#4979
GitHub user ID	`192209249`
Reporter	@0z5a

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Oct 27, 2026): Thanks for opening this issue, @0z5a! | Field | Value | |-------|-------| | **Issue** | #4979 | | **GitHub user ID** | `192209249` | | **Reporter** | @0z5a | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:24:36 +00:00

Author

Owner

@arun-elr commented on GitHub (Oct 28, 2026):

looked at A. agree it can't work as is: inside the cudaLaunchHostFunc callback, nvlink_intra's submit creates streams/events, calls cudaEventRecord/cudaEventSynchronize and cudaMemcpyBatchAsync, and the status poll calls cudaStreamQuery. a host function isn't allowed to make any of those calls, so pre-creating the thread-local pools wouldn't fix it.

two options I see:

fail fast: transfer_*_on_cuda raises in Python up front when the target goes through nvlink_intra, instead of _exit(1) from the callback. small change.
stream-ordered path: for nvlink_intra, skip the host callback and enqueue the copies on the caller's stream (resolve addresses on the calling thread, then cudaMemcpyBatchAsync on that stream). that's what on_cuda should mean for a GPU-to-GPU transport, and it drops the host round trip. needs a transport hook, e.g. submitTransferOnStream(batch, entries, stream) that returns NotSupported by default.

which do you prefer? sending B (one bad entry drops the rest) as its own PR first.

@arun-elr commented on GitHub (Oct 28, 2026): looked at A. agree it can't work as is: inside the `cudaLaunchHostFunc` callback, nvlink_intra's submit creates streams/events, calls `cudaEventRecord`/`cudaEventSynchronize` and `cudaMemcpyBatchAsync`, and the status poll calls `cudaStreamQuery`. a host function isn't allowed to make any of those calls, so pre-creating the thread-local pools wouldn't fix it. two options I see: 1. fail fast: `transfer_*_on_cuda` raises in Python up front when the target goes through nvlink_intra, instead of `_exit(1)` from the callback. small change. 2. stream-ordered path: for nvlink_intra, skip the host callback and enqueue the copies on the caller's stream (resolve addresses on the calling thread, then `cudaMemcpyBatchAsync` on that stream). that's what on_cuda should mean for a GPU-to-GPU transport, and it drops the host round trip. needs a transport hook, e.g. `submitTransferOnStream(batch, entries, stream)` that returns NotSupported by default. which do you prefer? sending B (one bad entry drops the rest) as its own PR first.

bsakaguchi commented

2026-10-29 21:24:36 +00:00

Author

Owner

@he-yufeng commented on GitHub (Oct 29, 2026):

Both parts of this are covered by two PRs that are already open, so nobody needs to start a third.

Part A, CUDA API use from inside the cudaLaunchHostFunc callback, is #5038. The on-CUDA entry points reject an nvlink_intra target before allocating a batch or registering the host function, so the caller gets an explicit error saying why a CUDA host function cannot call CUDA APIs instead of the driver callback killing the process with _exit(1). Your point about the thread-local stream and event pools being empty on the driver's callback thread is exactly the diagnosis I worked from.

Part B, the no-P2P behavior, is #5022. Both NVLink submission entry points resolve every destination before publishing any slice, so a later IPC mapping failure returns an error without leaving earlier tasks holding pending slices that keep batch cleanup deferred. The error now names CUDA IPC peer access and buffer registration instead of the generic "device memory not registered".

Between them that covers the two failure rows in your table: the transfer_write_on_cuda process exit and the mixed batch where a valid same-GPU entry was dropped and the batch leaked.

One boundary both PRs state and I want to keep visible rather than let it read as done: neither enables P2P nor adds a fallback transport. Your expected behavior offered two options, fall back or reject the whole batch before creating any slice, and what landed is the second. Checking cudaDeviceCanAccessPeer up front and rejecting cleanly is the part that stops the data loss and the leak; a staged-copy fallback or a transport switchover is a larger change and is not in either PR.

#4104 and #4919 are unrelated to these two paths, and my search did not find a third PR touching the thread-local pools or submitTransferTask's early return.

@he-yufeng commented on GitHub (Oct 29, 2026): Both parts of this are covered by two PRs that are already open, so nobody needs to start a third. **Part A**, CUDA API use from inside the `cudaLaunchHostFunc` callback, is #5038. The on-CUDA entry points reject an `nvlink_intra` target before allocating a batch or registering the host function, so the caller gets an explicit error saying why a CUDA host function cannot call CUDA APIs instead of the driver callback killing the process with `_exit(1)`. Your point about the thread-local stream and event pools being empty on the driver's callback thread is exactly the diagnosis I worked from. **Part B**, the no-P2P behavior, is #5022. Both NVLink submission entry points resolve every destination before publishing any slice, so a later IPC mapping failure returns an error without leaving earlier tasks holding pending slices that keep batch cleanup deferred. The error now names CUDA IPC peer access and buffer registration instead of the generic "device memory not registered". Between them that covers the two failure rows in your table: the `transfer_write_on_cuda` process exit and the mixed batch where a valid same-GPU entry was dropped and the batch leaked. One boundary both PRs state and I want to keep visible rather than let it read as done: neither enables P2P nor adds a fallback transport. Your expected behavior offered two options, fall back or reject the whole batch before creating any slice, and what landed is the second. Checking `cudaDeviceCanAccessPeer` up front and rejecting cleanly is the part that stops the data loss and the leak; a staged-copy fallback or a transport switchover is a larger change and is not in either PR. #4104 and #4919 are unrelated to these two paths, and my search did not find a third PR touching the thread-local pools or `submitTransferTask`'s early return.

bsakaguchi commented

2026-10-29 21:24:36 +00:00

Author

Owner

@arun-elr commented on GitHub (Oct 29, 2026):

thanks, I missed #5022 when I posted. fine with both, so I'll drop my branch:

A (transfer_*_on_cuda killing the process from the host callback): #5038, failing fast on an nvlink_intra target.
B (one unmappable entry in a mixed batch): #5022, rejecting the whole batch before any slice is published.

I have a two-process test that runs mixed batches (good+bad, bad+good, good+good) through the Python API on a real GPU. happy to run it against #5022 and #5038 on an H200 and post the results if that's useful.

@arun-elr commented on GitHub (Oct 29, 2026): thanks, I missed #5022 when I posted. fine with both, so I'll drop my branch: - A (`transfer_*_on_cuda` killing the process from the host callback): #5038, failing fast on an `nvlink_intra` target. - B (one unmappable entry in a mixed batch): #5022, rejecting the whole batch before any slice is published. I have a two-process test that runs mixed batches (good+bad, bad+good, good+good) through the Python API on a real GPU. happy to run it against #5022 and #5038 on an H200 and post the results if that's useful.

bsakaguchi referenced this issue

2026-10-29 21:32:53 +00:00

[PR #1261] [CLOSED] [bugfix] Metrics modifications: Deleting and adding some metrics #2114