[GH-ISSUE #2432] [Py] batch_transfer_async / transfer_submit_write return -1 through uint64 batch_id_t — documented failure sentinel 0 never fires, callers crash #648

New issue

Closed

opened 2026-10-29 21:17:36 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-29 21:17:36 +00:00

Owner

Originally created by @bp-cheng on GitHub (Jun 11, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2432

Originally assigned to: @bp-cheng on GitHub.

TransferEnginePy::batchTransferAsync returns batch_id_t, which is uint64_t (mooncake-transfer-engine/include/transfer_engine_c.h:27). When openSegment fails it does return -1; (mooncake-integration/transfer_engine/transfer_engine_py.cpp:609), so Python receives 18446744073709551615 instead of an error:

the same function's other failure paths return 0 (lines 618, 651),
the Python API reference documents "Batch ID for tracking the operation, or 0 on failure" for batch_transfer_async / batch_transfer_async_write / batch_transfer_async_read (docs/source/python-api-reference/transfer-engine.md),
the repo's own test follows that contract: assertNotEqual(batch_id, 0, ...) (mooncake-wheel/tests/transfer_engine_initiator_test.py:224).

A caller that follows the documented batch_id != 0 check treats the bogus id as valid and passes it to get_batch_transfer_status, which reinterpret_casts it to BatchDesc* and dereferences it (transfer_engine_py.cpp:667-668) → SIGSEGV.

transferSubmitWrite has the same defect twice (lines 733, 748). Its documented contract is "negative value on failure", which a uint64_t return can never satisfy as seen from Python, so failures are undetectable there as well. The line-748 path additionally leaks the freshly allocated BatchDesc (its siblings batchTransferAsync / batchTransferSync call freeBatchID on the equivalent path).

Trigger condition is mundane — any openSegment failure (typo'd, unregistered, or down target hostname); no special hardware. Reproduced on current main (474f6785) with tcp + the HTTP metadata server, Ubuntu 22.04:

batch_transfer_async_write -> 18446744073709551615
  == 2**64 - 1 ? True
  == 0 (documented failure sentinel) ? False
transfer_submit_write -> 18446744073709551615
  == 2**64 - 1 ? True
  < 0 (documented failure sentinel) ? False
passing the id to get_batch_transfer_status (documented check batch_id != 0 lets it through) ...
exit code 139 (SIGSEGV)

Fix incoming: normalize the failure return of the three sites to 0 (batch ids are heap BatchDesc* values, never 0), free the allocated batch in transferSubmitWrite like its siblings do, and align the transfer_submit_write docs ("negative value" → "0").

Originally created by @bp-cheng on GitHub (Jun 11, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2432 Originally assigned to: @bp-cheng on GitHub. `TransferEnginePy::batchTransferAsync` returns `batch_id_t`, which is `uint64_t` (`mooncake-transfer-engine/include/transfer_engine_c.h:27`). When `openSegment` fails it does `return -1;` (`mooncake-integration/transfer_engine/transfer_engine_py.cpp:609`), so Python receives `18446744073709551615` instead of an error: - the same function's other failure paths return `0` (lines 618, 651), - the Python API reference documents "Batch ID for tracking the operation, or 0 on failure" for `batch_transfer_async` / `batch_transfer_async_write` / `batch_transfer_async_read` (`docs/source/python-api-reference/transfer-engine.md`), - the repo's own test follows that contract: `assertNotEqual(batch_id, 0, ...)` (`mooncake-wheel/tests/transfer_engine_initiator_test.py:224`). A caller that follows the documented `batch_id != 0` check treats the bogus id as valid and passes it to `get_batch_transfer_status`, which `reinterpret_cast`s it to `BatchDesc*` and dereferences it (`transfer_engine_py.cpp:667-668`) → SIGSEGV. `transferSubmitWrite` has the same defect twice (lines 733, 748). Its documented contract is "negative value on failure", which a `uint64_t` return can never satisfy as seen from Python, so failures are undetectable there as well. The line-748 path additionally leaks the freshly allocated `BatchDesc` (its siblings `batchTransferAsync` / `batchTransferSync` call `freeBatchID` on the equivalent path). Trigger condition is mundane — any `openSegment` failure (typo'd, unregistered, or down target hostname); no special hardware. Reproduced on current main (474f6785) with `tcp` + the HTTP metadata server, Ubuntu 22.04: ``` batch_transfer_async_write -> 18446744073709551615 == 2**64 - 1 ? True == 0 (documented failure sentinel) ? False transfer_submit_write -> 18446744073709551615 == 2**64 - 1 ? True < 0 (documented failure sentinel) ? False passing the id to get_batch_transfer_status (documented check batch_id != 0 lets it through) ... exit code 139 (SIGSEGV) ``` Fix incoming: normalize the failure return of the three sites to `0` (batch ids are heap `BatchDesc*` values, never 0), free the allocated batch in `transferSubmitWrite` like its siblings do, and align the `transfer_submit_write` docs ("negative value" → "0").

bsakaguchi closed this issue

2026-10-29 21:17:36 +00:00

bsakaguchi commented

2026-10-29 21:17:37 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 11, 2026):

Thanks for opening this issue, @bp-cheng!

Field	Value
Issue	#2432
GitHub user ID	`194346132`
Reporter	@bp-cheng

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jun 11, 2026): Thanks for opening this issue, @bp-cheng! | Field | Value | |-------|-------| | **Issue** | #2432 | | **GitHub user ID** | `194346132` | | **Reporter** | @bp-cheng | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.