verbs.h` include (and wheel setup.py hard-blocks darwin) #996

New issue

Closed

opened 2026-10-29 21:21:46 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:21:46 +00:00

Owner

Originally created by @now-ing on GitHub (Sep 6, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3896

Summary

Mooncake's TCP transport cannot even be compiled on macOS today, because tcp_transport.h unconditionally includes the RDMA verbs header, and the wheel setup.py explicitly exits on darwin. With PD-disaggregated serving now reaching Apple-Silicon clusters (e.g. vllm-metal's new experimental PD path: vllm-project/vllm-metal#709), a TCP-only Mooncake client on macOS would be a natural fit for Mac prefill/decode pools — but the build blocks it before any transport logic is reached.

Evidence (main @ 2026-09-06)

mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h:17-18:

#ifndef TCP_TRANSPORT_H_
#define TCP_TRANSPORT_H_

#include <infiniband/verbs.h>

The TCP transport header includes infiniband/verbs.h unconditionally — on macOS there is no libibverbs/rdma-core, so any TU that pulls this header fails immediately. Per the docs, protocol=tcp is a first-class DRAM↔DRAM path that shouldn't need verbs at all (https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols/).

mooncake-wheel/setup.py:10-12:

unsupported_platforms = ["win32", "darwin"]  # Still blocking non-Linux builds
if sys.platform in unsupported_platforms:
    sys.exit(
        f"Error: mooncake does not support {platform.system()} at this time. "
        "Please use a supported Linux distribution."
    )

(Notably the same file already contains pre-written darwin arm64/x86_64 + macosx tag recognition helpers — the guard is the only thing standing in the way of at least experimental wheels.)

Additionally mooncake-transfer-engine/include/transport/common.h includes <numa.h> (Linux-only libnuma) unconditionally, and dependencies.sh covers only rpm/deb distros, so a TCP-only macOS build would need that include gated too.

Environment

macOS 15.6 / 26.0, Apple Silicon (M3 Max / M4 Max)
Attempted: transfer-engine CMake configure + pip install mooncake-transfer-engine (no darwin wheel exists on PyPI — only manylinux x86_64/aarch64)

Suggestion (happy to PR)

Gate the verbs include in tcp_transport.h behind #ifdef (or move verbs-dependent bits to an rdma-specific header) so the TCP transport compiles where only a plain socket stack exists.
Same for <numa.h> in common.h (fallback stubs when __APPLE__).
Relax the setup.py darwin hard-exit into an explicit --allow-unsupported / experimental wheel path, since the tag helpers are already written.

If maintainers are open to it, I can start with (1) + (2) as a small PR targeting a TCP-only USE_TCP=ON-style CMake option for non-Linux platforms. Context: we're wiring PD disaggregation across a pure Mac cluster (vllm-metal#709) and Mooncake's TCP backend is the obvious production-grade transfer layer to grow into.

Originally created by @now-ing on GitHub (Sep 6, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3896 ## Summary Mooncake's TCP transport cannot even be *compiled* on macOS today, because `tcp_transport.h` unconditionally includes the RDMA verbs header, and the wheel `setup.py` explicitly exits on darwin. With PD-disaggregated serving now reaching Apple-Silicon clusters (e.g. vllm-metal's new experimental PD path: vllm-project/vllm-metal#709), a TCP-only Mooncake client on macOS would be a natural fit for Mac prefill/decode pools — but the build blocks it before any transport logic is reached. ## Evidence (main @ 2026-09-06) `mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h:17-18`: ```cpp #ifndef TCP_TRANSPORT_H_ #define TCP_TRANSPORT_H_ #include <infiniband/verbs.h> ``` The **TCP** transport header includes `infiniband/verbs.h` unconditionally — on macOS there is no libibverbs/rdma-core, so any TU that pulls this header fails immediately. Per the docs, `protocol=tcp` is a first-class DRAM↔DRAM path that shouldn't need verbs at all (https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols/). `mooncake-wheel/setup.py:10-12`: ```python unsupported_platforms = ["win32", "darwin"] # Still blocking non-Linux builds if sys.platform in unsupported_platforms: sys.exit( f"Error: mooncake does not support {platform.system()} at this time. " "Please use a supported Linux distribution." ) ``` (Notably the same file already contains pre-written darwin arm64/x86_64 + macosx tag recognition helpers — the guard is the only thing standing in the way of at least experimental wheels.) Additionally `mooncake-transfer-engine/include/transport/common.h` includes `<numa.h>` (Linux-only libnuma) unconditionally, and `dependencies.sh` covers only rpm/deb distros, so a TCP-only macOS build would need that include gated too. ## Environment - macOS 15.6 / 26.0, Apple Silicon (M3 Max / M4 Max) - Attempted: transfer-engine CMake configure + `pip install mooncake-transfer-engine` (no darwin wheel exists on PyPI — only manylinux x86_64/aarch64) ## Suggestion (happy to PR) 1. Gate the verbs include in `tcp_transport.h` behind `#ifdef` (or move verbs-dependent bits to an rdma-specific header) so the TCP transport compiles where only a plain socket stack exists. 2. Same for `<numa.h>` in `common.h` (fallback stubs when `__APPLE__`). 3. Relax the `setup.py` darwin hard-exit into an explicit `--allow-unsupported` / experimental wheel path, since the tag helpers are already written. If maintainers are open to it, I can start with (1) + (2) as a small PR targeting a TCP-only `USE_TCP=ON`-style CMake option for non-Linux platforms. Context: we're wiring PD disaggregation across a pure Mac cluster (vllm-metal#709) and Mooncake's TCP backend is the obvious production-grade transfer layer to grow into.

bsakaguchi closed this issue

2026-10-29 21:21:46 +00:00

bsakaguchi commented

2026-10-29 21:21:46 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Sep 6, 2026):

Thanks for opening this issue, @now-ing!

Field	Value
Issue	#3896
GitHub user ID	`24534365`
Reporter	@now-ing

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Sep 6, 2026): Thanks for opening this issue, @now-ing! | Field | Value | |-------|-------| | **Issue** | #3896 | | **GitHub user ID** | `24534365` | | **Reporter** | @now-ing | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:21:47 +00:00

Author

Owner

@he-yufeng commented on GitHub (Sep 6, 2026):

Duplicate of #3894 (identical body, same submitter, same minute). Closing to keep one thread; please continue in #3894.

@he-yufeng commented on GitHub (Sep 6, 2026): Duplicate of #3894 (identical body, same submitter, same minute). Closing to keep one thread; please continue in #3894.

bsakaguchi referenced this issue

2026-10-29 21:31:33 +00:00

[PR #996] [MERGED] add log #1940