mirror of
https://github.com/kvcache-ai/Mooncake.git
synced 2026-11-05 01:17:55 +00:00
[GH-ISSUE #3108] [Installation] Publish prebuilt ROCm/HIP wheels to PyPI #796
Labels
No labels
ascend
auto-closed
bug
bug
CI/Build
Common
documentation
enhancement
feature request
good first issue
help wanted
high priority
MetaX
nightly-failure
pull-request
PyTorch Backend
question
RFC
Roadmap
stale
Store
Tests
Transfer Engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/Mooncake#796
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @charxwu on GitHub (Jul 24, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3108
Originally assigned to: @Aionw, @amd-arozanov on GitHub.
Summary
We are bringing up MiniMax-M3 MXFP4 vLLM disaggregated benchmarks on AMD MI355X with Mooncake DRAM KV offload (
MooncakeStoreConnector+ MultiConnector with MoRIIO P/D). HIP support landed in #1742, but today the only PyPI packages are CUDA (mooncake-transfer-engine,mooncake-transfer-engine-cuda13), CPU-only (mooncake-transfer-engine-non-cuda), and NPU (mooncake-transfer-engine-npu). There is no ROCm/HIP wheel.Our current workaround is to build Mooncake from source at job runtime with
-DUSE_HIP=ON -DUSE_CUDA=OFFand cache the artifact — workable but slow on first run and hard to satisfy downstream CI policies that require pinned images to run as shipped (no runtime builds).Request
Please publish a prebuilt ROCm/HIP Python wheel to PyPI, e.g.:
mooncake-transfer-engine-rocm(or-hip), versioned alongside existing releases (we pinv0.3.11.post1today)Minimum requirements for our MI355X / ROCm 7.x stack:
-DUSE_HIP=ON,-DWITH_STORE=ONmooncake.engine,mooncake.store,mooncake_masterCLI)libamdhip64.sobut does not vendor incompatible ROCm runtime libs (same approach validated in #1742dist-rocm-hipwheels)manylinux_2_28_x86_64for cp310–cp313 (matching existing CUDA wheel matrix)Why not
mooncake-transfer-engine-non-cuda?The non-CUDA wheel is CPU-only. Our vLLM disagg path validates HIP linkage (
libamdhip64.so) for the Store/TE stack on MI355X. CPU-only wheels do not satisfy this.Current workaround
Runtime build in
server_vllm.sh:@github-actions[bot] commented on GitHub (Jul 24, 2026):
Thanks for opening this issue, @charxwu!
296364977A maintainer will triage this when possible. To help us respond faster, please include:
Useful links: Documentation · Contributing guide
@ykwd commented on GitHub (Jul 25, 2026):
Sounds good! We just released v0.3.12.post1, so this is a great version for us to start publishing an official ROCm/HIP wheel.
@amd-arozanov Would you be interested in collaborating on this when you have time? We'd really appreciate your expertise to help make official AMD wheels happen.
@functionstackx commented on GitHub (Jul 27, 2026):
@andyluo7 @chunfangamd
@Aionw commented on GitHub (Jul 27, 2026):
I'd be happy to collaborate with the ROCm and AMD teams to make Mooncake ROCm/HIP packages available on PyPI.
@amd-arozanov commented on GitHub (Jul 28, 2026):
@Aionw, thanks! Feel free to reach me if you need any help or you have any questions
@tjtanaa commented on GitHub (Aug 23, 2026):
@amd-arozanov how is this effort going? It would be nice to have mooncake wheel as it is used in a very broad ecosystem, from inferencing to RL training as well.
@functionstackx commented on GitHub (Aug 23, 2026):
@tjtanaa andy luo has it here https://github.com/kvcache-ai/Mooncake/pulls?q=is%3Apr+is%3Aclosed+author%3Aandyluo7
@tjtanaa commented on GitHub (Aug 23, 2026):
Thank you @functionstackx , so we will get the wheel only in the next release.