[PR #710] [CLOSED] [WIP] Use VRAM as buffer #1761

New issue

Closed

opened 2026-10-29 21:30:07 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:30:07 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/Mooncake/pull/710
Author: @XucSh
Created: 8/4/2025
Status: ❌ Closed

Base: main ← Head: hbm-devel

📝 Commits (6)

bfe6167 Use VRAM as buffer
c68c5f7 Fix UAM
9b61319 Format code and update dockerfile
c3c3535 Add cuda dependency in test wheel
0f95238 Fix some error
2273f40 Fix lint of other commit

📊 Changes

22 files changed (+407 additions, -66 deletions)

View changed files

📝 .github/workflows/ci.yml (+8 -0)
📝 Dockerfile (+2 -0)
📝 mooncake-integration/store/store_py.cpp (+137 -1)
📝 mooncake-integration/store/store_py.h (+34 -0)
📝 mooncake-store/CMakeLists.txt (+1 -1)
📝 mooncake-store/include/allocation_strategy.h (+5 -0)
📝 mooncake-store/include/client.h (+10 -1)
📝 mooncake-store/include/client_buffer.hpp (+2 -1)
📝 mooncake-store/include/segment.h (+19 -2)
📝 mooncake-store/include/types.h (+6 -4)
📝 mooncake-store/include/utils.h (+2 -1)
📝 mooncake-store/src/CMakeLists.txt (+4 -1)
📝 mooncake-store/src/client.cpp (+52 -2)
📝 mooncake-store/src/master_client.cpp (+1 -1)
📝 mooncake-store/src/master_service.cpp (+8 -3)
📝 mooncake-store/src/segment.cpp (+8 -1)
📝 mooncake-store/src/utils.cpp (+56 -1)
📝 mooncake-store/tests/client_integration_test.cpp (+2 -2)
📝 mooncake-store/tests/e2e/client_wrapper.cpp (+2 -2)
📝 mooncake-store/tests/master_service_test.cpp (+31 -30)

...and 2 more files

📄 Description

This PR introduce VRAM as L1 cache.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/Mooncake/pull/710 **Author:** [@XucSh](https://github.com/XucSh) **Created:** 8/4/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `hbm-devel` --- ### 📝 Commits (6) - [`bfe6167`](https://github.com/kvcache-ai/Mooncake/commit/bfe6167201d79ac7ce27a394724a359612d43712) Use VRAM as buffer - [`c68c5f7`](https://github.com/kvcache-ai/Mooncake/commit/c68c5f7d47a7b4de7b7712085db2a384e6733a43) Fix UAM - [`9b61319`](https://github.com/kvcache-ai/Mooncake/commit/9b6131944d648451b3ca6d1e0e1ca39387f8417f) Format code and update dockerfile - [`c3c3535`](https://github.com/kvcache-ai/Mooncake/commit/c3c3535487a09368b53d62cce919661249a93937) Add cuda dependency in test wheel - [`0f95238`](https://github.com/kvcache-ai/Mooncake/commit/0f9523827e99e547134a41bc40c38f01caefe69f) Fix some error - [`2273f40`](https://github.com/kvcache-ai/Mooncake/commit/2273f40a682d923c599faed8ddb727a2297f4187) Fix lint of other commit ### 📊 Changes **22 files changed** (+407 additions, -66 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/ci.yml` (+8 -0) 📝 `Dockerfile` (+2 -0) 📝 `mooncake-integration/store/store_py.cpp` (+137 -1) 📝 `mooncake-integration/store/store_py.h` (+34 -0) 📝 `mooncake-store/CMakeLists.txt` (+1 -1) 📝 `mooncake-store/include/allocation_strategy.h` (+5 -0) 📝 `mooncake-store/include/client.h` (+10 -1) 📝 `mooncake-store/include/client_buffer.hpp` (+2 -1) 📝 `mooncake-store/include/segment.h` (+19 -2) 📝 `mooncake-store/include/types.h` (+6 -4) 📝 `mooncake-store/include/utils.h` (+2 -1) 📝 `mooncake-store/src/CMakeLists.txt` (+4 -1) 📝 `mooncake-store/src/client.cpp` (+52 -2) 📝 `mooncake-store/src/master_client.cpp` (+1 -1) 📝 `mooncake-store/src/master_service.cpp` (+8 -3) 📝 `mooncake-store/src/segment.cpp` (+8 -1) 📝 `mooncake-store/src/utils.cpp` (+56 -1) 📝 `mooncake-store/tests/client_integration_test.cpp` (+2 -2) 📝 `mooncake-store/tests/e2e/client_wrapper.cpp` (+2 -2) 📝 `mooncake-store/tests/master_service_test.cpp` (+31 -30) _...and 2 more files_ </details> ### 📄 Description This PR introduce VRAM as L1 cache. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:30:07 +00:00