kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #2581] [RFC]: Mooncake Store Hardware Abstraction #678

New issue

Open

opened 2026-10-29 21:18:01 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:18:01 +00:00

Owner

Originally created by @Aionw on GitHub (Jun 23, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2581

Changes proposed

RFC: Mooncake Store Hardware Abstraction

Summary

This RFC proposes introducing a focused hardware abstraction layer inside Mooncake Store.

The current maintenance pain is mainly concentrated in two areas:

Accelerator-specific logic, where Store needs to reason about device memory, device type, buffer constraints, and hardware-dependent access behavior.
Ascend shared memory logic, where Store contains platform-specific shared memory handling that is guarded by compile-time branches and mixed into common client paths.

The proposed direction is:

Mooncake Store should expose small, Store-level abstractions for accelerator resources and shared-memory resources, while keeping platform-specific implementation details behind isolated adapters.

This RFC is intended as a parent / roadmap RFC. Follow-up RFCs and PRs should define the exact interfaces and migrate existing code incrementally.

Motivation

Mooncake Store already has relatively clear abstractions for distributed storage and filesystem-backed storage. Those areas should continue to evolve within their existing interfaces.

The less clean part today is hardware-specific Store logic that leaks into common client and buffer-management paths. In particular:

accelerator-related behavior is represented through scattered device checks and conditional code
Ascend shared memory behavior is implemented as a special path guarded by compile-time flags
common Store code needs to know too much about platform-specific resource handling

This creates several problems:

Problem	Impact
Hardware-specific branches are scattered	Reviewers need to inspect common Store paths to understand one device-specific change.
Accelerator behavior is not centralized	Memory type, device ownership, and buffer capability checks can drift across call sites.
Ascend shared memory is special-cased	Shared memory lifecycle and lookup logic becomes harder to review and test.
Ownership boundary is unclear	Adding or changing one hardware path can accidentally affect unrelated Store behavior.
Follow-up hardware support becomes harder	New accelerator support needs a clear extension point instead of more common-code branching.

The goal is not to create a broad abstraction for every Store resource. The goal is to address the current hardware hotspots directly.

Goals

Reduce accelerator-related #ifdefs and device-specific branches in Mooncake Store common logic.
Isolate Ascend shared memory handling behind a shared-memory adapter boundary.
Centralize Store-level hardware capability checks.
Make future accelerator support easier to review and maintain.
Preserve existing distributed storage and filesystem abstractions.
Provide a parent RFC for follow-up RFCs and PRs.

Non-goals

This RFC does not propose:

redesigning distributed storage or filesystem abstractions
replacing existing storage backend interfaces
removing every compile-time guard immediately
changing public Store APIs in one large step
forcing storage, filesystem, accelerator, and shared memory into one universal abstraction
changing existing behavior without a dedicated follow-up RFC or PR

Design Principles

1. Keep the abstraction close to current pain points

The follow-up RFCs should not try to model every possible hardware resource. They should focus on accelerator-related behavior and shared memory behavior because these are the places where current conditional logic hurts review and maintenance.

2. Keep platform details behind adapters

Platform adapters may still use compile-time guards and SDK-specific code. Those details should stay local to the adapter.

Common Store code should depend on stable Store-level interfaces.

3. Make fallback explicit

If a resource does not support a required capability, Store should explicitly choose a fallback path or return a clear error.

Silent fallback makes performance and correctness issues hard to diagnose.

Follow-up RFCs / PRs

Area	Status	Expected output
Accelerator abstraction	#2582	RFC and follow-up PRs for accelerator-related Store logic
Shared memory abstraction	WIP	RFC and follow-up PRs for Ascend shared memory related Store logic

Review Guidance

Future PRs in this area should answer:

Does this change address one of the two current hotspots: accelerator logic or Ascend shared memory logic?
Does it reduce hardware-specific branching in Mooncake Store common paths?
Are platform-specific details isolated from common client and buffer-management code?
Is unsupported-build behavior explicit?
Does it preserve existing Store behavior unless explicitly changed?

Before submitting a new issue...

Make sure you already searched for relevant issues and read the documentation

Originally created by @Aionw on GitHub (Jun 23, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/2581 ### Changes proposed # RFC: Mooncake Store Hardware Abstraction ## Summary This RFC proposes introducing a focused hardware abstraction layer inside Mooncake Store. The current maintenance pain is mainly concentrated in two areas: 1. **Accelerator-specific logic**, where Store needs to reason about device memory, device type, buffer constraints, and hardware-dependent access behavior. 2. **Ascend shared memory logic**, where Store contains platform-specific shared memory handling that is guarded by compile-time branches and mixed into common client paths. The proposed direction is: > **Mooncake Store should expose small, Store-level abstractions for accelerator resources and shared-memory resources, while keeping platform-specific implementation details behind isolated adapters.** This RFC is intended as a parent / roadmap RFC. Follow-up RFCs and PRs should define the exact interfaces and migrate existing code incrementally. ## Motivation Mooncake Store already has relatively clear abstractions for distributed storage and filesystem-backed storage. Those areas should continue to evolve within their existing interfaces. The less clean part today is hardware-specific Store logic that leaks into common client and buffer-management paths. In particular: - accelerator-related behavior is represented through scattered device checks and conditional code - Ascend shared memory behavior is implemented as a special path guarded by compile-time flags - common Store code needs to know too much about platform-specific resource handling This creates several problems: | Problem | Impact | | --- | --- | | Hardware-specific branches are scattered | Reviewers need to inspect common Store paths to understand one device-specific change. | | Accelerator behavior is not centralized | Memory type, device ownership, and buffer capability checks can drift across call sites. | | Ascend shared memory is special-cased | Shared memory lifecycle and lookup logic becomes harder to review and test. | | Ownership boundary is unclear | Adding or changing one hardware path can accidentally affect unrelated Store behavior. | | Follow-up hardware support becomes harder | New accelerator support needs a clear extension point instead of more common-code branching. | The goal is not to create a broad abstraction for every Store resource. The goal is to address the current hardware hotspots directly. ## Goals - Reduce accelerator-related `#ifdef`s and device-specific branches in Mooncake Store common logic. - Isolate Ascend shared memory handling behind a shared-memory adapter boundary. - Centralize Store-level hardware capability checks. - Make future accelerator support easier to review and maintain. - Preserve existing distributed storage and filesystem abstractions. - Provide a parent RFC for follow-up RFCs and PRs. ## Non-goals This RFC does not propose: - redesigning distributed storage or filesystem abstractions - replacing existing storage backend interfaces - removing every compile-time guard immediately - changing public Store APIs in one large step - forcing storage, filesystem, accelerator, and shared memory into one universal abstraction - changing existing behavior without a dedicated follow-up RFC or PR ## Design Principles ### 1. Keep the abstraction close to current pain points The follow-up RFCs should not try to model every possible hardware resource. They should focus on accelerator-related behavior and shared memory behavior because these are the places where current conditional logic hurts review and maintenance. ### 2. Keep platform details behind adapters Platform adapters may still use compile-time guards and SDK-specific code. Those details should stay local to the adapter. Common Store code should depend on stable Store-level interfaces. ### 3. Make fallback explicit If a resource does not support a required capability, Store should explicitly choose a fallback path or return a clear error. Silent fallback makes performance and correctness issues hard to diagnose. ## Follow-up RFCs / PRs | Area | Status | Expected output | | --- | --- | --- | | Accelerator abstraction | #2582 | RFC and follow-up PRs for accelerator-related Store logic | | Shared memory abstraction | WIP | RFC and follow-up PRs for Ascend shared memory related Store logic | ## Review Guidance Future PRs in this area should answer: - Does this change address one of the two current hotspots: accelerator logic or Ascend shared memory logic? - Does it reduce hardware-specific branching in Mooncake Store common paths? - Are platform-specific details isolated from common client and buffer-management code? - Is unsupported-build behavior explicit? - Does it preserve existing Store behavior unless explicitly changed? ### Before submitting a new issue... - [ ] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

bsakaguchi added the

RFC

label

2026-10-29 21:18:01 +00:00

bsakaguchi commented

2026-10-29 21:18:01 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jun 23, 2026):

Thanks for opening this issue, @Aionw!

Field	Value
Issue	#2581
GitHub user ID	`41376987`
Reporter	@Aionw

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jun 23, 2026): Thanks for opening this issue, @Aionw! | Field | Value | |-------|-------| | **Issue** | #2581 | | **GitHub user ID** | `41376987` | | **Reporter** | @Aionw | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:18:01 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Sep 29, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Sep 29, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.