kvcache-ai/Mooncake

Fork 0

mirror of https://github.com/kvcache-ai/Mooncake.git synced 2026-11-05 01:17:55 +00:00

[GH-ISSUE #3068] [RFC]: Add Go DummyClient support and a multi-process benchmark #787

New issue

Open

opened 2026-10-29 21:19:32 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 21:19:32 +00:00

Owner

Originally created by @BGQ99 on GitHub (Jul 23, 2026).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3068

Motivation

Mooncake provides a C++ DummyClient that communicates with a standalone
mooncake_client, but the Go bindings currently lack the APIs needed to create
and configure a DummyClient, access the shared-memory buffers prepared during
setup, and exercise the single-object PutFrom path.

This RFC proposes adding the required bindings and a multi-process Go benchmark
for validating multiple independent application processes sharing one
standalone mooncake_client.

An implementation is available in
kvcache-ai/Mooncake#3015.
It ports the relevant work from
zchuango/Mooncake#29
onto the current Mooncake main branch.

Runtime Architecture

The request path is:

Go worker -> DummyClient -> RPC -> standalone mooncake_client
-> RealClient -> Mooncake Store

The standalone mooncake_client owns the RealClient and exposes the required
RPC handlers. Each benchmark worker is an independent operating-system process
with its own DummyClient, address space, and registered shared-memory context.

Separate processes are required because goroutines would still share one address
space and would not reproduce the intended multi-client shared-memory setup.

Proposed Changes

1. Expose DummyClient through the Go and C bindings

Allow the Store bindings to explicitly create either a RealClient or a
DummyClient.

The proposed Go API uses:

New() to create the default RealClient;
NewWithType(MOONCAKE_CLIENT_DUMMY) to create a DummyClient.

The public Go API uses the Go-defined ClientType type and does not expose CGo
types to callers.

2. Preserve the existing Go Setup API

Keep the existing seven-argument Go Setup method for RealClient and add:

DummySetup(memPoolSize, localBufferSize, serverAddress, ipcSocketPath)

Go does not support optional function parameters. Extending the existing
Setup signature would therefore break existing Go call sites.

3. Preserve C API compatibility

Keep the existing C creation and setup APIs unchanged for current direct C
callers, and add explicit entry points for dummy clients:

keep mooncake_store_create() as the default RealClient constructor;
add mooncake_store_create_with_type(client_type);
keep the existing mooncake_store_setup(...);
add mooncake_store_setup_dummy(...).

The Go wrapper will use the new entry points for DummyClient. Existing direct
C callers remain source-compatible, while the additional symbols provide the
configuration required by the Go binding.

Feedback on the exact function names and parameter layout is welcome, but this
RFC does not propose changing the signatures of the existing C APIs.

4. Provide access to DummyClient registered buffers

The benchmark must access the shared-memory buffers prepared during
DummyClient setup before calling PutFrom and GetInto.

The current implementation exposes operations to:

obtain the number of registered buffers;
query a buffer pointer and size by index;
determine whether a buffer belongs to the hot cache;
unregister the buffers.

The current implementation exposes these operations through the public Store
binding.

5. Complete the single-object PutFrom path

DummyClient already supports batch put_from, while the Go benchmark uses the
single-object PutFrom API.

The current implementation adds a dedicated RealClient::put_from_dummy_helper RPC. It:

finds the shared-memory context using the client ID;
translates the DummyClient address into a pointer valid in the standalone
mooncake_client process;
invokes the existing RealClient single-object put_from implementation.

The handler is registered in both the production mooncake_client and the test
RPC server.

The current proposal retains a dedicated single-object RPC. This keeps the
single-object API explicit, preserves its direct forwarding path, and reuses
the existing RealClient single-object implementation. Reusing the batch RPC
with a one-object request was considered, but not selected because it would
make the single-object API depend on batch semantics internally.

6. Add a multi-process Go benchmark

The benchmark:

starts independent worker processes with os/exec;
creates one DummyClient per worker;
distributes keys among workers;
writes deterministic data with PutFrom;
reads data with GetInto;
verifies the returned contents;
reports failures, verification errors, and throughput.

Using separate processes, rather than goroutines, provides independent address
spaces, DummyClient instances, shared-memory contexts, and CPU affinity.

Shared-Memory Safety

Pointers are process-local virtual addresses. RealClient must not directly
dereference an address received from DummyClient.

The RPC request includes the DummyClient address, size, device ID, and client
ID. RealClient uses the corresponding shared-memory context to translate the
dummy address into a valid local pointer.

Before accessing registered memory, the Go benchmark validates:

conversion to the platform int range;
the required size against the registered buffer capacity.

Go slices are constructed using the actual accessible buffer length.

Scope

Included:

C and Go DummyClient bindings;
compatibility-preserving C creation and setup APIs;
a separate Go DummySetup;
access to registered DummyClient buffers;
single-object PutFrom support;
production and test RPC registration;
a multi-process Go DummyClient benchmark.

Explicitly excluded from this proposal:

changes to mooncake-store/benchmarks/stress_cluster_bench.cpp;
CI and workflow changes;
unrelated repository changes.

Validation

The current implementation has been validated by:

building mooncake_store, mooncake_master, and mooncake_client;
building the existing Go examples/basic program;
validating Put, Exists, GetSize, and Get through the Go binding;
completing a DummyClient write run with 2/2 successful operations and no
failures;
completing a DummyClient read run with 2/2 successful operations, no
failures, and no verification errors;
rebuilding both Go examples after automated review fixes;
confirming that the pull request contains only the intended 11 files;
confirming successful benchmark runs return exit status 0;
confirming operation and worker failures return a non-zero exit status;
verifying deliberately corrupted data produces Verify Errors: 1 and
non-zero worker and master exit statuses;
running Rust cargo check --locked against the compatibility-preserving
C API;
running gofmt, targeted pre-commit hooks, and the project C/C++ format
check.

These checks provide small-scale functional validation, not full performance,
high-concurrency, or stress coverage.

Current Status

The implementation PR is ready for review. All 12 existing inline review threads
have been addressed and resolved.

References

Source proposal:
zchuango/Mooncake#29
Implementation:
kvcache-ai/Mooncake#3015

Originally created by @BGQ99 on GitHub (Jul 23, 2026). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/3068 ## Motivation Mooncake provides a C++ `DummyClient` that communicates with a standalone `mooncake_client`, but the Go bindings currently lack the APIs needed to create and configure a `DummyClient`, access the shared-memory buffers prepared during setup, and exercise the single-object `PutFrom` path. This RFC proposes adding the required bindings and a multi-process Go benchmark for validating multiple independent application processes sharing one standalone `mooncake_client`. An implementation is available in [kvcache-ai/Mooncake#3015](https://github.com/kvcache-ai/Mooncake/pull/3015). It ports the relevant work from [zchuango/Mooncake#29](https://github.com/zchuango/Mooncake/pull/29) onto the current Mooncake `main` branch. ## Runtime Architecture The request path is: Go worker -> DummyClient -> RPC -> standalone `mooncake_client` -> RealClient -> Mooncake Store The standalone `mooncake_client` owns the `RealClient` and exposes the required RPC handlers. Each benchmark worker is an independent operating-system process with its own `DummyClient`, address space, and registered shared-memory context. Separate processes are required because goroutines would still share one address space and would not reproduce the intended multi-client shared-memory setup. ## Proposed Changes ### 1. Expose DummyClient through the Go and C bindings Allow the Store bindings to explicitly create either a `RealClient` or a `DummyClient`. The proposed Go API uses: - `New()` to create the default `RealClient`; - `NewWithType(MOONCAKE_CLIENT_DUMMY)` to create a `DummyClient`. The public Go API uses the Go-defined `ClientType` type and does not expose CGo types to callers. ### 2. Preserve the existing Go Setup API Keep the existing seven-argument Go `Setup` method for `RealClient` and add: `DummySetup(memPoolSize, localBufferSize, serverAddress, ipcSocketPath)` Go does not support optional function parameters. Extending the existing `Setup` signature would therefore break existing Go call sites. ### 3. Preserve C API compatibility Keep the existing C creation and setup APIs unchanged for current direct C callers, and add explicit entry points for dummy clients: - keep `mooncake_store_create()` as the default `RealClient` constructor; - add `mooncake_store_create_with_type(client_type)`; - keep the existing `mooncake_store_setup(...)`; - add `mooncake_store_setup_dummy(...)`. The Go wrapper will use the new entry points for `DummyClient`. Existing direct C callers remain source-compatible, while the additional symbols provide the configuration required by the Go binding. Feedback on the exact function names and parameter layout is welcome, but this RFC does not propose changing the signatures of the existing C APIs. ### 4. Provide access to DummyClient registered buffers The benchmark must access the shared-memory buffers prepared during `DummyClient` setup before calling `PutFrom` and `GetInto`. The current implementation exposes operations to: - obtain the number of registered buffers; - query a buffer pointer and size by index; - determine whether a buffer belongs to the hot cache; - unregister the buffers. The current implementation exposes these operations through the public Store binding. ### 5. Complete the single-object PutFrom path `DummyClient` already supports batch `put_from`, while the Go benchmark uses the single-object `PutFrom` API. The current implementation adds a dedicated `RealClient::put_from_dummy_helper` RPC. It: 1. finds the shared-memory context using the client ID; 2. translates the `DummyClient` address into a pointer valid in the standalone `mooncake_client` process; 3. invokes the existing `RealClient` single-object `put_from` implementation. The handler is registered in both the production `mooncake_client` and the test RPC server. The current proposal retains a dedicated single-object RPC. This keeps the single-object API explicit, preserves its direct forwarding path, and reuses the existing `RealClient` single-object implementation. Reusing the batch RPC with a one-object request was considered, but not selected because it would make the single-object API depend on batch semantics internally. ### 6. Add a multi-process Go benchmark The benchmark: - starts independent worker processes with `os/exec`; - creates one `DummyClient` per worker; - distributes keys among workers; - writes deterministic data with `PutFrom`; - reads data with `GetInto`; - verifies the returned contents; - reports failures, verification errors, and throughput. Using separate processes, rather than goroutines, provides independent address spaces, `DummyClient` instances, shared-memory contexts, and CPU affinity. ## Shared-Memory Safety Pointers are process-local virtual addresses. `RealClient` must not directly dereference an address received from `DummyClient`. The RPC request includes the `DummyClient` address, size, device ID, and client ID. `RealClient` uses the corresponding shared-memory context to translate the dummy address into a valid local pointer. Before accessing registered memory, the Go benchmark validates: - conversion to the platform `int` range; - the required size against the registered buffer capacity. Go slices are constructed using the actual accessible buffer length. ## Scope Included: - C and Go `DummyClient` bindings; - compatibility-preserving C creation and setup APIs; - a separate Go `DummySetup`; - access to registered `DummyClient` buffers; - single-object `PutFrom` support; - production and test RPC registration; - a multi-process Go `DummyClient` benchmark. Explicitly excluded from this proposal: - changes to `mooncake-store/benchmarks/stress_cluster_bench.cpp`; - CI and workflow changes; - unrelated repository changes. ## Validation The current implementation has been validated by: - building `mooncake_store`, `mooncake_master`, and `mooncake_client`; - building the existing Go `examples/basic` program; - validating Put, Exists, GetSize, and Get through the Go binding; - completing a DummyClient write run with 2/2 successful operations and no failures; - completing a DummyClient read run with 2/2 successful operations, no failures, and no verification errors; - rebuilding both Go examples after automated review fixes; - confirming that the pull request contains only the intended 11 files; - confirming successful benchmark runs return exit status 0; - confirming operation and worker failures return a non-zero exit status; - verifying deliberately corrupted data produces `Verify Errors: 1` and non-zero worker and master exit statuses; - running Rust `cargo check --locked` against the compatibility-preserving C API; - running `gofmt`, targeted pre-commit hooks, and the project C/C++ format check. These checks provide small-scale functional validation, not full performance, high-concurrency, or stress coverage. ## Current Status The implementation PR is ready for review. All 12 existing inline review threads have been addressed and resolved. ## References - Source proposal: [zchuango/Mooncake#29](https://github.com/zchuango/Mooncake/pull/29) - Implementation: [kvcache-ai/Mooncake#3015](https://github.com/kvcache-ai/Mooncake/pull/3015)

bsakaguchi added the

stale

label

2026-10-29 21:19:32 +00:00

bsakaguchi commented

2026-10-29 21:19:33 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 23, 2026):

Thanks for opening this issue, @BGQ99!

Field	Value
Issue	#3068
GitHub user ID	`300298448`
Reporter	@BGQ99

A maintainer will triage this when possible. To help us respond faster, please include:

Mooncake version or commit SHA
Environment (OS, CUDA/driver, RDMA stack if relevant)
Steps to reproduce and expected vs. actual behavior

Useful links: Documentation · Contributing guide

This message was posted automatically by the issue bot.

@github-actions[bot] commented on GitHub (Jul 23, 2026): Thanks for opening this issue, @BGQ99! | Field | Value | |-------|-------| | **Issue** | #3068 | | **GitHub user ID** | `300298448` | | **Reporter** | @BGQ99 | A maintainer will triage this when possible. To help us respond faster, please include: - Mooncake version or commit SHA - Environment (OS, CUDA/driver, RDMA stack if relevant) - Steps to reproduce and expected vs. actual behavior Useful links: [Documentation](https://kvcache-ai.github.io/Mooncake/) · [Contributing guide](https://github.com/kvcache-ai/Mooncake/blob/main/CONTRIBUTING.md) > This message was posted automatically by the issue bot.

bsakaguchi commented

2026-10-29 21:19:33 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Oct 26, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Oct 26, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi referenced this issue

2026-10-29 21:30:37 +00:00

[PR #787] [CLOSED] Fixnixlbench #1822